DIS: run the variational refinement on the whole pyramid - #748
Merged
Conversation
…Native-Emu#750) Restores the C optical-flow chain (vkr_dis.c and its shader set) so the per-pyramid-level variational refinement in WinNative-Emu#748 is the DIS work that is kept. This reverts commit 4e95335.
The variational refinement ran once, at the finest level, after the
coarse-to-fine loop had already finished. OpenCV's DIS runs it inside that
loop, at every level, and the difference is not cosmetic: every level was
handing the next finer one a raw densified patch flow as its initialisation
instead of a refined, coherent one, so patch-search errors compounded all the
way down the pyramid and the single pass at the end had to undo the lot at the
most expensive resolution.
Each refinement intermediate becomes a mip chain with a view per level, so the
VR shaders keep deriving their extent from imageSize() and are unchanged. The
dispatch block moves into the level loop as dis_vr_level(), and the patch
search now initialises from view_flow_refined[l + 1] rather than
view_dense[l + 1].
On a level that is not getting the full solve, prep and add still run: prep
zeroes dW as a side effect, so add stores flow_refined = dense exactly. Two
dispatches and seven taps a texel keep the descriptor wiring static -
flow_refined is always the valid flow for its level - which is much cheaper
than switching descriptor sets between refined and raw sources per frame.
How many levels get the full solve is a tier on the generation count, beside
the iteration counts already there. The flow chain runs once per SOURCE frame,
so at x4 the guest sits at a quarter of the panel rate and the chain runs a
quarter as often: half the load per second of the x2 case, which is the
heaviest the system ever gets. x2 therefore keeps the refinement to the finest
level and is bit-identical to before; x3 and x4 spend the headroom that is
already there on the whole pyramid, and that is exactly where the flow is worst
because the motion between two source frames is twice as large.
Modelled per source frame at the Balance preset (448x252), against today's x2
load as the ceiling:
x2 30 fps guest 33.0M taps +0.6% 0.99G/s 101% of ceiling
x3 20 fps guest 45.3M taps +11.8% 0.91G/s 92% of ceiling
x4 15 fps guest 48.2M taps +12.9% 0.72G/s 74% of ceiling
Descriptor pool: the VR sets now exist once per level, 10 per level instead of
10 total. Pool and demand are both derived from the same constants and match
exactly at 227 sets / 1375 samplers / 307 storage images (was 157 / 815 / 167).
An undersized pool here fails with OUT_OF_POOL_MEMORY and frame generation
silently stops existing, which is why it is computed rather than written out.
Memory cost is the extra mips on nine scratch images: +33% of their level-0
footprint, about 3.4 MB at the Balance preset.
Also points the DEVAR credit in the README at https://devar.ai/.
maxjivi05
force-pushed
the
dis-vr-pyramid
branch
from
September 12, 2026 20:42
8ff2495 to
83a2378
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The variational refinement ran once, at the finest level, after the coarse-to-fine loop had already finished. OpenCV's DIS runs it inside that loop, at every level, and the difference is not cosmetic: every level was handing the next finer one a raw densified patch flow as its initialisation instead of a refined, coherent one, so patch-search errors compounded all the way down the pyramid and the single pass at the end had to undo the lot at the most expensive resolution.
Each refinement intermediate becomes a mip chain with a view per level, so the VR shaders keep deriving their extent from imageSize() and are unchanged. The dispatch block moves into the level loop as dis_vr_level(), and the patch search now initialises from view_flow_refined[l + 1] rather than view_dense[l + 1].
On a level that is not getting the full solve, prep and add still run: prep zeroes dW as a side effect, so add stores flow_refined = dense exactly. Two dispatches and seven taps a texel keep the descriptor wiring static - flow_refined is always the valid flow for its level - which is much cheaper than switching descriptor sets between refined and raw sources per frame.
How many levels get the full solve is a tier on the generation count, beside the iteration counts already there. The flow chain runs once per SOURCE frame, so at x4 the guest sits at a quarter of the panel rate and the chain runs a quarter as often: half the load per second of the x2 case, which is the heaviest the system ever gets. x2 therefore keeps the refinement to the finest level and is bit-identical to before; x3 and x4 spend the headroom that is already there on the whole pyramid, and that is exactly where the flow is worst because the motion between two source frames is twice as large.
Modelled per source frame at the Balance preset (448x252), against today's x2 load as the ceiling:
Descriptor pool: the VR sets now exist once per level, 10 per level instead of 10 total. Pool and demand are both derived from the same constants and match exactly at 227 sets / 1375 samplers / 307 storage images (was 157 / 815 / 167). An undersized pool here fails with OUT_OF_POOL_MEMORY and frame generation silently stops existing, which is why it is computed rather than written out.
Memory cost is the extra mips on nine scratch images: +33% of their level-0 footprint, about 3.4 MB at the Balance preset.
Also points the DEVAR credit in the README at https://devar.ai/.
Claude-Session: https://claude.ai/code/session_01MCkAQJH8ik5iJZqf2NX3w6