Skip to content

DIS: run the variational refinement on the whole pyramid - #748

Merged
maxjivi05 merged 2 commits into
WinNative-Emu:mainfrom
qwertypower:dis-vr-pyramid
Sep 12, 2026
Merged

DIS: run the variational refinement on the whole pyramid#748
maxjivi05 merged 2 commits into
WinNative-Emu:mainfrom
qwertypower:dis-vr-pyramid

Conversation

@qwertypower

Copy link
Copy Markdown
Contributor

The variational refinement ran once, at the finest level, after the coarse-to-fine loop had already finished. OpenCV's DIS runs it inside that loop, at every level, and the difference is not cosmetic: every level was handing the next finer one a raw densified patch flow as its initialisation instead of a refined, coherent one, so patch-search errors compounded all the way down the pyramid and the single pass at the end had to undo the lot at the most expensive resolution.

Each refinement intermediate becomes a mip chain with a view per level, so the VR shaders keep deriving their extent from imageSize() and are unchanged. The dispatch block moves into the level loop as dis_vr_level(), and the patch search now initialises from view_flow_refined[l + 1] rather than view_dense[l + 1].

On a level that is not getting the full solve, prep and add still run: prep zeroes dW as a side effect, so add stores flow_refined = dense exactly. Two dispatches and seven taps a texel keep the descriptor wiring static - flow_refined is always the valid flow for its level - which is much cheaper than switching descriptor sets between refined and raw sources per frame.

How many levels get the full solve is a tier on the generation count, beside the iteration counts already there. The flow chain runs once per SOURCE frame, so at x4 the guest sits at a quarter of the panel rate and the chain runs a quarter as often: half the load per second of the x2 case, which is the heaviest the system ever gets. x2 therefore keeps the refinement to the finest level and is bit-identical to before; x3 and x4 spend the headroom that is already there on the whole pyramid, and that is exactly where the flow is worst because the motion between two source frames is twice as large.

Modelled per source frame at the Balance preset (448x252), against today's x2 load as the ceiling:

x2   30 fps guest   33.0M taps   +0.6%   0.99G/s   101% of ceiling
x3   20 fps guest   45.3M taps  +11.8%   0.91G/s    92% of ceiling
x4   15 fps guest   48.2M taps  +12.9%   0.72G/s    74% of ceiling

Descriptor pool: the VR sets now exist once per level, 10 per level instead of 10 total. Pool and demand are both derived from the same constants and match exactly at 227 sets / 1375 samplers / 307 storage images (was 157 / 815 / 167). An undersized pool here fails with OUT_OF_POOL_MEMORY and frame generation silently stops existing, which is why it is computed rather than written out.

Memory cost is the extra mips on nine scratch images: +33% of their level-0 footprint, about 3.4 MB at the Balance preset.

Also points the DEVAR credit in the README at https://devar.ai/.

Claude-Session: https://claude.ai/code/session_01MCkAQJH8ik5iJZqf2NX3w6

maxjivi05 and others added 2 commits September 12, 2026 16:40
…Native-Emu#750)

Restores the C optical-flow chain (vkr_dis.c and its shader set) so the
per-pyramid-level variational refinement in WinNative-Emu#748 is the DIS work that is kept.

This reverts commit 4e95335.
The variational refinement ran once, at the finest level, after the
coarse-to-fine loop had already finished. OpenCV's DIS runs it inside that
loop, at every level, and the difference is not cosmetic: every level was
handing the next finer one a raw densified patch flow as its initialisation
instead of a refined, coherent one, so patch-search errors compounded all the
way down the pyramid and the single pass at the end had to undo the lot at the
most expensive resolution.

Each refinement intermediate becomes a mip chain with a view per level, so the
VR shaders keep deriving their extent from imageSize() and are unchanged. The
dispatch block moves into the level loop as dis_vr_level(), and the patch
search now initialises from view_flow_refined[l + 1] rather than
view_dense[l + 1].

On a level that is not getting the full solve, prep and add still run: prep
zeroes dW as a side effect, so add stores flow_refined = dense exactly. Two
dispatches and seven taps a texel keep the descriptor wiring static -
flow_refined is always the valid flow for its level - which is much cheaper
than switching descriptor sets between refined and raw sources per frame.

How many levels get the full solve is a tier on the generation count, beside
the iteration counts already there. The flow chain runs once per SOURCE frame,
so at x4 the guest sits at a quarter of the panel rate and the chain runs a
quarter as often: half the load per second of the x2 case, which is the
heaviest the system ever gets. x2 therefore keeps the refinement to the finest
level and is bit-identical to before; x3 and x4 spend the headroom that is
already there on the whole pyramid, and that is exactly where the flow is worst
because the motion between two source frames is twice as large.

Modelled per source frame at the Balance preset (448x252), against today's x2
load as the ceiling:

    x2   30 fps guest   33.0M taps   +0.6%   0.99G/s   101% of ceiling
    x3   20 fps guest   45.3M taps  +11.8%   0.91G/s    92% of ceiling
    x4   15 fps guest   48.2M taps  +12.9%   0.72G/s    74% of ceiling

Descriptor pool: the VR sets now exist once per level, 10 per level instead of
10 total. Pool and demand are both derived from the same constants and match
exactly at 227 sets / 1375 samplers / 307 storage images (was 157 / 815 / 167).
An undersized pool here fails with OUT_OF_POOL_MEMORY and frame generation
silently stops existing, which is why it is computed rather than written out.

Memory cost is the extra mips on nine scratch images: +33% of their level-0
footprint, about 3.4 MB at the Balance preset.

Also points the DEVAR credit in the README at https://devar.ai/.
@maxjivi05
maxjivi05 merged commit 30b49f2 into WinNative-Emu:main Sep 12, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants