Repository navigation
Conversation
quality_probe feeds raw BGRA frames through a CompressionSession with the production settings, at the presentation times a script gives, and writes the stream (Annex B) and a CSV of which frames came out, how big, and which were keyframes. Decoding the stream with ffmpeg and comparing it with the source frames measures picture quality without a viewer or a network.
The 20 s interval was a duration, and a still desktop sends no frames, so the first frame after any pause of 20 s or more was a forced keyframe. Measured with quality_probe at 2 Mbps, 1080p60 HEVC, on a text-heavy desktop, scrolling again after a still page: pause first frame next 9 frames 1 s P, 30 KB, 40.4 dB 38.0-40.5 dB 25 s IDR, 124 KB, 33.8 dB 28.2-33.9 dB (before) 25 s P, 30 KB, 40.4 dB 38.0-40.5 dB (after) 124 KB is half a second of link time at 2 Mbps, on the first input after reading a page. The interval is now 20 s worth of frames at the session's rate, which only sustained motion reaches: at 10 fps the safety keyframe still lands on frame 200.
EncodePipeline::last_frame_bytes, stored before the output count moves, so whoever sees a new frame counted can read how big it came out.
A `pass` line is a re-encode sent the way the host's refiner sends one: held back a spacing at a time while a link of the session's bitrate would still have more than 10 ms queued. The encoder budgets a frame by the time since the previous one, so passes that wait on the link come out larger; scripts with fixed timestamps understated them. A trailing `@<kbps>` changes the encoder's bitrate before a frame, leaving the simulated link alone.
…g detail The refiner stopped after two passes came out of the encoder. Each pass codes what the last one left out, so the picture keeps sharpening well past that. Measured with quality_probe on a still 1080p text page after a scroll, passes 100 ms apart and held back while the simulated link has more than 10 ms queued, as the refiner does (luma PSNR against source): codec link 2 passes converged passes done after stop HEVC 2 Mbps 38.2 dB 47.4 dB 9 1.4 s HEVC 1.5 Mbps 34.5 dB 47.2 dB 17 2.3 s H.264 2 Mbps 36.9 dB 47.9 dB 18 2.4 s Passes now continue until two in a row each come out under 20 ms of link time at the current bitrate, or 24 have come out. 50 ms or 33 ms spacing made smaller passes that tripped that rule early (43.5-45 dB), so the spacing stays at 100 ms. The cost is latency if the screen moves again mid-refinement: the next frame waits for the pass in flight, and later passes are larger than the first two (largest 206-366 ms of link against 83-151 ms). Running them at a lower bitrate does not help: at a quarter of the bitrate single passes still reached 32-65 KB, convergence just took 2-3 times longer.
… has been still a second Refining to convergence put larger passes on the link in the second or two after the screen stops, and a frame of motion that comes along meanwhile waits behind the pass in flight: measured at 1.5-2 Mbps, the largest took 206-366 ms of link against 83-151 ms for the first two. The first two passes still run as soon as the screen stops, as before; the rest wait until it has been still for a second. A pause in the middle of scrolling gets exactly the old behaviour, and a page being read reaches full detail about a second later than it otherwise would. If the two quick passes already settle (a keystroke), nothing waits.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two changes to what the encoder sends at the same bitrate, found and measured with a new offline probe that runs real desktop frames through the production VideoToolbox settings.
Evidence
All numbers:
quality_probe, 1080p60 text-heavy desktop (sidebar, syntax-coloured code, document pane), stream decoded with ffmpeg, luma PSNR against the source frame.First scroll after reading a page (HEVC, 2 Mbps)
After: pause 25 s → first frame P, 30 KB, 40.4 dB; next 9 frames 38.0-40.5 dB (same as a 1 s pause)
Still page after a scroll (passes 100 ms apart, held while the simulated link has >10 ms queued)
Unit test
test_passes_run_until_the_picture_convergesreplays the measured 2 Mbps pass sizes: stops at pass 13, and the third pass waits for 1 s of stillness.Tried and rejected (numbers in the commit bodies):
Loopback (extend mode, 2 Mbps): streams, refines (
Still screen refined passes=2on an idle virtual display), no errors.cargo nextest run --workspace: 358 passed. Clippy clean.Not run over a real slow link to the Windows viewer.
Merge Danger
Door: two-way
Encoder settings and refiner timing only; no protocol, config or viewer change. Reverting restores the old behaviour exactly.
Blast Radius: host-encoding