perf(editor): halve seek latency and stop seeks showing frames from the wrong GOP

[richiemcilroy]

Seeking in the macOS editor took 40-300 ms to show a frame, and could show a different frame than playback or export would show at that time. There were two causes, both in the AVAssetReader decode path.

Stacked on #2595 (fix/export-camera-frame-runaway). The reset rule here is decided per request from the decoder's cache and its pts-hole map. #2595 changes how that hole map is kept when it fills up and how long IOSurface sources are held, and the broad reset below is only correct with #2595's hole map underneath. On main alone, camera playback of a 180 s recording also drifts ahead after about 35 s (fixed by #2595), so seek answers cannot be scored against main's own sequential render. Merge #2595 first; this PR then retargets to main with no conflicts.

Readers started just before the keyframe

compute_seek_time picks the keyframe at or before the request, but the reader was then created with cm::Time::with_secs(time, 100), which rounds to 10 ms. About half the time that lands a fraction of a tick before the keyframe. AVAssetReader then starts decoding at the previous sync sample, decodes and discards that whole GOP (~25 frames at 3K), and vends its last frame retimed to the start. The preview could therefore show a frame from the previous GOP under the keyframe's frame number.

// crates/video-decode/src/avassetreader.rs
fn exact_time(&self, secs: f64) -> cm::Time {
    let ticks = (secs * f64::from(denominator) / numerator as f64).round() as i64;
    cm::Time::new(ticks * numerator, denominator) // the keyframe's own stream tick
}

• KeyframeIndex keeps the stream time_base, and reader_start_time starts every reader on the keyframe's exact tick.

A reused decoder could answer from an unrelated seek

After scattered seeks, the decoder chosen for a request could answer with the newest cached frame at or before it, even when that frame was left over from an earlier, unrelated seek. In one run a request for frame 1899 showed frame 948.

flowchart LR
  Q[request frame N] --> S[select_best_decoder]
  S --> C{newest cached frame <= N<br/>was vended in this reader's<br/>current run?}
  C -- yes --> A[answer from cache / keep decoding]
  C -- no, or nothing cached <= N --> R[reset the reader at N's keyframe]

Loading

• DecoderInstance tracks run_start, the first frame of its current uninterrupted vend run (crates/rendering/src/decoder/avassetreader.rs).
• reuse_would_serve_stale_frame forces a reset when the newest cached frame at or before the request predates that run, or when nothing at or before the request is cached at all. The second case covers returning into a pts hole whose held frame was evicted by a distant seek: the reader has already passed the request, so reusing it could only answer with a later frame.

Benchmark

Seek to rendered frame through FrameRenderer (decode + render at 1248x702), M4 Max, #2595 head (905e093) vs this PR on top of it. Warm-seek p50, same harness, both builds run back to back on a machine shared with other builds, so treat single numbers as ±20%:

Warm seek p50	#2595	This PR
studio-cam-60s, golden-ratio seeks (2 runs)	72 / 84 ms	40 / 53 ms
studio-cam-60s, 836 shuffled seeks next to keyframes	77 ms	49 ms
studio-cam-60s, 3,000 random seeks	75 ms	53 ms
studio-cam-180s, golden-ratio seeks (2 runs)	305 / 237 ms	124 / 158 ms
studio-cam-180s, 2,709 shuffled seeks next to keyframes	283 ms	124 ms
studio-cam-180s, 3,000 random seeks	107 ms	61 ms
scroll-30s, 1,047 shuffled seeks next to keyframes	96 ms	54 ms

Frames rendered right after a seek are unchanged (p50 3.8-4.9 ms on both).

Seek answers that differ from sequential playback	#2595	This PR
Golden-ratio seeks, 60s / 180s (240 each)	5 / 4	0 / 0
Shuffled seeks next to keyframes, 60s / 180s	70 / 190	0 / 0
3,000 random seeks, 60s / 180s	81 / 45	0 / 3
Random walks (drag back and forth), 60s / 180s	61 / 43	4 / 14

• On the 60s and 180s fixtures no seek answer is wrong in this PR that fix(rendering): stop exported camera frames running ahead on long recordings #2595 answers correctly; the remaining mismatches are frames fix(rendering): stop exported camera frames running ahead on long recordings #2595 also gets wrong.
• Sequential playback output is byte-identical to fix(rendering): stop exported camera frames running ahead on long recordings #2595 on studio-cam-60s (with and without camera), studio-cam-180s, scroll-30s, a 9 min recording, cut and transition timelines, and two camera-gap fixtures, including export's cache mode and exports starting at 95 s and 300 s. The FFmpeg decode path renders identically on both builds.
• Playback started mid-file (17 s into the 60s fixture, and on two camera fixtures) now renders exactly what playback from zero renders at the same frames. fix(rendering): stop exported camera frames running ahead on long recordings #2595 differed there on 2, 1 and 570 frames.
• On two synthetic camera fixtures with long pts holes (1,500-seek random walks, camera region only, 4 runs per build), this PR answers 3-14 seeks differently from sequential playback against 21-42 on fix(rendering): stop exported camera frames running ahead on long recordings #2595. Which frames miss varies run to run on both builds; most of this PR's misses are frames fix(rendering): stop exported camera frames running ahead on long recordings #2595 also misses, and a few (1-2 per run) land next to frames fix(rendering): stop exported camera frames running ahead on long recordings #2595 misses in the same hole.
• Recordings whose camera file starts later than its first requested frame (a synthetic fixture with camera pts starting at 3 s; Cap's recorder writes camera files starting at 0) now hold the camera's first frame until it starts. Before, the camera ran ahead at double speed for about 15 s there. Those pre-start requests now reset the camera reader, costing about 25% more instructions in that window.
• Composes with perf(rendering): convert IOSurface frames straight into the layer frame texture #2601: fix(rendering): stop exported camera frames running ahead on long recordings #2595 + perf(rendering): convert IOSurface frames straight into the layer frame texture #2601 + this PR passes the cap-rendering and cap-video-decode tests, renders sequential playback byte-identically, and answers the shuffled and random seek sets on the 60s and 180s fixtures with no wrong frames.

Confidence Score: 5/5

This PR appears safe to merge after #2595.

Summary

This PR starts macOS readers on exact keyframe ticks and resets reused readers when their cached earlier frame is missing or predates the current run.

• AVAssetReader seeks start on the chosen keyframe’s exact tick.
• Seek requests reset pooled readers when their cache belongs to another run.

_{Reviews (2) · Last reviewed commit: "fix(rendering): reset a reused decoder w..." · Reviewed by Greptile}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
cap-status	Ready	Preview	Oct 15, 2026 3:54am UTC

[capy-ai]

Capy found no issues.

View 1 other finding in Capy.

[richiemcilroy]

@greptileai review

[capy-ai]

Capy found no issues.

View 1 other finding in Capy.

[richiemcilroy]

Superseded by #2638 (merged), which fixes the same decoder request handling with the suite's oracle proof.