fix(rendering): stop exported camera frames running ahead on long recordings

[richiemcilroy]

Exports and editor playback can show the wrong camera or screen frame. On a 180 s Studio recording with a 720p camera that delivered ~15 fps with jitter, the 1080p60 export served a camera frame from the future for 9,023 of 10,799 output frames, up to 117 s ahead. A real 9 min recording served the wrong screen frame for 23,884 of 32,402 output frames. Separately, the camera bubble could differ between two exports of the same project, because the GPU sometimes sampled a decoder buffer that had already been recycled.

The correct answer to a frame request is the newest source frame at or before the requested time. Every change below makes the macOS decoder (crates/rendering/src/decoder/avassetreader.rs) give that answer, or seek again until it can.

1. Pts holes are kept

record_pts_hole (crates/rendering/src/decoder/mod.rs) records gaps in a track's timestamps, so a request inside a gap is answered with the frame before it without decoding. It kept 64 gaps and evicted the narrowest, which meant every new 2-frame gap was dropped as soon as it was inserted.

• The map now holds up to 16,384 gaps (a 9 min recording has ~8k). Past that it drops the gap farthest from the newest one, since requests trail the reader.
• Both the AVAssetReader and FFmpeg decoders use it.

2. A pooled reader only answers what it can prove

The decoder keeps a pool of readers. Each one vends samples in pts order, so the frame it vended just before a jump is the answer for every request inside that jump. Three ways to break that are now closed:

sequenceDiagram
  participant R as Request
  participant D as Decoder
  participant P as Pooled reader
  R->>D: frame 1054 (not cached)
  D->>P: pool position 1053 (capped), but the reader already vended 1054
  alt reader passed the request
    D->>P: seek back first (was: read on and answer 1052, stale)
  else reader steps over the request
    P-->>D: vends 1057; frame vended before it = 1054
    D-->>R: 1054 if still held, otherwise retry once from a fresh seek
  end

Loading

• Passed readers seek. Pool positions are capped to the newest request, so a reader that decoded ahead looked usable for a frame it had already passed and evicted. It now resets first (reader_passed_request).
• Answers are checked against the previous vend. When the reader steps over a request, the answer must be the frame it vended just before. If that frame is no longer held anywhere, the request is deferred and retried once with a forced seek, which re-vends and keeps it (answer_misses_previous_vend).

if !req.retried
    && answer_misses_previous_vend(vended_before, req.frame, best_before.as_ref().map(|d| d.number))
{
    retry_requests.push(PendingRequest { retried: true, ..req });
    continue;
}

• Each reader's last frame stays cached. Transitions read two positions of one recording through the same decoder. A reset purged the other position's frames, and inserts evicted the other reader's newest frame, so both readers kept losing the frame they needed next. The last frame vended by every other pooled reader is now pinned against eviction and the reset purge (cache_eviction_victim).
• Tracks that start late. A track whose first sample is after 0 s answers earlier requests with its first frame straight from the cache, so the reader does not step ahead or reset on each one.

3. IOSurface sources held until the GPU reads them

convert_nv12_from_iosurface_to_encoder (crates/rendering/src/yuv_converter.rs) wraps the decoded IOSurface in Metal textures without retaining the CVPixelBuffer, and the owning DecodedFrame could be dropped before the command buffer ran, letting VideoToolbox decode the next picture into the same buffer. The converter now keeps its last 4 source buffers in a ring (one pipelined frame plus a transition's two conversions). Nothing is copied.

Results

Every decoder answer was logged and checked against the file's packet timestamps, at 960x540 cap export, for 30 and 60 fps output on both decoders (AVF = AVAssetReader, FF = --force-ffmpeg-decoder). "Wrong where main right" counts answers this PR gets wrong that main gets right.

Project	Output	main wrong	This PR wrong	Wrong where main right
180 s Studio, camera, AVF / FF	60 fps	9,023 / 2,314 of 10,799	0 / 0	0
9 min Studio, screen, AVF / FF	60 fps	23,884 / 15,817 of 32,402	0 / 0	0
9 min Studio, screen, AVF / FF	30 fps	12,548 / 5,147 of 16,201	0 / 0	0
9 min, timeline jumping back 5 times, screen, AVF / FF	60 fps	4,651 / 4,733 of 12,300	0 / 180	0
180 s, 8 clips with 9 s transitions, camera, AVF / FF	60 fps	6,993 / 202 of 9,600	0 / 23	0
same, screen, AVF / FF	60 fps	5,562 / 28 of 9,600	0 / 28	0
180 s with cuts, camera, AVF / FF	60 fps	8,004 / 1,780 of 9,600	0 / 0	0
Synthetic: 1,440 gaps up to 10 s, timeline jumping back into them, screen, AVF	60 / 30 fps	1,622 / 0	0 / 0	0
Synthetic: camera starting at 3 s, AVF	60 fps	3,354 of 3,600	0	0
60 s Studio, scroll 30 s, camera blur project	30 + 60 fps	0	0	0

• 117 project/fps/decoder/stream combinations (721k answers) were checked, including synthetic camera tracks with long gaps, a short camera and a late camera. "Wrong where main right" is 0 in every one, and the remaining FFmpeg misses are a subset of main's.
• The generated-fixture oracle suite on ci/engine-verify (export, playback, seek; 2 runs): every case main passes also passes here. This PR additionally passes export/jitter-camera-40s/full/mp460/avf, export/backward-hold/backward-jumps/mp460/avf and seek/seek-gop/{keyframes30,storm30}/avf, which fail on main.
• Editor playback (playback-hash: sequential, 150 random seeks, drag back/forward, and play-jump-back lists on the 180 s, 9 min, synthetic and static-screen projects, both decoders): 0 answers wrong where main is right, and main's 47 wrong seek answers on the 9 min recording are now 0.

Export output, 1080p60	main	This PR
60 s Studio (AVF, FF), scroll 30 s (AVF, FF, 4K), camera blur project	reference	bit-identical video and audio
180 s Studio, 3 runs AVF + 1 run FF	runs differ in 802 to 1,964 frames	all 4 runs bit-identical, AVF = FF
Instructions retired, 180 s camera export, AVF	243.3e9	225.3e9 (−7%)
Instructions retired, 9 min export, AVF / FF	707.9e9 / 1,023.4e9	647.2e9 / 837.0e9
Peak footprint	unchanged	unchanged

Editor seek latency, plain builds, 4 interleaved runs	main avg (ms)	This PR avg (ms)
Drag back, 180 s	232 / 133 / 106 / 106	165 / 112 / 104 / 110
Drag back, 9 min	224 / 147 / 96 / 117	120 / 129 / 95 / 103
Play and jump lists, synthetic gaps	5.8 / 5.0 / 3.3 / 3.4	3.8 / 3.4 / 3.4 / 3.5
Static-screen recording	4.5 / 4.1 / 3.4 / 3.4	4.7 / 3.5 / 3.5 / 3.5

Confidence Score: 5/5

The PR appears safe to merge; no new blocking issue was found.

Summary

The PR fixes camera timing on long recordings and keeps decoded macOS buffers alive while the GPU reads them.

• Camera exports keep the frame that belongs at each requested time.
• GPU conversions keep their IOSurface source buffers alive while sampling them.

Review used source inspection; no runtime checks were performed.

_{Reviews (3) · Last reviewed commit: "fix(rendering): answer requests before a..." · Reviewed by Greptile}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
cap-status	Ready	Preview	Oct 15, 2026 7:38am UTC

[capy-ai]

Capy found no issues.

View 2 other findings in Capy.

[richiemcilroy]

hey @greptileai, please re-review the PR

[capy-ai]

Capy found no issues.

View 1 other finding in Capy.

[richiemcilroy]

@greptileai review

[capy-ai]

Capy found no issues.

View 1 other finding in Capy.

[richiemcilroy]

Superseded by #2638 (merged), which fixes the same decoder request handling with the suite's oracle proof.