Skip to content

fix(recording): align mixed Instant audio sources to their capture times - #2647

Merged
richiemcilroy merged 20 commits into
mainfrom
fix/instant-mixer-input-alignment
Oct 16, 2026
Merged

richiemcilroy merged 20 commits into
mainfrom
fix/instant-mixer-input-alignment

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Instant recordings that mix the microphone with system audio could place system audio up to 60 ms before the microphone. Which offset a recording got depended on when the mixer thread first polled.

Cause

sequenceDiagram
    participant Mic
    participant System
    participant Mixer
    participant amix
    Mic->>Mixer: first sample captured at 200 ms
    System->>Mixer: first sample captured at 260 ms
    Note over Mixer: inputs reach amix without pts
    Mixer->>amix: mic samples become sample 0
    Mixer->>amix: system samples also become sample 0
    Note over amix: system is now 60 ms early (2,880 samples)
Loading
  • Each input's first sample became its own zero inside amix, so the gap between the two sources' first samples was lost.
  • A startup branch sometimes prepended one 2,560-sample silence block to a source that had not delivered yet. That left two modes, -60 ms or -6.67 ms, depending on the poll schedule.
  • Output is stamped start + samples_out / rate, so the offset stayed for the rest of the recording.

Fix

Every mixer input now keeps a sample cursor on one shared timeline, which starts at the earliest captured sample (crates/recording/src/sources/audio_mixer.rs).

let correction = placement_correction(
    position - source.samples_supplied as i128, // capture time vs cursor, in samples
    source.info.sample_rate,
    buffer.inner.samples(),
);
Capture time vs the source's cursor Action
within 2 ms continue the source untouched (timestamp jitter)
2 to 10 ms move the cursor at most 1 sample per 256 (clock drift)
10 ms or more, later fill the gap with silence (late first packet, dropout, pause)
10 ms or more, earlier drop the input whose time is already committed, and count it
  • Watermark. Mixed audio is committed only up to the earliest end among inputs that have delivered. A source that has not delivered yet is waited for up to 1 s (STARTUP_GRACE). After that it is filled with silence, and its first packet lands at its capture time.
  • Stop. All mixed sources stop together, then the mixer drains every input and flushes amix, so audio still queued in the mixer reaches the file. The drain keeps only audio captured before Stop. A mixer that has not closed 2 s later, or an encoder that stops draining it, costs the audio not yet delivered, never the recording.
  • Pause. The mixer now delays its output, so audio reaching the pipeline after Pause can still be from before it. Mixed audio is now paused by capture time: SharedWallClockPause records each pause interval, and frames that span a pause boundary are split (crates/recording/src/output_pipeline/core.rs). Recordings with one audio source keep the wall-clock pause path.

Measured

Instant system-minus-mic offset, synthetic sources with known capture times:

main This PR
System minus mic -60 ms or -6.7 ms, depending on the poll schedule 0.0 ms in every run (macOS 20, Windows 20, Linux 3)
  • Both sources now share the microphone's existing whole-track offset, about 15 ms on Linux and inside the existing 5 to 34 ms bound. That offset comes from anchoring the mixed track (AudioAnchor::FirstFrame) and is unchanged here.
  • Recordings with one audio source do not go through the mixer and are unchanged.
  • On the built-in MacBook Pro microphone, capture timestamps showed no jitter and drifted 8 samples in 30 s (6 ppm), so the drift path moves the cursor about one sample every 4 s.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no blocking findings remain.

Summary

The PR aligns mixed audio sources by capture time and preserves queued audio during Stop.

  • Mixed microphone and system audio now line up by capture time.
  • Stopping a mixed recording keeps audio captured before Stop.
  • Mixed audio pauses now follow when each sample was captured.

All four earlier, unnumbered findings are addressed. No new actionable issues were found.

Reviews (6) · Last reviewed commit: "fix: read the pause flag under the pause..." · Reviewed by Greptile

Mixer inputs were placed at the exact sample of every capture timestamp.
Receipt-time stamps (Linux) jitter by milliseconds, so each late callback
became inserted silence followed by dropped input. Packets within 2 ms of
the source's sample cursor now continue it untouched; between 2 and 10 ms
the cursor moves at most one sample per 256 (clock drift); from 10 ms a
packet starts at its capture time as before.
A mixed audio source that took over 1 s to stop, or a mixer that had not
closed 2 s later, failed the recording. Bluetooth inputs can take that
long to tear down. Both now record a cleanup error and keep the audio
drained so far, as a failed source stop already did.
@vercel

vercel Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
cap-status Ready Ready Preview Oct 16, 2026 5:03pm UTC

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found 2 potential issues (2 medium).

Open in Capy

Comment thread crates/recording/src/output_pipeline/core.rs Outdated
Comment thread crates/recording/src/output_pipeline/core.rs Outdated
Comment thread crates/recording/src/sources/audio_mixer.rs Outdated
…ling

An encoder that stopped draining the mixer's output past the stop deadline
returned an error from finish(), which failed the recording. The mixer
now stops delivering and keeps the audio already sent.
@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review

Sources were stopped one after another, so a microphone kept capturing
while system audio tore down, and that audio after Stop reached the file
through the mixer drain. All sources now stop together, the drain keeps
only audio captured before Stop, and the source-stop and drain timeouts
no longer record cleanup errors (Instant treats any cleanup error as a
failed recording); the mixer closes its inputs at its own deadline.

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review this PR

Comment thread crates/recording/src/output_pipeline/core.rs Outdated

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@blacksmith-sh

This comment has been minimized.

… time

The source clock remap clamps a packet captured before the master clock's
origin to the origin. The first real-capture microphone packet started
3.6 ms early, so it was stamped at the origin with all 480 samples and
overlapped the next packet by 174 samples; the mixer then moved the
microphone back over two seconds. The mixer now drops only the samples
captured before the origin.
With mixer inputs aligned, system audio sits on the microphone's timeline
(system minus mic 0 ms) and shares its gated offset, replacing the -60 ms
and -6.7 ms mixer-mode entries.
The report test pinned the two removed mixer-mode entries; it now checks
the single strict entry accepts system audio within 1 ms of the mic and
rejects both old modes.
A source whose device teardown hangs held the pipeline stop forever.
The stop now waits at most MIXER_STOP_TIMEOUT, logs, and drains what the
mixer has; it records no cleanup error, which Instant treats as failure.
@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review

Comment thread crates/recording/src/output_pipeline/core.rs

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found 1 potential issue (1 medium).

Open in Capy

Comment thread crates/recording/src/output_pipeline/core.rs
…ing their stop

Pause and Resume now advance the wall-clock pause total at their own
instants, so video removes the same pause time as mixed audio even when a
static screen sends no frame through check() during the pause. A mixed
source that misses the stop deadline is dropped without its cleanup
scope, so the stop that hung is not spawned again as scope cleanup.
@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review

Comment thread crates/recording/src/output_pipeline/core.rs

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found 1 potential issue (1 medium).

View 1 other finding in Capy.

Open in Capy

Comment thread crates/recording/src/output_pipeline/core.rs
check() read the flag before taking the lock set_paused_at holds while it
records a pause, so a frame holding a stale flag could end a pause that
had just started and lose its time from the video total.

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review

@richiemcilroy
richiemcilroy merged commit e347a5a into main Oct 16, 2026
54 checks passed

This branch was successfully deployed

2 active deployments
Preview – cap-web — 5569b930 Deployed Oct 16, 2026 by vercel[bot]
Preview – cap-status — 5569b930 Deployed Oct 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant