fix: attribute playback evidence to its own segment on shared audio outputs#2055
Closed
chenghao-mou wants to merge 5 commits into
Closed
fix: attribute playback evidence to its own segment on shared audio outputs#2055chenghao-mou wants to merge 5 commits into
chenghao-mou wants to merge 5 commits into
Conversation
…warding completes Combines #1910 (false-interruption pause before first frame, #1909) and #1960 (deferred lk.playback_started from DataStream avatar outputs) into a single change: the PLAYBACK_STARTED listener moves to performAudioForwarding so it outlives the forwarding task, forwardAudio no longer settles firstFrameFut, and all reply paths (pipeline, realtime, and say()) settle a still-pending future via settleFirstFrameFut once the playout window ends. A reported non-zero playback position on interruption is also honored as evidence of partial playback. Co-authored-by: enrique <enrique.espaillat@gydehealth.ai> Co-authored-by: Jared Friedman <jared@ycombinator.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…ment count startedForwardingAt is set before AudioOutput.captureFrame resolves, so a frame that bails at a pause/interrupt gate (e.g. ParticipantAudioOutput while paused) still looked like playback evidence, letting a stale playback position from a previous segment commit a never-heard reply. Record the output's capturedPlayoutSegments when forwarding is set up and only trust a non-zero playback position when the count has since increased — the same condition under which waitForPlayout returns this segment's event rather than a stale one. Suggested by @chenghao-mou in #1966 review. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…audio outputs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 3201100 The changes in this PR will be included in the next version bump. This PR includes changesets to release 35 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
Could you resolve the conflicts from base branch? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1966; hardens its playback-evidence logic with per-segment attribution. Fixes AGT-3148.
Gate: the
playedOwnFramecounter-delta could be satisfied by a straggler frame from the previous interrupted speech (JS can't cancel an in-flightreader.read(), so a late frame bumps the shared counter after the next speech snapshots), committing a never-played reply aspartialwith full-text fallback. The gate now requiresownSegmentIndex, the segment count read after the first of this segment's frames was actually accepted bycaptureFrame. Both mirrored gate sites now sharehasOwnPlaybackEvidence.Listener:
PLAYBACK_STARTEDcarries no segment identity and the listener outlives forwarding (by design, per #1966), so a stale event from an overlapping segment (avatar's latelk.playback_startedRPC, or the next speech starting during this one's teardown) could resolve the wrong segment'sfirstFrameFut: spuriousspeakingstate, foreign timestamps in metrics, unheard text committed. The listener now only honors an event while the output's segment counter still points at its own segment, failing closed on ambiguity (genuine partial playback still commits via the position evidence).Known residual: a stale avatar RPC arriving while the counter still points at this segment is locally indistinguishable from the segment's own event; the full fix needs a segment id in the RPC protocol. Python counterpart: livekit/agents#6470.
🤖 Generated with Claude Code