fix(voice): clear unplayed realtime messages from chat ctx after interrupt#6450
Open
chakshu-dhannawat wants to merge 1 commit into
Open
Conversation
…rrupt When a realtime response produces multiple messages and the user interrupts after one has played but before a later one is pulled from the message stream, the later message is never played or committed locally, yet it stays in the server-side conversation. On the next turn the model thinks it already said something the user never heard. _realtime_generation already tries to handle this: _process_messages leaves unconsumed messages out of message_outputs so the update_chat_ctx cleanup can drop them server-side. But that cleanup was guarded on any_skipped, which is computed by scanning message_outputs, the very list those messages were left out of. So for the case the comment describes (interrupted before we pulled the message) any_skipped is always False and the sync never runs. Track the abandon condition where it actually happens: set a messages_abandoned flag in _process_messages when we break out on an interrupt or a partial play, and run the cleanup on (any_skipped or messages_abandoned). update_chat_ctx is idempotent, so in the rare case where the interrupted message was the last one this does one harmless extra sync. Added a hermetic regression test that reproduces the interrupt-between-messages race and asserts the never-played message does not survive server-side. It fails before this change and passes after. Fixes livekit#6424
This was referenced Jul 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Fixes #6424. When a realtime response produces multiple messages (e.g. a gpt-realtime preamble plus the final answer) and the user interrupts after message A has finished playing but before message B is pulled from
message_stream, B is never played and never committed to the local chat context, but it stays in the server-side conversation. On the next turn the model believes it already said B and references an answer the user never heard.Root cause
AgentActivity._realtime_generationalready intends to handle this._process_messagesdeliberately leaves unconsumed messages out ofmessage_outputs:But the cleanup is guarded on
any_skipped, which is computed by scanningmessage_outputs— the very list those messages were left out of:So for the exact case the comment describes ("interrupted before we pulled them"),
any_skippedis alwaysFalseandupdate_chat_ctx()never runs. The window whereany_skippedbecomesTrue(a message pulled intomessage_outputs, forwarded, and played zero frames) is very narrow, so in practice the guard is inverted relative to its intent.Fix
Track the abandon condition where it happens instead of inferring it from
message_outputs: set amessages_abandonedflag in_process_messageswhen we break out on an interrupt or a partial play, and run the cleanup on(any_skipped or messages_abandoned).update_chat_ctxis idempotent, so in the rare case where the interrupted message happened to be the last one of the response this performs one harmless extra sync.This matches the direction suggested in the issue (which the reporter verified locally, and a maintainer green-lit).
Testing
Added
tests/test_realtime_interrupt_chat_ctx.py, a hermetic unit test (no network) that reproduces the interrupt-between-messages race with a fake realtime model whose session commits conversation items ingenerate_reply()the way a real server does. It asserts the never-played message does not survive server-side. The test fails before this change and passes after.ruff check,ruff format, andmypy -p livekit.agentsare clean; the existing--unitsuite still passes (the only failures locally are pre-existingtests/test_room.pycases unrelated to this change).