Skip to content

fix(voice): clear unplayed realtime messages from chat ctx after interrupt#6450

Open
chakshu-dhannawat wants to merge 1 commit into
livekit:mainfrom
chakshu-dhannawat:fix/6424-realtime-interrupt-chat-ctx
Open

fix(voice): clear unplayed realtime messages from chat ctx after interrupt#6450
chakshu-dhannawat wants to merge 1 commit into
livekit:mainfrom
chakshu-dhannawat:fix/6424-realtime-interrupt-chat-ctx

Conversation

@chakshu-dhannawat

Copy link
Copy Markdown

Problem

Fixes #6424. When a realtime response produces multiple messages (e.g. a gpt-realtime preamble plus the final answer) and the user interrupts after message A has finished playing but before message B is pulled from message_stream, B is never played and never committed to the local chat context, but it stays in the server-side conversation. On the next turn the model believes it already said B and references an answer the user never heard.

Root cause

AgentActivity._realtime_generation already intends to handle this. _process_messages deliberately leaves unconsumed messages out of message_outputs:

async for msg in generation_ev.message_stream:
    if speech_handle.interrupted:
        break            # remaining messages left out of message_outputs
    entry = await _process_one_message(msg)
    message_outputs.append(entry)
    if entry.out.played == "partial":
        break

But the cleanup is guarded on any_skipped, which is computed by scanning message_outputs — the very list those messages were left out of:

if speech_handle.interrupted and any_skipped and self.llm.capabilities.mutable_chat_context:
    await self._rt_session.update_chat_ctx(self._agent._chat_ctx)

So for the exact case the comment describes ("interrupted before we pulled them"), any_skipped is always False and update_chat_ctx() never runs. The window where any_skipped becomes True (a message pulled into message_outputs, forwarded, and played zero frames) is very narrow, so in practice the guard is inverted relative to its intent.

Fix

Track the abandon condition where it happens instead of inferring it from message_outputs: set a messages_abandoned flag in _process_messages when we break out on an interrupt or a partial play, and run the cleanup on (any_skipped or messages_abandoned). update_chat_ctx is idempotent, so in the rare case where the interrupted message happened to be the last one of the response this performs one harmless extra sync.

This matches the direction suggested in the issue (which the reporter verified locally, and a maintainer green-lit).

Testing

Added tests/test_realtime_interrupt_chat_ctx.py, a hermetic unit test (no network) that reproduces the interrupt-between-messages race with a fake realtime model whose session commits conversation items in generate_reply() the way a real server does. It asserts the never-played message does not survive server-side. The test fails before this change and passes after. ruff check, ruff format, and mypy -p livekit.agents are clean; the existing --unit suite still passes (the only failures locally are pre-existing tests/test_room.py cases unrelated to this change).

…rrupt

When a realtime response produces multiple messages and the user interrupts
after one has played but before a later one is pulled from the message
stream, the later message is never played or committed locally, yet it stays
in the server-side conversation. On the next turn the model thinks it already
said something the user never heard.

_realtime_generation already tries to handle this: _process_messages leaves
unconsumed messages out of message_outputs so the update_chat_ctx cleanup can
drop them server-side. But that cleanup was guarded on any_skipped, which is
computed by scanning message_outputs, the very list those messages were left
out of. So for the case the comment describes (interrupted before we pulled
the message) any_skipped is always False and the sync never runs.

Track the abandon condition where it actually happens: set a
messages_abandoned flag in _process_messages when we break out on an
interrupt or a partial play, and run the cleanup on
(any_skipped or messages_abandoned). update_chat_ctx is idempotent, so in the
rare case where the interrupted message was the last one this does one
harmless extra sync.

Added a hermetic regression test that reproduces the interrupt-between-messages
race and asserts the never-played message does not survive server-side. It
fails before this change and passes after.

Fixes livekit#6424
@chakshu-dhannawat
chakshu-dhannawat requested a review from a team as a code owner July 16, 2026 02:37
@CLAassistant

CLAassistant commented Jul 16, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

voice: interrupted realtime response leaves never-played messages in the server-side chat context (any_skipped guard can't fire for them)

2 participants