feat(voice): support runtime model updates

[rosetta-livekit-bot]

Summary

Ports livekit/agents#6440 runtime model swapping to Agents JS.

Adds awaitable Agent.updateOptions() to swap STT, VAD, LLM, and TTS while inactive or while the live pipeline is active. null explicitly disables an agent-level model and overrides session fallback, matching Python None semantics.

The JS adaptation preserves Python's synchronous atomicity across asynchronous pipeline teardown:

• Serializes live model transactions with activity and recognition lifecycle locks.
• Propagates default/user VAD ownership and serializes VAD reset/replacement tasks.
• Keeps paused or handoff-stale updates state-only, then reconstructs resources on resume.
• Prewarms before mutation and fully rolls back references, recognition state, keyterms, context forwarding, and listeners after operational failures.
• Preserves STT/VAD pipeline and turn context unchanged when preflight fails.
• Preserves released public model-getter types while raw internal fields distinguish null disable from undefined inheritance.
• Re-exports AgentUpdateOptions through the public voice API.

Source diff coverage

Source file	Classification
livekit-agents/livekit/agents/voice/agent.py	Ported to agents/src/voice/agent.ts with JS camelCase, string resolution, running/inactive behavior, and internal null disable semantics without widening existing getter contracts.
livekit-agents/livekit/agents/voice/agent_activity.py	Ported and adapted for awaitable JS teardown, transaction serialization, stale-activity handling, listener rebinding, preflight, and failure rollback.
livekit-agents/livekit/agents/voice/audio_recognition.py	Ported with STT/VAD lifecycle locks, effective VAD ownership, context reset, and rollback support.
livekit-agents/livekit/agents/voice/keyterm_detection.py	Ported swapStt() with existing-keyterm propagation and rollback coverage.
tests/test_agent_update_options.py	Ported and expanded with JS-specific concurrency, handoff, listener, fallback/null, public compatibility, and failure-injection coverage.

Target-only adaptation: agents/src/voice/agent_v2.ts normalizes nullable agent model fields to undefined in existing optional hook contexts.

Validation

• Compile/lifecycle suite: 3 files, 30 tests passed.
• pnpm --filter @livekit/agents typecheck.
• pnpm --filter @livekit/agents build.
• Changed-file lint and formatting passed; package lint exits 0 with pre-existing warnings.
• pnpm build passed.
• API Extractor check/update remains blocked by its pre-existing inability to parse generated export * as ___ syntax in agents/dist/index.d.ts.

Changeset

Added a patch changeset for @livekit/agents.

Cue voice E2E

Verified exact commit 8b2ea3547edf7f0a20ed00daf3aa6b6f8867e705 in fresh Cue voice mode.

• Session: sid_3f553776f1ef.
• Providers: real Deepgram nova-3 STT and Cartesia sonic-3 TTS; initial OpenAI gpt-4.1-mini, replacement gpt-4.1-nano.
• A swap_runtime_llm tool awaited agent.updateOptions({ llm: replacement }).
• Path-specific persisted order: initial model actual chat() event → successful tool output → replacement model actual chat() event → speaking → assistant commit → listening.
• Result: resolved, exit code 0, with completed playout and no framework error event.
• Artifacts: inspected persisted events.jsonl, command result JSON, command WAV, and session WAV. Both channels in both WAVs are non-zero.

---

Ported from livekit/agents#6440

Original PR description

Agent.update_models(*, stt, vad, llm, tts) swaps a model in place. Only the models passed are changed; strings resolve to inference models like the constructor, and None disables a model (overriding the session), matching Agent(stt=None).

When the agent isn't running it replaces the stored model, applied on the next start. When running, it applies to the live pipeline: the STT and VAD streams are rewired, the LLM and TTS take effect on the next generation and synthesis respectively, and each model's metrics_collected/error listeners follow the new instance. The call is synchronous and atomic.

The Agent is the source of truth — the activity resolves agent.<model> if given else session.<model> live — so update_models just writes to the agent. This is distinct from AgentSession.update_agent (full handoff, new instance) and from a model's own update_options (same instance, provider options).

Realtime models

A RealtimeModel owns a live session created when the agent starts, so it can't be swapped in place. While running, passing a RealtimeModel as llm (or swapping away from one) raises RuntimeError pointing to AgentSession.update_agent. Validation happens before any mutation, so the call stays all-or-nothing.

[changeset-bot]

🦋 Changeset detected

Latest commit: d1d06d4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 37 packages

Name	Type
@livekit/agents	Patch
@livekit/agents-plugin-anam	Patch
@livekit/agents-plugin-anthropic	Patch
@livekit/agents-plugin-assemblyai	Patch
@livekit/agents-plugin-azure	Patch
@livekit/agents-plugin-baseten	Patch
@livekit/agents-plugin-bey	Patch
@livekit/agents-plugin-cartesia	Patch
@livekit/agents-plugin-cerebras	Patch
@livekit/agents-plugin-deepgram	Patch
@livekit/agents-plugin-did	Patch
@livekit/agents-plugin-elevenlabs	Patch
@livekit/agents-plugin-fishaudio	Patch
@livekit/agents-plugin-google	Patch
@livekit/agents-plugin-hedra	Patch
@livekit/agents-plugin-hume	Patch
@livekit/agents-plugin-inworld	Patch
@livekit/agents-plugin-lemonslice	Patch
@livekit/agents-plugin-liveavatar	Patch
@livekit/agents-plugin-livekit	Patch
@livekit/agents-plugin-minimax	Patch
@livekit/agents-plugin-mistral	Patch
@livekit/agents-plugin-mistralai	Patch
@livekit/agents-plugin-neuphonic	Patch
@livekit/agents-plugin-openai	Patch
@livekit/agents-plugin-perplexity	Patch
@livekit/agents-plugin-phonic	Patch
@livekit/agents-plugin-resemble	Patch
@livekit/agents-plugin-rime	Patch
@livekit/agents-plugin-runway	Patch
@livekit/agents-plugin-sarvam	Patch
@livekit/agents-plugin-silero	Patch
@livekit/agents-plugin-soniox	Patch
@livekit/agents-plugin-tavus	Patch
@livekit/agents-plugins-test	Patch
@livekit/agents-plugin-trugen	Patch
@livekit/agents-plugin-xai	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR