Sidecar serializes all agent prompts per process: a second session's prompt cannot start until the active turn completes (across sessions and VMs)

[tobowers]

Summary

One native sidecar process runs at most one ACP prompt at a time, across every session and every VM it hosts. While one session's turn is active, a second session's session/prompt does not reach its agent at all — the agent's model request fires only after the first turn completes. Session opens and unloads queue behind the active prompt the same way. Two sessions in one VM, and two VMs sharing the default shared sidecar, both serialize; only two separate sidecar processes run turns concurrently.

For any app that keeps a latency-sensitive session (chat) alongside long-running background sessions (maintenance, codegen) in one runtime, the chat session stalls for the full duration of the background turn — 30–50 s with a real model doing real work.

Reproduction

https://github.com/tobowers/agentos-prompt-serialization-repro — fully self-contained (bun install && bun repro.ts sessions); it serves a scripted mock OpenAI-compatible model on localhost so no API keys are needed. Prompt A's mock reply is delayed 20 s; prompt B (sent 1 s later) gets an instant mock reply.

One VM, two Pi sessions (bun repro.ts sessions; two VMs on the shared sidecar behave identically):

[1.479s] prompt A -> sent (slow turn: mock delays 20 s)
[1.544s] mock model request received (slow lane)
[2.481s] prompt B -> sent (trivial: mock replies instantly)
[21.630s] prompt A <- completed (end_turn)
[21.698s] mock model request received (fast lane)   <-- B reaches the model only now
[21.759s] prompt B <- completed (end_turn)

verdict[sessions]: prompt B took 19.3s -> SERIALIZED

Control (bun repro.ts sidecars — two dedicated sidecar processes):

[3.528s] prompt B -> sent (trivial: mock replies instantly)
[3.587s] mock model request received (fast lane)    <-- B reaches the model immediately
[3.652s] prompt B <- completed (end_turn)
[22.676s] prompt A <- completed (end_turn)

verdict[sidecars]: prompt B took 0.1s -> CONCURRENT

Where the serialization happens

crates/native-sidecar/src/stdio.rs: the frame loop awaits each request to completion before reading the next frame — handle_protocol_frame awaits dispatch_with_prompt_interrupt(...), and NativeSidecar::dispatch takes &mut self, so two requests can never be in flight on one process.

A session/prompt is a "blocking extension request" that holds the loop for the entire agent turn. While it runs, dispatch_with_prompt_interrupt selects on stdin only to service control-lane interrupts (cancel-prompt and permission-response, via the split-lane transport); any other frame — including another session's prompt, open, or unload — is stashed in the single pending_frame slot until the active turn finishes.

Additional observations

• limits.acp.maxPromptsPerSession / maxPromptsPerVm do not change the behavior (admission caps, not scheduling) — their existence suggests concurrent prompts are intended to be supported.
• Reproduces on @rivet-dev/agentos-core 0.2.15 and 0.2.16-rc.1.
• Guest filesystem operations are unaffected (served host-side), which makes the stall easy to misattribute to the agent or model when observed from an app.
• The practical workaround today is one sidecar process per latency class (AgentOs.createSidecar() + sidecar: { kind: "explicit", handle }), which gives up sharing a VM/workspace between the sessions.

Expected

Prompts for different sessions (at minimum across VMs, ideally within one VM) should run concurrently — e.g., by spawning prompt turns as background tasks instead of awaiting them inline in the dispatch loop. The control-lane interrupt plumbing already routes cancel/permission around an active turn, so the protocol seems ready for it.

Environment: macOS 15 (Darwin 25.5.0) arm64, Bun 1.3.14, @rivet-dev/agentos-core 0.2.15 (also 0.2.16-rc.1), agent @agentos-software/pi 0.2.7.