ACP session dies mid-run: bufferedBytes saturates at whatever limits.acp ceiling is set (64 MiB and 256 MiB), with kilobytes of actual traffic

[nhoelterhoff]

Running an ACP session (pi agent, in-guest) on @rivet-dev/agentos-core 0.2.14, sessions intermittently die mid-run with bufferedBytes saturating at whatever limits.acp ceiling is configured. Raising the ceiling 4x moved the wall but did not change the failure rate, and the session's own traffic is far too small to account for the bytes counted — which is why I think this is a buffer that never drains rather than a limit that is too low.

Observed

Default limits (maxCompletedMessageBytes = 64 MiB):

ERR_AGENTOS_RESOURCE_LIMIT: scope=session=main resource=bufferedBytes
used=67045124 requested=154216 limit=67108864; raise limits.acp.maxCompletedMessageBytes

After raising to 256 MiB (with maxTurnOutputBytes/maxSessionHistoryBytes raised alongside, see below):

ERR_AGENTOS_RESOURCE_LIMIT: scope=session=main resource=bufferedBytes
used=268423857 requested=154780 limit=268435456; raise limits.acp.maxTurnOutputBytes

Both fired ~20 s into a session, and in both cases used lands within ~12 KB of the configured ceiling.

Why I do not think this is a volume problem

For the 256 MiB failure I have the full trace of what the session actually did before it died:

• 2 tool calls total — one bash (find /workspace/transcripts -maxdepth 2 -type f -printf '%f\n' | sort, 29 characters of output) and one read (12 KB).
• $0.0022 of model tokens for the whole session, so generated text cannot account for it either.
• No large files were read, no long-running command ran, and the mounts are small host dirs.

29 characters + 12 KB of tool results and ~2 K tokens of model output do not add up to 256 MiB of buffered ACP bytes. Frequency across a day of identical runs: 2 failures in 23 sessions (~9%), roughly the same before and after the 4x raise.

Reproduction shape

Not reduced to a minimal script — it is intermittent and I have only ever seen it in a full session — but the setup is:

const vm = await AgentOs.create({
  sidecar: { kind: "explicit", handle: await AgentOs.createSidecar() },
  software: [pi, ripgrep],
  limits: { acp: { maxCompletedMessageBytes: 256 * 1024 * 1024, /* + companions */ } },
  // 3 host_dir mounts, deny-by-default network with one allowed host
});
await vm.openSession({ agent: "pi", cwd, env });
await vm.prompt({ content: [{ type: "text", text: kickoff }] });

The agent then reads a couple of files and the session dies. There is no recovery path from the host side: the ACP session is gone, so the whole run is lost.

Two smaller things found alongside

1. The byte-size limits.acp.* fields are undocumented. /docs/resource-limits documents the count-based caps in detail (maxSessionsPerVm, maxPromptsPerSession, …) and even documents that per-session caps are validated against their per-VM counterparts, but maxCompletedMessageBytes, maxTurnOutputBytes and maxSessionHistoryBytes are not listed. The same is true of limits.tls.maxBufferedBytes / limits.resources.maxSocketBufferedBytes, which this workload also has to raise.

2. The companion ceilings are only discoverable one at a time. Raising maxCompletedMessageBytes alone is accepted by core's Zod schema and then rejected by the native sidecar on createVm:

   sidecar rejected request 3: invalid_state: limits.acp.maxCompletedMessageBytes
   (2147483648) must be <= limits.acp.maxTurnOutputBytes (268435456)
   

   Fixing that surfaces the same error against maxSessionHistoryBytes (1 GiB default), and only after both are raised does the VM boot. Validating the whole limits.acp group up front — or documenting the ordering — would save the trial-and-error.

Happy to run a patched build or add instrumentation if you can tell me where bufferedBytes is accumulated and released; I can reproduce at ~9% on demand.

Environment: @rivet-dev/agentos-core 0.2.14, sidecar linux-arm64-gnu (also seen on darwin-arm64), @agentos-software/pi 0.2.7, guest shell brush 0.3.0.

[nhoelterhoff]

Small correction to my own numbers above: I wrote that used lands "within ~12 KB of the configured ceiling" in both cases. That is only true of the 256 MiB one. The exact gaps are:

• 64 MiB: 67108864 - 67045124 = 63,740 bytes (~62 KB short)
• 256 MiB: 268435456 - 268423857 = 11,599 bytes (~12 KB short)

So the accurate statement is that both come to rest at ~99.9% of whatever ceiling is configured, not that both stop within 12 KB of it. The point I was drawing from it is unchanged, but the figure was wrong and I would rather not have it stand.