Problem
Nobody can answer "how many concurrent calls does this handle per core", which is the first question anyone evaluating voice infrastructure asks and the last question this repo can answer.
It is also blocking a real decision. server.max_sessions exists and defaults to 0 (unlimited):
max_sessions = 0 # Global cap on live sessions; past it POST /whip returns 503 with Retry-After. 0 = unlimited.
The config comment says to "set this to what one instance can actually serve", and the project provides no way to find that number out. A default of unlimited is the honest choice given that, but it means the first person to get traffic discovers the ceiling by falling off it.
The cost model per session is not obvious enough to estimate on paper either: a DTLS/SRTP association, Opus decode and encode through a wazero WebAssembly module, a resampler, VAD on every 20 ms frame, a streaming STT websocket, a streaming LLM connection, a streaming TTS connection, plus the turn buffer and rolling-summary goroutines. Some of that is CPU-bound locally, some is just held file descriptors waiting on a vendor. Only measurement separates them.
Proposed change
A load-generation harness under internal/loadtest/ or a loadtest/ module, plus a documented result.
Client. N synthetic WHIP clients, each doing a full POST /whip handshake, publishing a real Opus track from a fixture WAV of actual speech (silence is not a load test — VAD gates on it and STT sends nothing), and consuming the returned track. The Go SDK already does all of this for one client; the harness is largely a driver around it.
Providers. Default to a mock STT/LLM/TTS with configurable latency, so a run measures the server and not a vendor's rate limit or the tester's credit card. Add a flag to point at real providers for the end-to-end number, and report the two separately, because they answer different questions.
Measure, per run: sessions established vs. attempted, CPU and RSS at plateau, goroutine count, time-to-first-audio p50/p95/p99, audio underruns or gaps on the outbound track, and the point at which p95 latency degrades — that last one, not an OOM, is the real capacity limit for voice.
Report. A table in docs/ for at least two instance shapes (a 2 vCPU box and something larger), stating the provider mode used, and a recommended max_sessions starting point derived from it. Include the methodology and the fixture, so the numbers can be re-derived and argued with rather than taken on faith.
This composes with #51 (metrics): if the Prometheus endpoint lands first, the harness can scrape it instead of reimplementing the measurement, and the same dashboard serves both the load test and production.
Acceptance criteria
Pointers
Problem
Nobody can answer "how many concurrent calls does this handle per core", which is the first question anyone evaluating voice infrastructure asks and the last question this repo can answer.
It is also blocking a real decision.
server.max_sessionsexists and defaults to0(unlimited):The config comment says to "set this to what one instance can actually serve", and the project provides no way to find that number out. A default of unlimited is the honest choice given that, but it means the first person to get traffic discovers the ceiling by falling off it.
The cost model per session is not obvious enough to estimate on paper either: a DTLS/SRTP association, Opus decode and encode through a wazero WebAssembly module, a resampler, VAD on every 20 ms frame, a streaming STT websocket, a streaming LLM connection, a streaming TTS connection, plus the turn buffer and rolling-summary goroutines. Some of that is CPU-bound locally, some is just held file descriptors waiting on a vendor. Only measurement separates them.
Proposed change
A load-generation harness under
internal/loadtest/or aloadtest/module, plus a documented result.Client. N synthetic WHIP clients, each doing a full
POST /whiphandshake, publishing a real Opus track from a fixture WAV of actual speech (silence is not a load test — VAD gates on it and STT sends nothing), and consuming the returned track. The Go SDK already does all of this for one client; the harness is largely a driver around it.Providers. Default to a mock STT/LLM/TTS with configurable latency, so a run measures the server and not a vendor's rate limit or the tester's credit card. Add a flag to point at real providers for the end-to-end number, and report the two separately, because they answer different questions.
Measure, per run: sessions established vs. attempted, CPU and RSS at plateau, goroutine count, time-to-first-audio p50/p95/p99, audio underruns or gaps on the outbound track, and the point at which p95 latency degrades — that last one, not an OOM, is the real capacity limit for voice.
Report. A table in
docs/for at least two instance shapes (a 2 vCPU box and something larger), stating the provider mode used, and a recommendedmax_sessionsstarting point derived from it. Include the methodology and the fixture, so the numbers can be re-derived and argued with rather than taken on faith.This composes with #51 (metrics): if the Prometheus endpoint lands first, the harness can scrape it instead of reimplementing the measurement, and the same dashboard serves both the load test and production.
Acceptance criteria
max_sessionsstarting point, justified by the numbers.Pointers
streamcoreai/go-sdk— existing WHIP client to build the synthetic client oninternal/session/manager.go— the cap being calibratedinternal/audio/— Opus encode/decode through wazero, the main per-session CPU costinternal/vad/— runs per 20 ms frame per sessionconfig.toml.example:10—max_sessionsand its comment