Skip to content

Load testing: nothing answers "how many concurrent calls per core" #56

Description

@jason-shen

Problem

Nobody can answer "how many concurrent calls does this handle per core", which is the first question anyone evaluating voice infrastructure asks and the last question this repo can answer.

It is also blocking a real decision. server.max_sessions exists and defaults to 0 (unlimited):

max_sessions = 0  # Global cap on live sessions; past it POST /whip returns 503 with Retry-After. 0 = unlimited.

The config comment says to "set this to what one instance can actually serve", and the project provides no way to find that number out. A default of unlimited is the honest choice given that, but it means the first person to get traffic discovers the ceiling by falling off it.

The cost model per session is not obvious enough to estimate on paper either: a DTLS/SRTP association, Opus decode and encode through a wazero WebAssembly module, a resampler, VAD on every 20 ms frame, a streaming STT websocket, a streaming LLM connection, a streaming TTS connection, plus the turn buffer and rolling-summary goroutines. Some of that is CPU-bound locally, some is just held file descriptors waiting on a vendor. Only measurement separates them.

Proposed change

A load-generation harness under internal/loadtest/ or a loadtest/ module, plus a documented result.

Client. N synthetic WHIP clients, each doing a full POST /whip handshake, publishing a real Opus track from a fixture WAV of actual speech (silence is not a load test — VAD gates on it and STT sends nothing), and consuming the returned track. The Go SDK already does all of this for one client; the harness is largely a driver around it.

Providers. Default to a mock STT/LLM/TTS with configurable latency, so a run measures the server and not a vendor's rate limit or the tester's credit card. Add a flag to point at real providers for the end-to-end number, and report the two separately, because they answer different questions.

Measure, per run: sessions established vs. attempted, CPU and RSS at plateau, goroutine count, time-to-first-audio p50/p95/p99, audio underruns or gaps on the outbound track, and the point at which p95 latency degrades — that last one, not an OOM, is the real capacity limit for voice.

Report. A table in docs/ for at least two instance shapes (a 2 vCPU box and something larger), stating the provider mode used, and a recommended max_sessions starting point derived from it. Include the methodology and the fixture, so the numbers can be re-derived and argued with rather than taken on faith.

This composes with #51 (metrics): if the Prometheus endpoint lands first, the harness can scrape it instead of reimplementing the measurement, and the same dashboard serves both the load test and production.

Acceptance criteria

  • Harness runs N concurrent real WHIP sessions with real Opus audio against a local server.
  • Mock provider mode with configurable latency, and a real-provider flag.
  • Reports established sessions, CPU, RSS, goroutines, and latency percentiles.
  • Documented results for at least two instance sizes, with methodology.
  • A recommended max_sessions starting point, justified by the numbers.
  • Runnable by a contributor in one command, with the fixture committed.

Pointers

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: mediaWebRTC, WHIP, RTP/Opus, VAD, turn-taking, barge-ineffort: mediumA day or two, spans a few packagesenhancementNew feature or requesthelp wantedMaintainers would welcome an outside contributor here

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions