[Bug] The tell-throughput benchmark reports 100,000 ops for an iteration that handled 9,999, so the published headline figure is about 10x too high and measures the drop path

[pathosDev]

Problem

benchmarks/single-node/tell-throughput.ts reports 100 000 operations for an iteration in which the actor handles 9 999 messages. The enqueue loop is fully synchronous, so nothing drains while it runs; the default bounded mailbox (10 000, drop-head) discards everything past its capacity; and opsPerIteration makes the harness compute totalOps from the intended count rather than the completed one.

The published figure for the headline case is therefore inflated by roughly 10x, and what it actually measures is the drop path — which, because drop-head does an Array.shift() on a 10 000-element queue, is itself an expensive operation that has nothing to do with throughput.

Nothing detects this. bench:smoke only proves the file runs, and the harness has no notion of completed versus attempted work.

Evidence

The synchronous enqueue loop and the op count it is paired with:

benchmarks/single-node/tell-throughput.ts:22-27
async function drain(system: ActorSystem, batch: number): Promise<void> {
  const ref = system.spawnAnonymous(Counter);
  for (let i = 0; i < batch; i++) ref.tell({ kind: 'increment' });
  await ref.ask<number>({ kind: 'get' }, 30_000);
  ref.stop();
}

benchmarks/single-node/tell-throughput.ts:39
    { name: 'batch=100k', unit: 'msg', iterations: 10,  opsPerIteration: 100_000, run: () => drain(system, 100_000) },

The harness multiplies the intended count:

benchmarks/lib/harness.ts:93
  const totalOps = iterations * opsPerIteration;

and the default that discards the difference:

src/util/Constants.ts:93
export const DEFAULT_MAILBOX_CAPACITY = 10_000;

Reproduced — the same drain() shape with the counter read back:

sent=1000    processed=1000    lost=0
sent=10000   processed=9999    lost=1
sent=100000  processed=9999    lost=90001

Against the reported table row:

│ batch=100k  │  1,212,482 msg/s │  825 ns │

Actual throughput for that row is ≈ 121 000 msg/s. It is also lower than the batch=10k row (≈ 311 000 msg/s), because 90 001 of the tells went through the drop-head path and paid an O(n) shift each.

The lost=1 at exactly 10 000 is worth recording separately: the mailbox bound counts the benchmark's own get message, so filling a mailbox to capacity evicts a real message to make room for the reply request.

Proposal

Make the harness measure completed work rather than intended work. Add a verify() (or actualOps()) hook to BenchmarkSpec that each benchmark uses to report what the system actually processed, and fail the run when it diverges from opsPerIteration. That single change makes this class of error impossible rather than merely fixed once.

Then fix this benchmark: either keep the batch under the mailbox capacity, or pin an unbounded mailbox for the measurement and say so in the file header. Until one of those lands, no msg/s figure from this suite should be published.

benchmarks/memory/queued-messages.ts has the same defect at its largest size and is filed separately in this batch.

Acceptance sketch

• A benchmark whose actor processed fewer operations than the harness was told fails the run rather than reporting a number.
• tell-throughput.ts reports a figure derived from messages actually handled.
• The batch=100k row is re-measured and any previously published figure is corrected.
• bench:smoke would catch a recurrence — today it only proves the file starts.

Verification status

Found in the ten-lens production-readiness review of 2026-08-05 (v0.13.0) and re-verified before filing: reproduced by execution — the sent/processed/lost table above is from an instrumented run of the benchmark's own drain() shape, and the reported row is from a real bun run benchmarks/single-node/tell-throughput.ts. Two independent reviewers found this separately.

Part of the production-readiness review batch — tracked in #913.