Token budget charges cached prompt reads at full weight; 98% of counted spend is cache

[drewstone]

Measurement

Across a 299-run research fleet, on the 122 runs whose result.json carries a fresh/cache split, spentTotal.tokens.input equals freshInput + cacheRead in 122 of 122 cases, and cache dominates it:

• fleet totals on those runs: 7,070,863 fresh input vs 392,633,152 cacheRead — 98.23% cache
• single example (arena-v9c1-q36-structure-b): input: 4,012,893 = freshInput: 79,453 + cacheRead: 3,933,440 → 98.02% cache

maxTokens is enforced against that rolled-up input, so a 2,000,000 cap stops a run that has consumed roughly 40,000 tokens of new prompt.

Cost, measured

• 88 of 226 capped runs exceeded their cap. Zero of the 88 became winners.
• Those 88 hold 379 of the fleet's 786 verified claims (48%) — the cap preferentially kills the runs producing the science.
• The 85 budget-exhausted runs with usable timing forfeited 39.3 h of 125.3 h total fleet wall-clock (31%), terminating at a median 35% of their allotted deadline.
• The same counter drains the delegation pool: leads were refused child spawns while holding most of their real budget.

The question I cannot answer from our artifacts, which is itself a gap

Whether full-weight charging is wrong depends on what the provider bills. I could not determine that here: no run journal in this fleet contains a promptCache object at all. The field exists in the type (PromptCacheUsage, whose validator was fixed in #818) but nothing on the bridge path populates it. Runtime is enforcing a cap against a composition whose price it cannot observe.

I am deliberately not proposing a specific weight, because guessing one would be as wrong as the current behavior. The pricing has to come from the provider.

Suggested shape

1. Charge from the components — freshInput + output + cacheWrite + w*cacheRead — with w configurable and recorded in the run manifest so the weight is auditable, rather than summing the rolled-up input.
2. Emit spentTotal.tokens as {freshInput, cacheRead, cacheWrite, output} on every harness, and treat an unknown breakdown as a refusal to enforce rather than a silent full-weight charge.
3. Populate promptCache on the bridge path so the provider's own accounting is visible to the budget.
4. Apply the same formula to worker-pool ticket reservation, which today refuses spawns on the same inflated number.

Correction to a related claim

An automated pass reported that some harnesses exclude cacheRead from input, making cross-arm comparisons invalid. I could not reproduce that — 122 of 122 testable runs fold it in consistently. Reporting it so nobody chases it.

Journals, the per-run table, or a PR available once the pricing question is settled.

Measured on runtime 0.133.3–0.133.5, bridge executor, pi and claude-code harnesses.

[drewstone]

Triage

Confirmed against origin/main and localized to one function.

The defect is three lines

src/runtime/supervise/budget.ts:262-264:

function totalTokens(usage: LoopTokenUsage): number {
  return usage.input + usage.output
}

Every reservation and every settlement charges through this, so one cache read costs exactly what one fresh token costs. At the measured 98.23% cache share, fresh input is 1.77% of input, which makes maxTokens about 56x tighter than nominal — consistent with the reported 2,000,000 cap stopping a run at roughly 40,000 fresh tokens.

Suggested shape #2 is mostly already built

The components do not need to be added:

• Spend.tokens already carries freshInput, cacheRead, and cacheWrite.
• cacheBreakdownKnown already exists as the explicit unknown signal — assertValidSpend requires it to be false when present (budget.ts:139-145), which is exactly the "refuse to enforce rather than silently charge full weight" state.
• The invariant freshInput + cacheRead + cacheWrite === spend.tokens.input is already enforced (budget.ts:147-155).

So the type and its validator already model the composition. Only the charging function ignores it. The change is smaller than the issue implies.

The pricing blocker may dissolve

The issue holds the fix behind provider pricing. That follows only if maxTokens is meant as a cost proxy. The pool already budgets money on a separate channel — maxUsd, usdLeft, usdCapped, and usdKnown, which is explicitly honest about its own uncertainty ("False once any recorded work reported an unknown dollar cost; the dollar totals are then a lower bound on real spend, not a measurement", budget.ts:82-84).

If money is bounded by maxUsd, then maxTokens does not need a price weight at all. It can bound novel work:

freshInput + cacheWrite + output

with cacheRead excluded, because re-presenting a cached prefix is not new consumption. Under that reading w is not unknown — it is not needed, and the decision is ours rather than the provider's.

What would falsify it: if maxUsd is unusable in practice, maxTokens is the de facto cost bound and the weight question returns. That is measurable on the same 299-run fleet, and it is the measurement I would run before writing any code:

1. How many runs set maxUsd at all?
2. On those, what fraction report usdKnown: true at settle?

If most runs never set a dollar ceiling, or the bridge path taints usdKnown the way it omits promptCache, then option B is not available yet and the honest sequence is: populate the dollar channel first, then let maxTokens measure work.

Severity

Not an accounting nit. The bias is systematic and it selects against the runs that produce the most: 88 of 226 capped runs exceeded their cap, none became winners, and they hold 48% of the fleet's verified claims. A budget that charges re-reading the same prompt as if it were new work terminates the longest-running investigations first, which is the exact opposite of what a research budget should do.

Agreed and worth keeping

Emitting the breakdown on every harness and treating an unknown breakdown as a refusal (shape #2) is right, and the field for it exists. Applying the same formula to worker-pool ticket reservation (shape #4) is required, not optional — reserve() and reconcile() both route through totalTokens, so fixing one site fixes both, and missing one would leave the delegation pool draining on the old number.

[drewstone]

Shipped

#835 (merged, 0.134.0) — the conserved pool charges each token once, when it first enters the context: freshInput + cacheWrite + output. src/runtime/supervise/budget.ts no longer sums the rolled-up input, which counted a cached prefix again on every turn that read it. The three cache classes are already a disjoint partition of input (assertValidSpend enforces it), so this needs no price weight and no provider pricing. No configurable w was added: a weight would turn a token budget into a secret price budget, and the pool already budgets money on maxUsd.

Unknown breakdown follows the file's existing precedent for an unknown token count — keep enforcing on what is known, report the uncertainty. BudgetReadout and Scope.budget gained cacheBreakdownKnown; false means the debited total is an upper bound on newly-presented work rather than a measurement. Refusing admission was rejected for the reason already written into the file: it would turn one provider that skipped a report into a dead run.

equalKOnCost moved to the same unit, so the cross-run equal-compute check and the within-run pool cannot disagree about what an arm cost.

Behavior change, stated loudly: existing configs keep their declared numbers, but the unit changed. Anyone who tuned maxTokens DOWN to compensate for the old inflation now gets much longer runs.

#837 (open) — an independent adversarial pass over the merged diff found that #835 computed the right unit in a form that is not additive. An aggregate folding one classified turn with one unclassified turn fell back to the rolled-up total for the whole aggregate (records charging 100 and 10 charged 1010 together), and assertValidSpend refused that shape outright, so a pool restored from such a record threw at construction. The charge is now input − cacheRead + output, identical under a complete split and additive over any fold; an incomplete split only has to fit inside input.

The measurement, reproduced on your journals

~/code/discovery-lab/pursuits, 292 runs carrying a result.json (the fleet grew past the 299 you measured):

quantity	value
runs with a complete cache split	129 / 292
fresh input / cacheRead / cacheWrite	7,154,376 / 399,835,200 / 0
cacheRead share of counted input	98.24%

Your 98.23% reproduces. Note cacheWrite is 0 on every run — on this bridge path the new charge reduces to freshInput + output. That is a fact about this telemetry, not evidence that cache-write reporting works.

The dollar channel cannot carry what the token cap just handed it

You asked what bounds cost once cache reads leave the token cap. I measured it:

count	value
runs that set maxUsd at all	6 / 292 (2.1%)
of those, usdKnown: true at settle	1 / 6 (16.7%)
runs reporting spentTotal.usd > 0	0 / 292

The dollar channel is effectively dead in practice. It was dead before this change too — the old token cap bounded cache reads only by accident, at 56x the declared rate, killing 88 of 226 capped runs to do it. But it means a campaign that cares about money must now set maxUsd explicitly, and item 3 below is the load-bearing next step, not a nice-to-have.

Remaining, with owners in the code

1. promptCache is still not populated on the bridge path — NOT in scope of either PR. src/runtime/sandbox-events.ts:106, src/runtime/supervise/sandbox-session.ts:217 and src/runtime/run-loop.ts:750 carry promptCache.readTokens on the event and still emit {input, output, cacheBreakdownKnown: false}. Those runs charge the conservative upper bound, which is correct but blind. Until this lands, the fix helps only the harnesses that already report the split.
2. assertValidSpend throws before the reservation closes in reconcile. The file's own header says a throw placed before settlement is how a ticket escapes the pool, and a malformed spend today leaks its ticket into the join barrier. fix(supervise): make the token charge additive over aggregated spend #837 removes the common trigger; what to CHARGE for a spend the pool cannot trust is a separate fail-loud design question.
3. estimateCost(tokens.input, tokens.output, model) (src/runtime/supervise/runtime.ts:1147, src/runtime/router-client.ts:487) prices cache reads at full input rate. That over-estimates dollars, which is the safe direction for a cap, but it is wrong and it needs the provider cache prices this issue correctly refused to guess.

Correction confirmed

Your note that some harnesses exclude cacheRead from input did not reproduce here either: on all 129 runs with a complete split, freshInput + cacheRead + cacheWrite === input holds exactly. Nobody should chase it.

[drewstone]

Both PRs are merged. Main is at 0.134.2 (162769b8).

#835 shipped the unit: charge each token once, when it first enters the context.
#837 shipped the arithmetic: input − cacheRead + output, which is the same number under a complete split and is ADDITIVE, so an aggregate charges exactly what its records charged. #835 computed the right unit in a form that broke once a spend folded a classified turn together with an unclassified one — records charging 100 and 10 charged 1010 together — and assertValidSpend refused that shape outright, so a pool restored from such a record threw at construction and killed the resumed run. Both are closed, with a per-record fit guard in addTokenUsage so a fold can never rescue a cache read the record refuses.

One correction to my own earlier comment on this issue: I wrote that bad cache telemetry can only over-charge and never buy free tokens. That overstated it. The guarantee the code holds is arithmetic — classes that do not FIT inside the total they partition credit nothing, so the charge never falls below the output tokens. A provider that reports a cache read it never served is trusted exactly as far as its reported input: the token channel is an accounting unit, not a trust boundary. The docstrings, changelog, and architecture note now say that.

One behavior at the version boundary, worth stating before someone finds it in a log. A run resumed from a journal written by a pre-0.134.0 process has its already-recorded spend re-charged in the new unit, so the resumed pool hands back the cache reads the old process charged. That is the same one-time migration every live config takes at 0.134.0, applied to work already on disk. The alternative — charging pre-restart work in the old inflated unit — would put two units in one pool and leave the resumed half carrying this exact bug.

Still open, unchanged from my previous comment: promptCache is not populated on the bridge path (this issue's item 3, and the load-bearing next step), assertValidSpend throws before the reservation closes in reconcile, and estimateCost prices cache reads at full input rate.

[drewstone]

Second measurement: the ticket reservation ignores limits.perWorker.maxTokens

New evidence from a live run (fd-superperm-audit-v3, runtime 0.134.1, bridge executor, codex harness).

Config in the run input:

"limits": { "perWorker": { "maxIterations": 40, "maxTokens": 3200000000 }, "maxLiveWorkers": 6, "maxDepth": 3 },
"budget": { "maxIterations": 150, "maxTokens": 200000000, "deadlineMs": 28800000 }

Children died anyway:

budget pool: ticket 7  spent 2,420,267 tokens > reserved   700,000
budget pool: ticket 12 spent 2,280,265 tokens > reserved   650,000

Two things are visible here:

1. The reservation does not come from perWorker.maxTokens. That field is 3.2e9; the tickets reserved 700,000 and 650,000. The reservation is derived somewhere else, so a caller cannot raise it through the documented knob.
2. An out-of-range value is silently ignored rather than refused. 3.2e9 is obviously wrong — I produced it with a careless blanket multiply — and nothing rejected it at launch. A config that cannot be honored should fail loudly at launch rather than be discarded, which is the same fail-closed principle the budget path already applies elsewhere.

Combined with the cache-accounting issue above, the effect compounds: the pool is measured in tokens that are ~98% cache re-reads, and the per-child reservation is computed from a number the caller cannot set. In this run that killed 2 of 6 settled children outright; 3 more died on child deadline exceeded, so 5 of 6 deaths were budget or clock rather than research outcomes.

Suggested, in addition to the weighted charge above: derive the ticket reservation from limits.perWorker.maxTokens when present, and refuse at launch when it exceeds the root pool rather than silently substituting a default.

🤖 Generated with Claude Code

[drewstone]

Item 3 of the suggested shape — "populate promptCache on the bridge path so the provider's own accounting is visible to the budget" — is implemented in #839. Open, not yet merged; CI runners and the review service are both saturated right now, so I am reporting it here rather than waiting.

The premise held, with one correction

"No run journal in this fleet contains a promptCache object at all" is right in effect. Re-measured on 301 journals under discovery-lab/pursuits: exactly one carries one, and it reads

{"freshInput": 27649, "readInput": 1758080, "writeInput": 0}

That is the bridge executor's private accumulator shape, and no consumer reads it. profile-chat-client.ts:223, improvement/agentic-generator.ts:572, and readPromptCache all key on readTokens / writeTokens. So 1.76M cache reads were reported and still charged in full. The observability gap was slightly worse than "nothing populates it": one path populated it in a dialect nothing speaks.

Now observable

The counters were already on the wire on three separate producers, and all three were dropped at the same extractor:

Producer	Field	Was dropped at
sandbox terminal done	tokenUsage.cacheReadInputTokens	sandbox-events.ts:100-107 — the comment named the field; the code forwarded only input/output/cost
cli-bridge SSE	cache_read_input_tokens, cache_write_input_tokens, fresh_input_tokens	same extractor
environment-provider	cacheReadInputTokens, cacheCreationInputTokens	same extractor

Each now surfaces a promptCache in the PromptCacheUsage vocabulary, and spend classes derived from it: a read and a write that fit inside input give a complete partition; one counter alone gives that counter plus cacheBreakdownKnown: false; a claim that overflows the total it partitions credits nothing. A counter the provider did not report stays absent — a zero would assert the provider measured no cache, which is a different fact.

This is what answers the question you flagged as unanswerable from the artifacts. The budget can now see which prompt tokens are a re-read of content it already charged, per turn, on the paths the fleet actually runs.

Still NOT observable

• Cache on the codex backend. Codex reports cached_input_tokens; cli-bridge/src/backends/codex.ts:240 reads only input_tokens / output_tokens, so nothing reaches the wire.
• The write counter on the claude backend. Anthropic's cache_read_input_tokens passes through by name-luck, but cache_creation_input_tokens does not match the declared cache_write_input_tokens (backends/claude.ts:63,73), so every claude turn is read-only and its split stays incomplete.
• claude's prompt total is fresh-only. Anthropic's input_tokens excludes cache tokens, unlike pi.ts:1235-1236 which sums them. So on that backend a credited read is subtracted from a total that never contained it.

All three are one cli-bridge file. Not fixed in #839, which is agent-runtime only.

The dollar channel, as asked

maxUsd is dead because no backend the fleet runs delivers a billed figure — with one exception nobody is collecting.

cli-bridge emits cost only with cost_known: true and provider/billing provenance (src/usage.ts:59-67, src/streaming/sse.ts:108-121). pi declares costKnown: false by contract (backends/types.ts:138 — "Provider-billed dollars are not reported by Pi or the proxy") and offers only a catalog estimate; codex reports no cost at all. But claude does receive one: ClaudeStreamResult.total_cost_usd is declared at backends/claude.ts:74, and claude.ts:364 yields usage: r.usage where total_cost_usd is a sibling of usage, not a member — so the one real billed number on this wire is dropped before it leaves the backend.

So the channel is fixable, in two tiers:

1. As a priced estimate, today. The bridge already sends the response model on every chunk (supervise/runtime.ts:3808-3814) alongside the token counts — exactly the arguments estimateCost(input, output, model) takes, already used at supervise/runtime.ts:1148. Nothing new is needed on the wire.
2. As a provider receipt, for claude only, after the total_cost_usd forwarding fix above.

It cannot be revived as a receipt for pi or codex without provider pricing. That part of your original conclusion stands.

[drewstone]

The dollar channel, after two changes

This issue refused to guess provider pricing. The dollar channel had the same gap, and it was worse than a guess: it reported a measured $0. Two changes landed. Here is exactly what it can now say and what it still cannot.

Measured before

Re-ran the count on discovery-lab/pursuits (306 result.json, 291 carrying a spentTotal):

	count
spentTotal.usd > 0	0 of 291
spentTotal.usdKnown === true	44 — every one of them reporting usd: 0
runs setting a maxUsd	18 of 326 run-input.json (16 at $0.056, 2 at $1.50); 268 carry the key as null

The 44 are the sharp part. Those runs did not report unknown dollars — they certified a measurement of zero. One of them, arena-v11a1-superperm-b, certifies usd: 0 alongside 2,615,311 input tokens.

What it now reports

A provider receipt, for the claude harness only. drewstone/cli-bridge#159 (b153652). ClaudeStreamResult declared total_cost_usd but the backend yielded only usage — the figure is a sibling of usage, not a member, so the one genuinely provider-billed number the bridge receives never left the backend. It now travels as cost_known: true with provider-receipt provenance at cost_scope: total, covering the whole claude -p invocation including every internal model call. It reaches spentTotal.usd with usdKnown true.

A priced estimate, everywhere else on the bridge path. #840 (60b97b6d). A turn that billed nothing is priced from agent-eval's catalog against that turn's own tokens, using the response model the bridge already sends. Nothing new on the wire.

The two never read the same:

• Spend.usd — all dollars attributed
• Spend.usdEstimated — the part priced from the catalog
• usd - usdEstimated — what a provider is known to have billed
• usdKnown — whether the total is a measurement at all

A turn holding a receipt is never priced on top of it. An unpriced model contributes no dollars and stays usdKnown: false — unknown, never free. A dollar cap is unchanged: observe and reconcile still refuse unknown dollars under a maxUsd root.

What it still cannot

A provider receipt for anything except claude. Not a limitation of the wiring — nothing else reports billed dollars:

harness	what it reports
claude	total_cost_usd — a real provider receipt
pi	costKnown: false by contract. Pi exposes a local catalog estimate; neither Pi nor the scoped proxy reports what the provider billed
opencode, prime	a local figure, correctly routed to the catalog-estimate lane — never a receipt
codex, gemini, kimi-code	no dollar figure at all

So for every harness but claude, the total is our catalog price, not the provider's invoice. Closing that needs provider pricing — the thing this issue was right not to invent.

The cache-rate gap is this issue's gap, inherited

This issue's core point was that cached reads dominate (98.23% of counted input) and their price has to come from the provider. The dollar channel inherits exactly that.

agent-eval's pricing table is { input, output } per model — verified on the published @tangle-network/agent-eval@0.145.2 type. There is no cache-read rate. So a prefix the provider served from cache is priced at the full input rate.

That direction is deliberate: it overstates a cache-heavy turn rather than inventing a discount the catalog cannot support, and an invented discount understates spend. It also means the estimate is an upper bound on a cache-heavy run, and given the 98% figure above, that bound is loose. The real number needs the provider's cache rate.

Worth noting against this issue's item 3: #839 landed promptCache on the bridge path, so the cache split is now visible where this issue found none. What is still missing is the rate, not the composition.