Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/performance/CONTRACT.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,9 @@ the completion offset and observed surplus are recorded. Bytes never read are no
claimed as retained. Missing usage stays null. Reported lengths exceeding the cap,
exact-length mismatches and unmet reported-prefix requirements make the wave
ineligible. Usage is provider evidence, not verified billing or engine attestation.
In particular, a reported prefix miss after warmup can reflect server-specific
cache-block alignment rather than collector malfunction; priming alone does not
establish reusable cache blocks. The reported-hit requirement is not relaxed.
Contradictory reported reasoning counts greater than completion counts are
ineligible; they are not repaired or silently included in a rate.

Expand Down
11 changes: 11 additions & 0 deletions docs/performance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,17 @@ Cache modes:
| `reported-prefix-zero` | Unique per-request salt; require reported prefix-cache count zero. |
| `reported-prefix-hit` | Stable salt per cell/lane; require explicit warmup and reported hits in measured waves. |

Warmup is necessary for `reported-prefix-hit`, but does not guarantee a hit.
Servers may reuse only complete cache blocks, with engine-specific alignment
requirements. A short prompt can therefore report zero cached tokens after priming.
Consult the server's cache mechanism and inspect `cached_prompt_tokens`; use a
longer appropriately aligned prompt when testing warm decode, without assuming
length alone guarantees reuse.

If cache hits are not required for your measurement, use a separate `observe`
workload. That changes the workload and the claim; it does not repair an
ineligible `reported-prefix-hit` run or make the two workloads comparable.

Required prefix modes need `vllm-fixed-v1`. Plans name the declared mechanism
and the provider observation source; the response record retains the actual
reported count. A flag plus a reported zero is **not universal proof of cold
Expand Down