knowledge: LLM output-cap budgeting + verification-harness reverse controls (2 of 3 insights; 1 already in flight)

log.md

@@ -34,3 +34,6 @@ Append-only. Format: `## [YYYY-MM-DD] <ingest|revise|lint|gap|contradiction|drif
 ## [2026-07-12] revise | security/secrets-in-code +1 edge case: third-party HTTP client (httpx/requests) logs the full request URL — including a query-param API key — at INFO, so root/DEBUG logging leaks it; keep the client logger above INFO. Found when a standalone sync process set logging.basicConfig(INFO) and httpx wrote the data.go.kr serviceKey to the log file. last_verified bumped to 2026-07-12.
 ## [2026-07-13] ingest | databases +1 (query-optimization): streaming-large-result-sets — memory-bounded export of a huge single-query result. Client-side cursor pulls the whole set to libpq on execute (fetchmany caps only the Python-list explosion); only a server-side/named cursor truly streams but needs a transaction, so it fails under autocommit or a proxy that blocks BEGIN → fall back to client-side fetchmany + disk spool + openpyxl write_only (measured 300k rows 838MB→38MB). Derived from RNR-3440 (potential-listing weekly extract memory peak); QueryPie BEGIN-block generalized to "read-only access proxy", field-tested. Sources: psycopg2 usage/cursor docs (named-cursor WITHOUT HOLD + autocommit exception), openpyxl optimized-modes (write-only near-constant memory, lxml=speed-not-memory).
 ## [2026-07-23] ingest | databases +2: schema-design/online-schema-changes (ACCESS EXCLUSIVE lock avoidance — non-volatile default fast path, ADD CONSTRAINT NOT VALID + VALIDATE at SHARE UPDATE EXCLUSIVE, CHECK-NOT-NULL trick, CREATE INDEX CONCURRENTLY, expand-and-contract to decouple DB migration from app deploy, lock_timeout for lock-queue pile-up) + operations/autovacuum-and-wraparound (NEW category operations: per-table scale_factor/cost_limit tuning for hot tables, age(datfrozenxid)/relfrozenxid + n_dead_tup monitoring, wraparound read-only cliff and superuser VACUUM recovery, VACUUM FULL vs pg_repack). Derived from the Hatchet "Postgres survival guide"; both cross-checked against PostgreSQL official docs (sql-altertable, routine-vacuuming).
+## [2026-08-02] ingest | backend +1 (NEW category common/llm): context-window-budget — sizing an LLM client's output cap to the model actually serving it. The context window holds request and generation together, so `max_tokens` is a reservation inside it, not a separate allowance: derive `max_output <= context_window - worst_case_input` from the serving model rather than inheriting a default sized for the vendor's flagship. Knob table per client (request body / `CLAUDE_CODE_MAX_OUTPUT_TOKENS` + `CLAUDE_CODE_MAX_CONTEXT_TOKENS` / gateway per-model config), base URL points at the endpoint root not `/v1` (the client appends `/v1/messages` itself), and a first-request 400 is read as arithmetic rather than a transient to retry. Derived from a field reproduction (Claude Code 2.1.220 → LiteLLM → 128k OpenAI-compatible model: 99,073 input + default cap vs a 131,072 window returned 400 on the first request; an 8,192 cap ran the session through tool calls). Sources: Claude context-windows docs, LiteLLM exception_mapping + anthropic_unified, Claude Code env-vars, vLLM claude_code integration; both env var names re-confirmed present in the shipped v2.1.220 binary (10 and 6 occurrences, against an ANTHROPIC_BASE_URL positive control at 43).
+## [2026-08-02] ingest | testing +1 (quality): harness-reverse-controls — citing a verification harness's own score. Run a case whose correct verdict is the opposite of the failure being hunted before quoting any number: for a mutation harness that is a semantics-preserving no-op, which is an equivalent mutation and must SURVIVE; a caught verdict there means the cases are dying before the rule ever runs. Reads a uniform verdict as a property of the harness rather than the code, requires observing a verdict on the unmutated artifact first (Stryker's `dryRunOnly`), gives the harness a working tree equivalent to the real runner's (copy the repository, not the directory under test), keeps "never ran" distinct from "ran and passed", and scores `detected / valid` so errored cases leave the denominator instead of counting as catches. Distinct layer from the per-check negative controls in the open spec-artifact-checks / document-conformance-checks work: those ask "can this check go red", this asks "can this harness go green". Cross-linked both ways with quality/tests-that-cannot-fail. Derived from a field reproduction (36/36 CAUGHT cited across five commits and a README while all 105 tests died on FileNotFoundError; a docstring-capitalization no-op reproduced the red verdict, and the rebuilt harness moved to 34/36 and surfaced two rules no test asserted). Sources: PIT basic concepts, Stryker configuration/troubleshooting/mutant-states/equivalent-mutants, Google Testing Blog mutation testing.
+## [2026-08-02] dedup | knowledge-flush candidate "non-interactive CLI hangs on TTY stdin; close stdin then split client from server with the gateway log" is already in flight as wiki/platforms/processes/non-interactive-cli-invocation.md in open PRs #11 and #12 (same page id, same directive, better sourced there: BatchMode/GIT_TERMINAL_PROMPT prompt-channel closure, timeout bounding, three-row arrival table). No third page opened; candidate retired against PR #12. Note for review: #11 and #12 add the same path and need dedup against each other, and neither cites the POSIX SIGTTIN mechanism for bare `&`, the BSD nohup(1) man page (stdin unmentioned, vs GNU's "redirect it from an unreadable file"), or the Claude Code headless docs' stdin-closure exit gating — worth folding into whichever survives.

wiki/backend/common/llm/context-window-budget.md

@@ -0,0 +1,91 @@
+---
+id: backend-common-llm-context-window-budget
+domain: backend
+category: llm
+applies_to: [general]
+confidence: verified
+sources:
+  - https://platform.claude.com/docs/en/build-with-claude/context-windows
+  - https://docs.litellm.ai/docs/exception_mapping
+  - https://docs.litellm.ai/docs/anthropic_unified
+  - https://code.claude.com/docs/en/env-vars
+  - https://docs.vllm.ai/en/stable/serving/integrations/claude_code/
+last_verified: 2026-08-01
+related: [backend-common-reliability-timeouts-and-retries, backend-common-api-design-error-responses]
+---
+
+# Sizing an LLM Client's Output Cap to the Model Actually Serving It
+
+## When this applies
+
+You are repointing an LLM client or agent CLI at a different model or endpoint — a
+self-hosted server (vLLM/Ollama), a gateway (LiteLLM), or a smaller/cheaper model —
+or setting `max_tokens` for a client whose default was picked for a larger model.
+Also when the first request after such a switch returns 400 with a context-window
+error while the same client worked against the original endpoint.
+
+## Do this
+
+1. **Budget input and reserved output against one number.** The context window
+   holds the request *and* the generation: system prompt, every message (tool
+   results, images, documents), tool definitions, plus the output including
+   thinking tokens. `max_tokens` is a reservation inside that window, not a
+   separate allowance, so a cap sized for a 1M-token model overflows a 128k one
+   at identical input.
+2. **Derive the cap from the serving model:**
+   `max_output ≤ context_window − worst_case_input`, where worst-case input is the
+   system prompt + tool definitions + history at the point the client compacts.
+   Set it explicitly rather than inheriting the client's default — the default
+   encodes the vendor's flagship window, and nothing revalidates it when the
+   endpoint changes.
+3. **Set the cap at the client's own knob**, since the client sends `max_tokens`
+   on every request:
+
+| Client | Knob |
+|--------|------|
+| Direct SDK/HTTP call | `max_tokens` in the request body |
+| Claude Code | `CLAUDE_CODE_MAX_OUTPUT_TOKENS` (output reservation) and `CLAUDE_CODE_MAX_CONTEXT_TOKENS` (input budget) — both present in the shipped v2.1.220 binary; the CLI's own message names the first as the fix for "exceeded the … output token maximum" |
+| Gateway in front of many clients | The gateway's own per-model output-cap setting (key name varies by gateway — read its model-config reference), so a client that omits the cap still gets a valid one |
+
+4. **Point the base URL at the endpoint root, not `/v1`.** Claude Code's
+   `ANTHROPIC_BASE_URL` "override[s] the API endpoint to route requests through a
+   proxy or gateway", and vLLM's own integration sets it to `http://localhost:8000` —
+   the root, no `/v1`. The client appends `/v1/messages` itself, so a base URL already
+   ending in `/v1` yields `/v1/v1/messages` and 404s while looking like an auth or
+   routing fault (observed in the reproduction below; the doc gives the root form as
+   the example, not the append rule).
+5. **Read a 400 on the first request as a budget error before touching the
+   network.** LiteLLM raises `ContextWindowExceededError` (400) as a "special error
+   type for context window exceeded error messages". In the reproduction below the
+   message carried the arithmetic itself (input + requested output vs the window);
+   when it does, recompute step 2 from those three numbers rather than measuring
+   anything.
+
+## Edge cases
+
+| Case | Then |
+|------|------|
+| The provider accepts input + `max_tokens` > window (Claude 4.5 and newer) | The request succeeds and generation stops with `stop_reason: "model_context_window_exceeded"` — branch on the stop reason, because a truncated answer is not an error and arrives as a normal 200 |
+| Extended thinking is on | Thinking tokens are a subset of `max_tokens` and billed as output — raise the reservation for thinking instead of assuming it is free, then re-derive step 2 |
+| The gateway advertises a window that differs from the server's | Trust the serving engine's configured length (the value that rejects the request), and correct the gateway's model config so its fallbacks compute against the same number |
+| Prompt caching is enabled | Cached prefixes still occupy the window — caching changes billing, not occupancy, so the input side of step 2 is unchanged |
+| The client is Claude Code and the base URL is not `api.anthropic.com` | MCP tool search is disabled by default (`ENABLE_TOOL_SEARCH=true` when the proxy forwards `tool_reference` blocks) and Remote Control is off as of v2.1.196 — budget the tool definitions as always-present input |
+| The window is large enough but responses truncate mid-structure | The cap, not the window, is the limit — raise `max_output` toward the step-2 ceiling ([backend-common-api-design-error-responses] for surfacing the truncation to callers) |
+
+## Instead of
+
+| If you are about to | Do this instead | Why |
+|---------------------|-----------------|-----|
+| Retry or raise the timeout after a context-window 400 | Recompute the cap from the error's own three numbers and resend once | The rejection is arithmetic, not transient; every retry fails identically ([backend-common-reliability-timeouts-and-retries] owns what is retryable) |
+| Keep the client's default output cap when swapping in a smaller model | Set the cap from the new model's window before the first request | The default is sized for the vendor's largest window, so the smaller model fails on the first full-context turn rather than degrading later |
+| Set the base URL to `http://host:4000/v1` | Set it to `http://host:4000` | The client appends `/v1/messages` itself, so the duplicated prefix 404s while looking like an auth or routing fault |
+| Lower the cap until requests stop failing | Compute `window − worst_case_input` once and set that | Trial-and-error lands on a number that holds for today's history length and breaks as the conversation grows |
+
+## Sources
+
+- https://platform.claude.com/docs/en/build-with-claude/context-windows — the window "holds the conversation history plus the new output"; "Everything in the request counts toward the context window: the system prompt, every message in `messages` (including tool results, images, and documents), and your tool definitions"; overflow behavior: input alone over the window → 400 `invalid_request_error`, while on Claude 4.5+ input + `max_tokens` over the window is accepted and stops with `stop_reason: "model_context_window_exceeded"` (earlier models return a validation error); thinking tokens "are a subset of your `max_tokens` parameter"; "Cached prompt prefixes still occupy the context window"
+- https://docs.litellm.ai/docs/exception_mapping — "400 | ContextWindowExceededError | litellm.BadRequestError | Special error type for context window exceeded error messages - enables context window fallbacks"
+- https://docs.litellm.ai/docs/anthropic_unified — LiteLLM serves the Anthropic-format `/v1/messages` endpoint for "All LiteLLM supported providers" (openai, bedrock, vertex_ai, gemini, azure …), which is what lets an Anthropic-protocol client sit in front of an OpenAI-compatible model
+- https://code.claude.com/docs/en/env-vars — `ANTHROPIC_BASE_URL`: "Override the API endpoint to route requests through a proxy or gateway. When set to a non-first-party host, MCP tool search is disabled by default. Set `ENABLE_TOOL_SEARCH=true` if your proxy forwards `tool_reference` blocks"; Remote Control disabled for non-`api.anthropic.com` hosts as of v2.1.196
+- https://docs.vllm.ai/en/stable/serving/integrations/claude_code/ — `ANTHROPIC_BASE_URL=http://localhost:8000` "Points to your vLLM server (default port is 8000)" — the root, with no `/v1` suffix
+- Field reproduction 2026-07-31 (Claude Code 2.1.220 → LiteLLM → 128k-window OpenAI-compatible model): default output cap + 99,073 input tokens exceeded the 131,072-token window and returned 400 on the first request; an 8,192-token cap ran the same session through tool calls. `CLAUDE_CODE_MAX_OUTPUT_TOKENS` and `CLAUDE_CODE_MAX_CONTEXT_TOKENS` confirmed present in the shipped binary, and absent from the published env-vars reference

wiki/backend/index.md

@@ -70,3 +70,9 @@ Match your situation to a "load when" line; load only matching pages.
 |------|-----------|
 | [shared-state-and-pools](common/concurrency/shared-state-and-pools.md) | Request handlers share in-process mutable state — concurrency-safe structures/confinement vs shared store in multi-instance deployments; sizing thread/connection pools; same-pool nested-acquisition deadlock; bounding queues; debugging deadlock or starvation under load |
 | [distributed-locks](common/concurrency/distributed-locks.md) | Only one instance may perform an action at a time — Redis-style lock with owner token and TTL/watchdog, safe release, when a DB constraint/advisory lock suffices instead; debugging locks released by the wrong holder or work done twice despite a lock |
+
+### llm
+
+| Page | Load when |
+|------|-----------|
+| [context-window-budget](common/llm/context-window-budget.md) | Repointing an LLM client or agent CLI at a different model, a self-hosted server (vLLM/Ollama), or a gateway (LiteLLM); setting `max_tokens` for a client whose default was sized for a larger model; the first request after such a switch returns 400 with a context-window error; deciding where to set the cap (request body vs client env var vs gateway config) and how to point the base URL at a proxy; handling truncation that arrives as a normal 200 |

wiki/testing/index.md

@@ -22,6 +22,7 @@ Match your situation to a "load when" line; load only matching pages.
 | [minimum-case-set](quality/minimum-case-set.md) | Writing tests for a function/endpoint/change and choosing which cases to cover; reviewing whether coverage suffices; picking boundary values by input type; adding a regression test for a bug fix |
 | [behavior-not-implementation](quality/behavior-not-implementation.md) | Deciding what a test should assert; a behavior-preserving refactor broke tests; tempted to expose privates for testing; deciding whether a snapshot test is appropriate |
 | [tests-that-cannot-fail](quality/tests-that-cannot-fail.md) | Reviewing tests that always pass; a bug shipped through an area the suite reported as covered; auditing a suspiciously green suite; judging whether an assertion, error-path test, or mock-based test can actually detect a defect |
+| [harness-reverse-controls](quality/harness-reverse-controls.md) | You built a harness that scores how well something is verified (mutation run, doc/spec gate suite, CI check matrix) and are about to cite its score in a commit, PR, README, or report; its verdicts come out uniform (every case caught, or every case green); deciding what control run proves the harness discriminates, how to score errored/never-ran cases, and what the harness's isolated working tree must contain |
 
 ## data