feat: gate quantization and speculative decoding on measured evidence

[Yash-Chindam]

Fifth PR closing gaps between the design spec and the implementation.

Gap this closes

§9: quantization formats "as independent variants" measured for memory, throughput, latency and quality; speculative decoding "as an experiment"; and the design target "documented quality delta for every optimization variant". Quantization was one field on a model card and speculative decoding appeared nowhere.

What changed

• EngineVariant in the catalog (variants:), bound to one immutable base revision, of kind quantization or speculative-decoding.
• In development, a variant renders under experiments in config/ray-serve.yaml as its own application with no warm replica.
• A variant cannot reach staging/production without a baseline and a variant benchmark. Runs that differ in dataset, workload, hardware, or concurrency are rejected at load.
• variant_verdict: quality loss beyond tolerance is always a regression; speculative decoding must lower p95 or raise throughput; quantization must reduce measured GPU memory.
• A promoted variant is applied to the base model's engine arguments and added to the cache fingerprint.
• GET /v1/registry/variants lists every variant with its deltas and regressions.
• python -m llm_router.evaluation runs a dataset against an OpenAI-compatible endpoint and prints a catalog benchmark record, reading GPU memory and draft acceptance from the engine's /metrics when present.

Not done

• Nothing has been measured. Both committed variants (general-gptq, high-capability-speculative) are in development with no evidence. Producing it needs a GPU and a running vLLM engine.
• The benchmark CLI wrapper (main) is excluded from coverage and has not been run against a real engine; the transport, measurement parsing, and record shaping underneath it are tested with a mocked endpoint.
• The CLI sends requests one at a time, so its throughput figure is sequential.

Test plan

• 21 new tests: promotion blocked on quality loss, on speculative overhead, on unmeasured or unreduced memory, and on non-comparable runs; experiment rendering; cache identity on promotion; committed ray-serve.yaml matches the catalog
• ruff format --check ., ruff check ., mypy clean
• pytest tests/unit tests/integration: 262 passed, coverage 98%
• Playwright end-to-end suite: 9 passed locally

🤖 Generated with Claude Code