feat(experiments): A/B routing — serve the candidate to a fraction of users (Step 5.7c)

[officialCodeWork]

Summary

Step 5.7c — A/B routing: deterministic variant assignment that actually serves the candidate retrieval config to a fraction of live users. This is the first 5.7 slice that can change which response a user receives (5.7a was observe-only analyzer/tracker; 5.7b shadow mode is observe-only background fan-out).

• ABRouter (rag_gateway.experiments, beside ShadowRunner) — assign(request_id) hashes deterministically to a variant (hash(request_id) < routing_sample_rate ? candidate : control); also is_candidate / candidate / record / tag.
• Served-path wiring in /v1/query + /v1/retrieve: the candidate arm routes inline via _route_query (one retrieval, no added latency vs. a normal query). Not degrade-open — a candidate failure surfaces as a normal RetrievalError (→ 502) rather than silently swapping back to control, which would corrupt the experiment and hide a broken candidate. The served arm's outcome_metric (mean retrieval score) is recorded once per fresh retrieval (skipped on a cache hit, same apples-to-apples discipline as shadow).
• Variant-partitioned retrieval cache — compute_plan_hash / compute_params_hash fold in the assigned variant, so control and candidate never share a cache entry. Byte-identical key when routing is off.
• ExperimentAssignment wire tag (experiment / variant / is_candidate) on QueryResponse / RetrieveResponse (experiment field, additive + defaulted None). REST-only for now — the gRPC RagService predates A/B routing, so the proto mirror is deferred exactly like corpus_decision. Served variant also rides the existing gateway.query_complete log.
• Config: ShadowCandidateConfig generalised to the shared CandidateConfig (used by shadow_candidate + the new routing_candidate); cfg.experiments gains routing_enabled / routing_sample_rate / routing_experiment / routing_candidate. Gated separately from shadow_enabled because routing can change a response; both feed one tracker under distinct experiment ids.
• Governance: the candidate routes through the same HybridRetriever.retrieve (the canonical read_chunk PDP site), so ACLs are enforced identically and the policy coverage linter needs no new entry.
• Inert + opt-in by default: build_app leaves ab_router unset; build_app_from_config builds it only when enabled and routing_enabled. Production injects a candidate over real backends via build_app(ab_router=…).
• ragctl ab drives the full assign → serve → record → analyze flow against in-process stubs (e.g. ragctl ab --rate 0.5 --lift 0.25).

dist/schemas (+ ExperimentAssignment.json), dist/openapi.*, and dist/rag.schema.* regenerated.

Documentation

• ADR-0032 — new "Step 5.7c (A/B routing, landed)" section
• docs/reference/experiments.md — A/B routing section, config rows, ragctl ab
• docs/architecture/ab-routing.md — new architecture doc
• docs/README.md — registered the new doc
• TRACKER.md — 5.7c ✅, next action → 5.7d, PR history

Test plan

• ruff check + ruff format --check clean
• mypy --strict clean (292 source files)
• RAG001 logging gate clean
• schema-drift + openapi-drift regenerated (committed)
• gRPC proto-compat gate (REST-only experiment allowlisted with rationale)
• policy-coverage + SPI-contract gates pass
• full suite 2497 passed, 4 skipped (integration deselected)
• ~25 new tests: ABRouter unit (assignment/record/tag), gateway integration (candidate served + tagged, control not calling candidate, /v1/retrieve, dashboard sees served experiment, inert default), variant cache-partition (test_cache_keys), config wiring + routing_* (test_experiments_config)
• ragctl ab ran end-to-end (52% candidate split, control mean ≈0.50 vs candidate ≈0.75, lift +50%, significant)

🤖 Generated with Claude Code