AgentContextOS — Build Tracker

How to use: Read this first in every session to know exactly where the build stands. When a step lands on main: flip its Status + PR, add a row to PR & Branch History, and refresh the Status block.

Standing requirement (all steps): Every deliverable — scripts, tooling, CI, dev-stack — must work on Windows, macOS, and Linux without WSL. Task targets via Taskfile.yml (cross-platform) + Makefile (Unix convenience). CI matrix always includes ubuntu-22.04, macos-14, and windows-latest.

Status


Last updated	2026-06-19
Current phase	Phase 7 — Pilot, Harden, GA (10 / 10 steps ✅) — all phases complete
Overall	84 / 84 steps ✅ — v1.0.0 GA
Next action	🎉 GA reached. Engineering scope (Phases 0–7) complete; `v1.0.0` cut. Remaining items are external/process (external pentest, SOC 2 audit, registry first-publish, signed referenceable customers, on-call staffing + status page, launch assets) — tracked in the GA readiness checklist. Post-GA: cut the v1.0.1 patch tag (serve-path fix below; note: GitHub Actions runs are currently blocked by an account billing/spending-limit failure — every `docker.yml` run since GA failed at job start, so the GHCR image was never actually published), then the V1.1 backlog. V1.1 underway: Step 3.10 corpus admin CRUD landed (#189); a follow-on post-GA batch in #194 adds corpus/webhook PATCH (completes the 3.10 update path), admin-UI bug fixes, real-backend (OpenAI LLM + content hydration) wiring, and the local Docker/demo stack.

Recently shipped

post-GA batch ✅ (V1.1, #194) — corpus/webhook PATCH completes the Step 3.10 CRUD update path (CorpusStore / SubscriptionStore.update — SPI + Noop + Pg; PATCH /v1/corpora/{id} + PATCH /v1/webhooks/subscriptions/{id}; embedding model/dim immutable, signing secret stays masked) + admin-UI bug fixes (BUG-002/003/004/007/008/009 — Edit dialogs, per-source demo badges, metrics hydration, stale copy) + real-backend wiring (config-driven OpenAILLM + content hydration so ingested docs are retrievable end-to-end) + a local Docker app stack (pg + gateway + admin-ui), branding, and GTM demo assets. Verified locally (CI billing-blocked): ruff / mypy --strict (333 files) / full pytest / proto·openapi·schema drift + admin-ui tsc·eslint·46 vitest·next build all green. Testing-findings tracker added at BUGS.md (BUG-005 audit-coverage + BUG-006 ingest doc-count remain open, root-caused). #194
3.10 ✅ (V1.1) Corpus admin CRUD — the console can now create / delete corpora against the gateway (was local-only React state with no write route — POST /v1/corpora 405'd, so "created" corpora never persisted). CorpusStore SPI gains tenant-scoped create / delete (Noop + Pg; same cross-tenant invisibility rule as get); gateway POST /v1/corpora (201, server-minted corpus_… id, tenant from ctx) + DELETE /v1/corpora/{id} (204/404) + a CreateCorpusRequest wire type; admin-ui createCorpus/deleteCorpus wired into the Corpora page (live→local-demo gate, mirrors Webhooks). Update (PATCH) followed post-GA in #194 — the Edit action is now wired end-to-end (corpus + webhook). Verified locally (CI billing-blocked): full create→list→delete→404 + tenant isolation + CORS preflight/POST from :3100; pytest / ruff / mypy --strict / admin-ui tsc + vitest + OpenAPI·schema drift all green. #189
7.9b ✅ Admin-ui Operate pages — the console can now drive the request path, not just observe it (closes the three High-priority gaps from the v2 API-coverage analysis; built to the Operate design brief, 1:1 on the real wire types in rag_core.gateway_types / rag_core.openai_types). New Operate nav group + three pages: Query Playground (/playground — full QueryRequest control rail: corpus multiselect / top_k / rerank·pack·generate switches / advanced pack_budget·generate_max_tokens·generate_temperature·filters; Run query POST /v1/query vs Retrieve only POST /v1/retrieve; results render the decision strip (shape · bm25-fallback · corpus strategy · packed tokens · experiment tag), the answer with clickable [n] citation chips + guard verdict, citations, the ranked-chunks table (trust badges, detail sheet), a stacked per-stage timings bar with skipped-stage markers, and a feedback bar (👍/👎/1–5★/copy/regenerate/PII-redacted comment → POST /v1/feedback, exact signal enum); View full trace deep-links to /trace?rid=… — the seeded run reuses DEMO_REQUEST_ID so the demo resolves end-to-end); Ingest (/ingest, Knowledge — drag-and-drop multipart POST /v1/ingest/document, staged queue with progress → IngestResult chunk/embedding badges or error+Retry, the parse → chunk → enrich → PII-redact → embed → index explainer); Chat (/chat — OpenAI-compatible: streaming SSE thread with the non-standard rag annotations surfaced per reply (shape/chunks/citations/guard/finish_reason/usage; RAG on by default), an Embeddings tab (one input per line, float/base64, dim + L2 norm + sparkline), and a Models tab (GET /v1/models)). All follow the live→seed hybrid: action surfaces simulate without a gateway (deterministic worked query, token-streamed replies honouring prefers-reduced-motion). New Slider/Tabs primitives + CorpusMultiSelect/GuardBadge/CitedText shared components; identity body-carried on /v1/query·/v1/retrieve·/v1/feedback, header-driven elsewhere. Gates: tsc / eslint / 46 vitest (12 new) / next build (17 static routes); browser-verified light+dark, all three pages + the trace deep-link. reference/admin-ui.md
v1.0.1 ✅ (patch) — production serve path fixed end-to-end. The image's default CMD (a Phase-0 placeholder that printed the version and exited) now serves via the new config-driven rag_gateway.serve:create_app uvicorn-factory entrypoint: RAG_CONFIG_PATH set → build_app_from_config(load(path)), unset → noop build_app(), set-but-broken → fail fast (ConfigError), never a silent noop fallback. The image build itself was also broken — the full uv sync needs every workspace member on disk but sdks/python was never copied (Distribution not found); both stages now copy it. New public GET /readyz backs the Helm chart's readiness probe (liveness stays /healthz). The chart's ConfigMap previously rendered an invalid rag.yaml (env:/log_level:/service: against the strict extra='forbid' schema → guaranteed CrashLoop); it now renders a minimal valid document with a config.ragYaml passthrough for full platform configs (${VAR} env interpolation keeps secrets out of the ConfigMap), chart 0.2.0 / appVersion 1.0.1 (the old 0.1.0 appVersion pointed the default image tag at a tag that never existed), values.dev.yaml env: development → local. Docs: image tags are published without the v prefix — :v1.0.0-style pulls in README/release-notes never matched. release notes #185
7.10 ✅ GA cutover — v1.0.0 🎉 — the close of an 8-phase, 84-step build. Cut v1.0.0 with Semantic Versioning over the drift-gated public contracts (REST/OpenAPI · gRPC proto · rag.yaml · SDKs · ragctl) + a deprecation policy (ADR-0052); the rag-platform meta bumped to 1.0.0. GA release notes (v1.0.0) + a GA readiness checklist (ga-readiness-checklist) that maps every Phase 7 exit gate to a CI gate / in-repo artifact, with inherently-external items (external pentest, SOC 2 audit, registry first-publish, signed customers, on-call staffing, launch assets) marked (external) — the same in-repo-machine vs external-deliverable split used across Phase 7. The v1.0.0 tag fans out to every channel via docker.yml + release.yml + release-airgap.yml.
7.9 ✅ Billing & metering — per-tenant usage accounting, mirroring the 5.6c cost pattern (dataclasses in rag-observability, the gateway wraps a Pydantic response → no dist/schemas churn). New rag_observability.billing: a UsageMeter (observe-only per-tenant, per-dimension counters: queries / docs / storage / reranker / LLM tokens) fed O(1) from record_request_usage independent of quotas; GET /v1/billing/usage serves the per-tenant dashboard; pure generate_invoice + PricingModel/load_pricing price an invoice from marketplace/pricing.yaml (a price change is config) and reconcile with metered usage ±0.5% by construction; a BillingProvider Protocol seam for Stripe / marketplace metering (degrade-open). Inert by default (cfg.billing); ragctl billing drives meter→invoice→reconcile; no governed SPI call (policy-coverage linter unchanged); only dist/openapi + dist/rag.schema regenerate. Deferred: the concrete Stripe/marketplace adapters, the docs/reranker/storage feed points, an admin usage card. ADR-0051 #182
7.8 ✅ Support, SLAs & on-call — the support model + ops runbooks, grounded in the platform's own signals. Four support tiers + per-tier SLA targets (matching marketplace/pricing.yaml) in support-sla.md; a new docs/runbooks/ area with one runbook per alert type (alerts.md — acl/tenant escape · breaker.opened · drift.detected · cost anomaly · quota.exceeded · injection spike · gateway degraded · latency/availability SLO · eval.regression · ingest), each naming the exact event / /v1/status/* / SLO that raises it; an incident-response process + a blameless postmortem template; the status page reads measured health (/v1/status/health + Grafana SLOs), not manual toggles. Paging reuses the Step 3.9 webhook system (PagerDuty as a subscriber — no new code). Deferred: hosted status-page deploy, the PagerDuty account, staffing. ADR-0050 #181
7.7 ✅ Packaging & distribution — one-command install per channel + one tag fans out to every channel. New rag-platform PyPI meta-package (packaging/rag-platform/) pins the workspace component dists for a reproducible pip install rag-platform; the existing agentcontextos SDK scope is kept (no rename). A new release.yml publishes PyPI (OIDC Trusted Publishing) + npm (@agentcontextos/sdk, provenance) + the admin-ui image (new apps/admin-ui/Dockerfile, cosign + SBOM) + the Helm chart → OCI (cosign), alongside the existing docker.yml (gateway image, cosign + SBOM) and release-airgap.yml — all keyed off the same vX.Y.Z tag. Deferred: Go/Java/.NET SDK publish, worker/eval-runner images (single-process today). ADR-0049 #180
7.6 ✅ Marketplace listings — listing artifacts for AWS / Azure / GCP under a new marketplace/ (approval + procurement stay an external per-cloud process, like the 7.3 pentest). One canonical pricing.yaml (tiers + usage-metered dimensions whose ids match the Step 7.9 metering signals) + one shared listing-content.md mapped to all three clouds, so listings can't drift and prices reconcile with metering. Each per-cloud spec maps the model to that cloud's billing primitives and reuses existing delivery (Helm / AMI / air-gap bundle / GHCR — no new packaging); per-cloud submission checklists + a procurement acceptance test in the guide. ADR-0048 #179
7.5 ✅ Documentation site — a Docusaurus 3 app (website/) that serves the repo docs/ tree in place (single source of truth; markdown.format: 'detect' renders hand-written .md as CommonMark). The REST API reference (docs/reference/rest-api.md) is generated + drift-gated from dist/openapi.json by scripts/gen_api_reference.py (task docs:api) — never hand-edited. Doc honesty is a tests/docs/ suite (every ragctl / /v1/ reference in a quickstart must be real) + lychee offline link-check + codespell, wired into a new docs CI workflow. Deferred: Algolia search, versioned docs, the deploy workflow. ADR-0047 #178
7.4 ✅ Design-partner pilots — the pilot program shipped as a repo-backed machine (signing the referenceable partners is the external GTM deliverable, like the 7.3 pentest): an operator runbook + five fill-in templates (docs/pilots/), a customer-support/KB kit (docs/pilots/customer-support/ — sample corpus with synthetic PII + a planted injection probe; domain-calibrated criteria), and ragctl pilot (onboard renders the per-tenant config + checklist; report assembles a weekly-KPI dashboard + PASS/FAIL verdict from the platform's own feedback / drift / cost signal components — the same ones behind GET /v1/status/*). KPIs are pulled, not self-reported. A worked case study runs it end-to-end on the kit with real ragctl pilot report output (satisfaction +0.733, 0/5 drift, cost ok → PASS) + the PII-redaction + injection-blocked security demo. No dist/SPI/config change. #174
7.3 ✅ Red-team / security — turns the governance stack into an adversarial probe gate across four classes (prompt injection / PII egress / ACL bypass / tenant escape) under a redteam marker + a first-class redteam-gate CI job (task redteam). Closes the verified injection gap: new rag-injection package — a pluggable InjectionDetector (dependency-free HeuristicInjectionDetector, regexes anchored on attack grammar so benign prose isn't flagged) + PromptInjectionGuard.inspect that drops hijack chunks before the LLM, paired with INJECTION_RESISTANT_SYSTEM_PROMPT + build_user_message so untrusted context is fenced data in the user turn, never a system-trust position (fixes the OpenAI-chat surface that put context in a system message); wired on /v1/query + /v1/chat/completions + MCP; off by default (cfg.injection); PII-free injection.blocked event. Deterministic gate: a ≥ 500 known + ≥ 500 generated corpus (eval/redteam_v0/) hits 96.6 % block (≥ 95 % bar) at 0 false positives, plus the no-system-position invariant end-to-end — building the corpus hardened the detector (~20 missed phrasings). New PII-egress probe over PiiPolicyEngine (zero leakage, second-detector verified); pip-audit is the CVE gate; the external pentest is a documented process item. Injection types stay internal (no attacker signal, dist/schemas/openapi untouched). #166
7.2 ✅ Chaos engineering — a deterministic in-process kill-matrix gate (eval/gateway_chaos_v0/kill_matrix.py + tests/perf/test_chaos_kill_matrix.py) that extends 7.1 from the three retrieval backends to the full hot-path set (vector/keyword/graph/embedder/retrieval_cache/reranker/llm): kill each backend in turn (100% unavailable) behind the real breakers + fallback, drive /v1/query, and assert no single failure 5xx-es the gateway (no 5xx, on-path retrieval breaker opens, expected degraded shape; a seeded keyword corpus + real hydrate make rerank/generate actually run). Chaos fixed what it found — the matrix exposed that a down retrieval cache or reranker 5xx-ed, so the gateway gained two minimal degrade-open guards (gateway.cache.degraded → miss on /v1/query + /v1/retrieve; gateway.rerank.degraded → retrieval-only, honouring RerankPipeline's "caller decides" contract); LLM + embedder already degraded. LitmusChaos cluster manifests (infra/chaos/: gateway pod-delete + backend pod-network-loss/latency with httpProbe acceptance) as the cluster runbook + task chaos-kill. No dist//SPI/config change. #165
7.1 ✅ Load + chaos testing — a chaos-under-load CI gate (eval/gateway_chaos_v0/): drives the in-process gateway under concurrent load while injecting backend faults (FaultSpec + Chaos{Vector,Keyword,Graph}RetrievalBackend SPI wrappers behind real breakers) and asserts graceful degradation — no 5xx, 100% success, the failing backend's breaker opens (validating the Phase-4 breakers + fallback, builds no new resilience); tests/perf/test_chaos_under_load.py (timing-independent, perf-marked) + task chaos-test. An extended Locust v1 suite (weighted read/write mix + varied queries + a ramp LoadTestShape) + documented acceptance targets (≥ 1000 RPS sustained, e2e p99 < 500 ms) as a cluster runbook — #164
6.10 ✅ Compliance posture (Phase 6 capstone) — new rag-compliance package: RetentionEnforcer drives tenant-scoped purge_* (data retention + GDPR right-to-erasure) over the Feedback/Provenance stores (non-abstract purge_before/purge_tenant SPI capability with dry_run so a preview counts-without-deleting; audit chain never purged in place — its retention is the 6.6b WORM export); POST /v1/compliance/erase (tenant-scoped, dry-run default + two-flag delete); per-tenant data_region enforced at ingest (ResidencyViolationError → 403); GET /v1/status/compliance posture + compliance_posture/residency_ok; PII-free compliance.* events; cfg.compliance + tenants[].data_region/retention_days; ragctl compliance report/demo; SOC 2 + GDPR control-mapping docs backed by the live posture — #162
6.9 ✅ Air-gapped install bundle — ragctl airgap build/inspect/verify/install: one signed .tar.gz of all runtime images (docker save) + the packaged Helm chart + rag.yaml + a standalone installer, for networks with no registry / internet. Integrity = a standard SHA256SUMS (verifiable with sha256sum -c, no cosign/network) pinned as manifest.content_hash + an optional cosign signature over it (the 6.6b content-hash + optional-signature pattern); the same SHA256SUMS drives the Python verifier and the standalone install.{sh,ps1} (need only docker+helm). Typed/tested logic in ragctl.airgap (pure core + stubbable docker/helm/cosign seam; --dry-run = verifiable bundle minus blobs); digest-pinned manifest-driven image set (infra/airgap/images.txt); task airgap:*; release-airgap.yml on tags — #161
6.8 ✅ SSO / SCIM — rag-sso: FederatedAuth (an Auth SPI backend over per-tenant OidcProvider / SamlProvider) federates a bearer token / SAML assertion → Principal at the existing authenticate seam (group claims → acl_labels); dependency-free defaults (stdlib HS256 JWT + defusedxml SAML), asymmetric OIDC / XML-DSig behind [oidc] / [saml] extras; algorithm-allowlist downgrade defense; SCIM 2.0 ScimService over the new tenant-scoped ScimStore SPI driving /scim/v2/{Users,Groups} (per-tenant bearer token); per-tenant IdP on tenants[].sso; cfg.sso / cfg.scim; GET /v1/status/sso; PII-free sso.* / scim.* events; ragctl sso / scim — #159
6.7 ✅ BYOK envelope encryption — KeyManager SPI + EncryptingStorage + EnvelopeKeyManager (AES-256-GCM DEK + tenant AAD); LocalKeyManager + four cloud KMS providers (Aws/Gcp/AzureKeyVault/Vault, behind [kms-*] extras); cfg.kms + per-tenant key + factory; per-tenant isolation + sealing + tamper-evidence; zero-downtime rotation (RotatingKeyManager + rewrap); ragctl kms — #155–#158
6.6 ✅ Immutable audit log — 6.6a shared AuditWriter/store on app.state + GET /v1/audit (tenant-scoped, chain_verified) + GET /v1/audit/verify (whole-log) + cfg.audit.enabled (#153); 6.6b AuditExporter self-verifying WORM bundle (SHA-256 content_hash + HMAC, offline verify()), POST /v1/audit/export (tenant-scoped) + ragctl audit, cfg.audit.export_secret (#154)
6.5 ✅ PII egress policies — PiiPolicyEngine (rag-pii) answers egress_text over the context (list[Chunk]) + agent answer (str) the gateway already passes, applying the per-tenant pii_policy (allow / redact / mask / block); reuses the Step 1.7 detector + rewriters; opt-in cfg.pii.enabled; emits pii.egress_blocked — #152
6.4 ✅ ACL egress verifier — AclEgressVerifier re-checks returned ChunkRefs against the principal's labels at the gateway router boundary (defense-in-depth, same overlap semantics, independent of the PDP); acl.egress_violation event; red-team zero-violation-rate gate — #151
6.3 ✅ ACL push-down — opt-in AclPolicyEngine And-merges any_in("acl_labels", principal.acl_labels) into every read_chunk query (overlap, fail-closed); acl.egress_denied event — #150
6.2 ✅ Physical tenancy — per-tenant dedicated vector index/collection; TenantConfig.dedicated_index → ctx.physical_index → backend <base>-<key> (Noop/Pinecone/Qdrant); cross-tenant probe gate — #149
6.1 ✅ Logical multi-tenancy — TenantResolver resolves per-tenant rag.yaml config (namespace / PII / ACL labels) → TenantSettings applied at the gateway boundary; RequestContext.namespace; GET /v1/status/tenant; ragctl tenant — #148
5.7d ✅ Experiments console card + Phase-5 close-out — admin Live-Status A/B card over GET /v1/status/experiments (lift + CI per experiment) — #147
5.7c ✅ A/B routing — ABRouter deterministically serves the candidate to a fraction of users (variant-partitioned cache, ExperimentAssignment response tag) — #146
5.7b ✅ Shadow mode — observe-only candidate fan-out (ShadowRunner, background task) feeding the A/B tracker — #145
5.7a ✅ A/B analyzer + experiment tracker + GET /v1/status/experiments dashboard — #143
5.6 ✅ Status & Metrics GUI (full build) — drift/feedback/cost cards, query-trace viewer, regression bisector, Grafana dashboards, cross-links — #137–#142
5.5 ✅ Drift monitors — rag-drift, five PSI / mean-drop monitors — #136
5.4 ✅ Online metrics & feedback — rag-feedback — #135
5.3 ✅ CI eval gate — #134 · 5.2 ✅ Offline golden-set eval harness — #133 · 5.1 ✅ Per-query tracing & provenance — #132

Full per-step detail is in the phase sections below; each step links its ADR / reference / architecture docs.

Status Legend

Symbol	Meaning
✅	Complete — PR merged to `main`
🚧	In Progress — branch open, PR exists
⏳	Not Started
🔒	Blocked — depends on incomplete step

Milestone Summary

Phase	Title	Steps	✅ Done
0	Foundation	13	13
1	Ingestion + Knowledge Store	16	16
2	Retrieval Engine	11	11
3	Gateway & Agent Runtime	11	11
4	Reliability	6	6
5	Eval & Observability	7	7
6	Governance & Tenancy	10	10
7	Pilot, Harden, GA	10	10
Total		84	84

Phase 0 — Foundation (Weeks 0–4) ✅

Step	Title	Status	PR
0.1	Monorepo & build system	✅	#3
0.2	Core domain types & errors	✅	#4
0.2c	Cross-platform tooling	✅	#6
0.3	Plugin SPI	✅	#7
0.4	`rag.yaml` schema & loader	✅	#8
0.5	CI/CD pipeline hardening	✅	#9
0.6	Local dev stack (Docker Compose)	✅	#26
0.7	Tracing & metrics foundation	✅	#27
0.7b	Structured logging foundation	✅	#29, #30
0.7c	Audit log skeleton	✅	#31
0.8	Eval skeleton	✅	#33
0.9	IaC foundation	✅	#34
0.10	`ragctl` CLI scaffold	✅	#36

0.1 — Monorepo & build system ✅ #3

pyproject.toml, Makefile, .pre-commit-config, CI workflows
docker-compose, lockfiles
ADR-0001

0.2 — Core domain types & errors ✅ #4

types.py — 11 Pydantic v2 models · errors.py — 14-type exception hierarchy
gen_schemas.py — JSON Schema export · proto/core.proto

0.2c — Cross-platform tooling ✅ #6

Taskfile.yml — Windows / macOS / Linux task runner
scripts/bootstrap.ps1 + bootstrap.sh; CI matrix expanded to windows-latest

0.3 — Plugin SPI ✅ #7

16 SPI ABCs in rag_core/spi/: VectorStore, KeywordStore, GraphStore, Embedder, Reranker, LLM, Cache, Queue, Storage, Secrets, Auth, Telemetry, Connector, Parser, OCR, PIIDetector
Noop in-memory impls in spi/noop/; 79 conformance tests in tests/contract/

0.4 — `rag.yaml` schema & loader ✅ #8

JSON Schema (dist/rag.schema.json) + Pydantic v2 schema (packages/config/)
Env-var interpolation, ragctl config validate, polling hot-reload
91 fixture tests; 5 example configs

0.5 — CI/CD pipeline hardening ✅ #9

Path-filtered jobs, dependency caching, Docker layer cache
SBOM generation, cosign image signing, Dependabot

0.6 — Local dev stack (Docker Compose) ✅ #26

Health-check scripts, seed-data loader, make dev-reset
pgvector schema migration, Qdrant collection init

0.7 — Tracing & metrics foundation ✅ #27

OTel SDK wired, TraceContext propagation
SpiMetrics counter + histogram, Grafana provisioned dashboards, ragctl traces

0.7b — Structured logging foundation ✅ #29, #30

Shared logger package + event registry (events.py); RAG001 pre-commit hook + ruff T201
Schema + PII CI gates in tests/logs/ (50 tests; log-gates job blocking)
Follow-up: extracted packages/observability/ (rag-observability) — 7-field JSON schema (ts, level, service, module, msg, env, version), set_log_context() contextvar manager; rag_core reduced to back-compat shims (391 tests)

0.7c — Audit log skeleton ✅ #31

AuditStore SPI (append / events / verify_chain) + NoopAuditStore (SHA-256 hash chain)
AuditWriter facade (store + structured log); 14 conformance tests

0.8 — Eval skeleton ✅ #33

rag_core.eval types (GoldenSample, EvalMetrics, EvalReport) + rag_config.eval metrics (recall@k, MRR, citation_precision)
RagasAdapter spike, ragctl eval run/show, 5-sample golden JSONL, tests/eval/ harness (39 tests)

0.9 — IaC foundation ✅ #34

Terraform modules for Postgres/pgvector, Redis, Qdrant, Elasticsearch (Kubernetes-native, Helm provider)
rag-platform Helm chart (Deployment, Service, ConfigMap, ServiceAccount, HPA, PDB, Ingress); dev + prod envs
task infra:* + task helm:*; ADR-0003

0.10 — `ragctl` CLI scaffold ✅ #36

packages/ragctl/ (Typer 0.12+), root ragctl entry point
Working config / eval / traces / version groups; scaffolded ingest / query / logs / tenant / plugin / secret
Shell completion; 19 tests; reference/ragctl.md + guides/ragctl-quickstart.md

Phase 1 — Ingestion + Knowledge Store (Weeks 4–8) ✅

Refactor window (1.1a–1.1f): before resuming connectors, a six-step refactor locked in architecture decisions expensive to retrofit — PolicyEngine PDP, RequestContext-threaded SPIs, split Retrieval/Index backends, bulk + streaming + ID-only methods, Pipeline + Batcher primitives, three-way cache split, hot-path discipline.

Step	Title	Status	PR
1.1	Storage backends	✅	#40
1.1a	Core type & SPI refactor	✅	#44
1.1b	SPI split — Retrieval/Index, bulk + streaming + ID-only	✅	#45
1.1c	PolicyEngine package	✅	#46
1.1d	Pipeline + Batcher primitives	✅	#50
1.1e	Cache SPI split + perf discipline + async telemetry	✅	#51
1.1f	ADRs 0005–0009 + reviewer checklist	✅	#52
1.2	Connectors framework	✅	#55
1.3	Document parsers	✅	#57
1.4	OCR pipeline	✅	#60
1.5	Structure-aware chunker	✅	#62
1.6	Metadata enricher	✅	#64
1.7	PII detection	✅	#66
1.8	Embedder pipeline	✅	#68
1.9	CDC connectors (incremental sync)	✅	#70
1.10	Write path & ingest API	✅	#72

1.1 — Storage backends ✅ #40

rag-backends: PgVectorStore (asyncpg + pgvector, ivfflat), QdrantVectorStore (query_points), RedisCache, S3Storage (aioboto3, MinIO-compatible), LocalFileStorage
Integration tests (skip-if-no-service); MinIO added to dev stack; task test-integration + test-backends; ADR-0004

1.1a — Core type & SPI refactor ✅ #44

RequestContext frozen model threaded through every SPI (always first arg)
Typed tenant_id + acl_labels on Chunk/Embedding (not metadata dict); trust_level on Chunk (injection defense); dtype on Embedding
BlobRef (lazy chunk text), QueryPlan, ChunkRef, Cost, PlanNode, typed StageEvent
spi_signature.py linter (ctx-first); backends migrated; ADR-0005 / 0007 / 0008 / 0009

1.1b — SPI split — Retrieval/Index, bulk + streaming + ID-only ✅ #45

Split Vector/Keyword/Graph into *RetrievalBackend (read) + *IndexBackend (write) composites
retrieve_ids → list[ChunkRef]; hydrate on the retrieval side
Bulk bulk_index / bulk_delete (+ graph variants); stream_index async-iterator default; Embedder split single embed + bulk_embed
IndexHint + WriteVolume passed to writes (ADR-0009)

1.1c — PolicyEngine package ✅ #46

rag-policy v0.1.0: PolicyEngine SPI + NoopPolicyEngine (always-ALLOW + tenant-scoped filter_pushdown)
PolicyDecision enum (read_chunk / ingest_doc / egress_text / quota_check / rate_limit / execute_plan); PolicyResult (allow/deny/transform)
FilterExpr mini-language (Eq / AnyIn / And / Or / Not / TrueExpr); PolicyWriter facade → policy.decision logs
Coverage linter tests/policy/coverage.py (greps governance SPI calls w/o PDP consultation); 20 tests; ADR-0005 finalized

1.1d — Pipeline + Batcher primitives ✅ #50

Pipeline (rag_core.pipeline) — async DAG, bounded queues, per-stage workers, backpressure, on_error fail/skip
Batcher[Req, Resp] (rag_core.batcher) — DataLoader coalescing, size + time triggers, per-request Future isolation, flush()
PEP-695 generics; 22 unit tests

1.1e — Cache SPI split + perf discipline + async telemetry ✅ #51

EmbeddingCache / RetrievalCache / AnswerCache SPIs (ctx-first, version-partitioned, invalidation hooks) + noop impls (18 tests)
AsyncTelemetrySink (rag-observability) — bounded buffer, non-blocking submit(), drop + error counters (9 tests)
Hot-path discipline doc in performance.md (hot/cold call-site table)

1.1f — ADRs 0005–0009 + reviewer checklist ✅ #52

ADR-0005 (PolicyEngine PDP), 0006 (two-stage reranker), 0007 (tiered storage + BlobRef), 0008 (cost-aware planner), 0009 (vector index + quantization) → Accepted
Reviewer checklist extended; module-level ADR backlinks on reranker + storage

1.2 — Connectors framework ✅ #55

Connector SPI → (Document, ConnectorState) async-iterator with resumable watermark
Internal Crawler[Raw] base; built-in FilesystemConnector, S3Connector, GCSConnector ([gcs] extra)
Connectors upstream of PolicyEngine (ingest pipeline is the enforcement point); 18 tests

1.3 — Document parsers ✅ #57

rag-parsers: 10 parsers — text, markdown, HTML, JSON, CSV, YAML, PDF, DOCX, PPTX, XLSX
Parser SPI → ParsedDocument (text + structural Block list + BlockType); detect_mime(); ParserRegistry
ragctl parse; 148 contract tests total

1.4 — OCR pipeline ✅ #60

rag-ocr: TesseractOCR (per-word) + PaddleOCRBackend (per-line) behind [tesseract] / [paddle] extras
BoundingBox / OCRRegion / OCRResult frozen models (length-weighted confidence)
ragctl ocr; 27 tests (engines stubbed at sys.modules)

1.5 — Structure-aware chunker ✅ #62

rag-chunker: HeadingAwareChunker — heading-level stack for parent_id, token budget (512/64 overlap), sentence-boundary splitting, never bleeds across headings
Chunker SPI + NoopChunker; TokenCounter Protocol + TiktokenCounter; OCR→ParsedDocument adapter
ragctl chunk; 40 tests

1.6 — Metadata enricher ✅ #64

rag-enricher: DefaultEnricher tags language, doc_type, created/modified, author, title, section_path, reading_level (Flesch-Kincaid)
Enricher SPI; LanguageDetector Protocol + seeded LangdetectDetector
ragctl enrich; 34 tests

1.7 — PII detection ✅ #66

rag-pii: RegexPIIDetector (EMAIL/PHONE/SSN/CREDIT_CARD/IP, Luhn-validated; shares definition with the log-leak gate) + PresidioPIIDetector ([presidio])
PiiProcessor — redact (default) / mask / block; pii.detected event carries metadata only (never raw text)
PIISpan promoted to frozen model; ragctl pii; 48 tests

1.8 — Embedder pipeline ✅ #68

rag-embedders: OpenAIEmbedder, CohereEmbedder, SentenceTransformersEmbedder (BGE + E5) — 4 model families behind Embedder SPI
BatchingEmbedder shared sub-batch split + retry (RetryPolicy) + dimension normalize
ragctl embed (noop default, no creds); 46 tests

1.9 — CDC connectors (incremental sync) ✅ #70

PostgresCDCConnector (pgoutput WAL LSN cursor, tombstones), S3EventsConnector (SQS long-poll, SNS-wrapped), WebhookReceiverConnector (push, backpressure modes)
DedupFilter (wraps any Connector + Cache, drops by Document.fingerprint); DedupStats
35 tests (SDKs stubbed)

1.10 — Write path & ingest API ✅ #72

🥈 CLI demo milestone — Phase 1 complete.
rag-ingest: IngestPipeline — parse → chunk → enrich → PII → embed → index (PII before embed so vectors never encode PII); PolicyEngine.evaluate(ingest_doc) short-circuit; per-doc error isolation
POST /v1/ingest/document (multipart) + build_app(pipeline=) DI; ragctl ingest wired for real
IngestPipeline is the canonical ingest_doc PDP consumer; 38 tests

Phase 2 — Retrieval Engine (Weeks 8–12) ✅

Step	Title	Status	PR
2.1	Knowledge store read layer	✅	#74
2.2	Vector retrieval backends	✅	#76
2.3	BM25 / keyword retrieval	✅	#82
2.4	Graph retrieval	✅	#84
2.5	Hybrid RRF fusion	✅	#86
2.6	Query understanding	✅	#90
2.7	Cross-encoder reranker	✅	#92
2.8	Context packer	✅	#94
2.9	GraphRAG	✅	#96
2.10	Retrieval router	✅	#98
2.11	Agent-loop validation spike	✅	#100

2.1 — Knowledge store read layer ✅ #74

FilterExpr AST + in-memory evaluate() relocated to rag_core.filter so SPI signatures take it directly (rag_policy.filter re-exports for back-compat)
retrieve_ids switched to typed FilterExpr | None on Vector + Keyword backends; new signature gate
Noop stores apply the predicate via evaluate() (conformance oracle); pgvector _filter_sql.translate() + acl_labels TEXT[] GIN column; Qdrant _filter_qdrant.translate(); unsupported fields raise (no silent push-down loss)
75 tests; rag-core 0.13 → 0.14; architecture/retrieval-read-layer.md

2.2 — Vector retrieval backends ✅ #76

WeaviateVectorStore ([weaviate]), PineconeVectorStore ([pinecone]), ElasticsearchDenseVectorStore ([elasticsearch]) — each with its own FilterExpr translator + tenant isolation + acl_labels push-down
All five vector backends consume IndexHint via shared select_index_variant() (ADR-0009 size/recall → ivfflat / HNSW / PQ)
Lazy SDK imports; 53 tests; reference/backends.md, architecture/vector-backends.md

2.3 — BM25 / keyword retrieval ✅ #82

ElasticsearchKeywordStore ([elasticsearch]) + TantivyKeywordStore ([tantivy]) on the Step 2.1 read-layer contract (no new SPI)
Per-field boosting is a backend config knob (field_boosts), not a per-call param; identical indexed-field schema across both (swap = config change)
_filter_es / _filter_tantivy translators (tantivy Not paired with all_query); 47 tests; architecture/keyword-backends.md

2.4 — Graph retrieval ✅ #84

Neo4jGraphStore ([neo4j]), MemgraphGraphStore ([memgraph]), NetworkXGraphStore ([networkx]); Neo4j + Memgraph share BoltCypherStore
New expand(ctx, seed_ids, hops, rel_types, direction, node/edge_filter, limit) → list[NeighborResult] (non-abstract default; NoopGraphStore BFS is the oracle)
FilterExpr push-down on edges (every hop — the ACL case) + final node; _filter_cypher.translate() with param_prefix; rel-type injection guard; tenant isolation on every edge
65+ tests; NeighborResult schema; architecture/graph-backends.md

2.5 — Hybrid RRF fusion ✅ #86

New rag-retrieval package. Pure rrf_fuse(rankings, weights, k=60, top_k) — rank-only Σ w/(k+rank), earliest-source metadata, cross-tenant raise
HybridRetriever — parallel vector + keyword + graph fan-out (asyncio.gather(return_exceptions=True)), per-call source skip, partial-failure tolerance (only all-fail raises)
Canonical read_chunk PDP call site via policy_engine.filter_pushdown And-merged with the caller filter; HybridWeights; GraphAdapter protocol
ragctl hybrid; 43 tests; reference/retrieval.md, architecture/hybrid-fusion.md

2.6 — Query understanding ✅ #90

New rag-query package — four components + a parallel orchestrator
LLMQueryRewriter; LLM / Heuristic / Noop QueryDecomposer (conjunction + sentence splits, [] sentinel); HyDEGenerator (Gao 2022, optional embedder); InMemoryGlossary + expander
UnderstoodQuery frozen output (rewrites / sub_queries / expansion_terms / hyde_*); QueryUnderstandingPipeline fan-out, per-component degrade to identity
query.understand span; ragctl understand; 60 tests; reference/query.md, architecture/query-understanding.md

2.7 — Cross-encoder reranker ✅ #92

New rag-reranker package — ADR-0006 two-stage cascade. Reranker SPI → fast_rerank / precise_rerank / should_early_exit; Chunk.score added (rag-core 0.14 → 0.15)
Pure mmr_rerank() (Carbonell-Goldstein, cosine cache, stable at λ=0); RerankPipeline — fast → early-exit? → hydrate → precise → optional MMR; strict failure (RerankerError(stage=))
Backends: SentenceTransformersCrossEncoder (BGE v2-m3 / MiniLM), CohereReranker, JinaReranker (lazy extras)
rerank span; ragctl rerank; 92 tests; architecture/reranking.md, reference/reranker.md

2.8 — Context packer ✅ #94

New rag-packer package. ContextPacker — tenant check → sort by score → hash dedup → opt-in Jaccard dedup → greedy token-budget fit (TokenCounter / TiktokenCounter, Chunk.token_count fast-path) → reorder (lost_in_middle default, Liu 2024) → heuristic conflict scan (trust-mismatch + numeric-disagreement)
PackedContext + ConflictAnnotation frozen models (rag-core 0.15 → 0.16); strict PackerError (budget ≤ 0 / mixed-tenant / content_ref w/o token_count); conflict scan is the one degrade-and-log path
pack span; ragctl pack; 78 tests

2.9 — GraphRAG ✅ #96

New rag-graphrag package: LouvainDetector (default) + LeidenDetector ([leiden]) community detection over GraphRetrievalBackend (no new SPI)
CommunitySummarizer (LLM strict-JSON, NoopLLM fallback), InMemoryCommunityStore (tenant-scoped), graphrag_adapter (entity-node → ChunkRef)
GraphRAGRetriever — set-overlap community scoring (2× on key_entities) → seed collection → expand() fan-out → returns ChunkRef for HybridRetriever
Community / CommunitySummary (rag-core 0.16 → 0.17); 3 graphrag.* spans; ragctl graphrag; 54 tests; reference/graphrag.md, architecture/graphrag.md

2.10 — Retrieval router ✅ #98

RetrievalRouter + pure classify_shape (KEYWORD_HEAVY / SEMANTIC / ENTITY_RICH / MIXED) layered on HybridRetriever (rag-retrieval 0.1 → 0.2)
In-memory BackendHealthTracker (rolling-window, injected clock); shape-bias weight table; vector-input priority (caller > HyDE > embed > skip)
BM25-only fallback when vector + graph are down; failure-feedback loop (record_failure / record_success) converges degradation without an external loop
RoutingDecision + QueryShape (rag-core 0.17 → 0.18); retrieve.route span; ragctl route; 42 tests; reference/router.md, architecture/retrieval-router.md

2.11 — Agent-loop validation spike ✅ #100

AgentLoopV0 (rag-retrieval 0.2 → 0.3) — thin iterative-retrieve orchestrator: per-iter budget enforcement, sticky routing, plan-reuse measurement, shared EmbeddingCache, sub-query fan-out; cross-package isolation via QueryUnderstander / UnderstoodQueryLike Protocols
StopReason / BudgetSpend / CacheStats / AgentLoopError (rag-core 0.18 → 0.19); agent_loop.run span
50-query harness in eval/agent_loop_v0/ (5 buckets). Headline finding: 0% embedding-cache hit on the decomposable bucket → drove the G-01 must-fix
ragctl agent-loop; 66 tests; reference/agent-loop.md, spikes/agent-loop-v0-gaps.md
Pre-3.1 must-fix window (#101–#105): G-02 split route → decide + execute · G-03 reject zero budgets · G-06 cache HyDE + rewriter · G-01 SemanticEmbeddingCache (hit rate 28% → 54%, ADR-0010) · G-04 per-stage CostEstimator (ADR-0008)

Phase 3 — Gateway & Agent Runtime (Weeks 12–18) ✅

Step	Title	Status	PR
3.1	Gateway service (REST + OpenAPI)	✅	#107
3.2	gRPC service	✅	#109
3.3	MCP server	✅	#111
3.4	OpenAI-compatible endpoints	✅	#113
3.5	Corpus router	✅	#115
3.6	Agent loop	✅	#120
3.7	Official SDKs	✅	#121
3.8	Framework adapters	✅	#122
3.9	Webhooks	✅	#123
3.10	Admin UI v0	✅	#124
3.11	Status & Metrics GUI (MVP)	✅	#125

3.1 — Gateway service (REST + OpenAPI) ✅ #107

🥈 Curl-able RAG milestone. rag-gateway 0.3 → 0.4: POST /v1/query (full RAG → QueryResponse), POST /v1/retrieve, GET /v1/corpora + /{id} (new CorpusStore SPI + NoopCorpusStore)
rag-core 0.19 → 0.20 wire types in rag_core.gateway_types (Corpus, QueryRequest/Response, RetrieveRequest/Response, Answer, StageTimings, GatewayError)
Middleware builds per-request RequestContext (X-Request-Id / traceparent / Bearer); domain errors → HTTP (403 / 401 / 429 / 502 / 400) via GatewayError envelope; PolicyEngine.egress_text before every LLM call
build_app(...) injectable factory (noop defaults, zero infra); OpenAPI 3.1 + Swagger / ReDoc; ragctl query; 43 tests; reference/gateway.md, guides/curl-quickstart.md

3.2 — gRPC service ✅ #109

proto/rag.proto — rag.gateway.v1.RagService: server-streaming Query (QueryStreamEvent oneof) + unary Retrieve / IngestDocument / ListCorpora / GetCorpus; health + reflection co-registered (grpcurl works)
Python stubs via grpcio-tools + mypy-protobuf (committed, drift-gated); task proto:lint / gen / check-drift on full OS matrix
GrpcRagService reuses the same Phase 2 composition as REST (build_grpc_server(...)); error → StatusCode mapping mirrors REST; GatewayError packed on trailing metadata
ragctl grpc-query; 48 tests; ADR-0011, reference/grpc.md

3.3 — MCP server ✅ #111

New rag_gateway.mcp — FastMCP over the official mcp SDK (stdio); three tools query / retrieve / ingest
ToolBackend Protocol is the topology seam: InProcessBackend shares one process wiring with REST + gRPC; a future GatewayClientBackend slots in (ADR-0012)
@ragplatform/mcp npm launcher (pure-Node bin → python -m rag_gateway.mcp, cross-platform interpreter discovery, no injection surface)
ragctl mcp-query / mcp-serve; 21 tests; ADR-0012, reference/mcp.md

3.4 — OpenAI-compatible endpoints ✅ #113

rag_gateway.openai_compat: POST /v1/embeddings, POST /v1/chat/completions, GET /v1/models — drop-in for OpenAI SDKs / LangChain / LlamaIndex
Retrieval pre-fetch: chat runs the Phase 2 pipeline over the last user turn, egress-policed, injected as a system message; on by default (rag.enabled); RagOptions request field + RagAnnotations response field
stream=true → OpenAI-faithful chat.completion.chunk SSE; base64 embeddings; header identity w/ NoopAuth dev-anon fallback; OpenAI {error:{...}} envelope
rag-core 0.20 → 0.21 openai_types; ragctl chat / embeddings; 36 tests; ADR-0013, guides/openai-quickstart.md

3.5 — Corpus router ✅ #115

CorpusRouter sits above the Step 2.10 RetrievalRouter — it answers which corpora a query touches. decide() precedence: explicit pin → static rules → learned classifier → all (auto-detects effective strategy); UNCONSTRAINED preserves pre-3.5 behaviour
route() fan-out (default): one RetrievalRouter.route per corpus, rrf_fuse weighted by corpus score; StaticRuleClassifier (auditable) + dependency-free LearnedCorpusClassifier (softmax over term_weights) behind CorpusClassifier Protocol
CorpusRoutingStrategy / CorpusScore / CorpusRoutingDecision (rag-core); corpus_decision on Query/Retrieve responses; Postgres PgCorpusStore (lazy); config-driven build_app_from_config in gateway wiring (keeps the config → core boundary)
corpus.route span + audit event; ragctl corpus list/seed/route; 35 tests; ADR-0014, reference/corpus-router.md

3.6 — Agent loop ✅ #120

New rag-agent package (rag-core only) graduates the 2.11 spike. AgentLoop is an explicit state machine (planning → acting → observing / finalizing) driven by a pluggable Controller; stream / run / resume
Four nullable budget dims spent per step (stop checked top-of-step); ScriptedController / HeuristicController (creds-free) / LLMController; RetrieveTool (recoverable RetrievalError); resumable via frozen AgentSnapshot + CheckpointStore
rag_core.agent_types; surfaces: REST POST /v1/agent (SSE) + gRPC RagService/Converse (streaming) sharing one flat event frame + egress helpers; final-answer egress-policed once at the boundary
agent.run / grpc.converse spans; ragctl agent; full unit + gRPC + proto-compat suites; ADR-0015, reference/agent.md

3.7 — Official SDKs ✅ #121

Committed, drift-gated OpenAPI 3.1 (dist/openapi.{json,yaml} via export_openapi.py; openapi-drift CI job)
Hand-written flagships: Python agentcontextos (sync + async, SSE, reuses rag-core models; 32 tests vs a live gateway) + TypeScript @agentcontextos/sdk (zero-dep fetch / ReadableStream, Node + browser; 10 vitest)
Generated on demand: Go / Java / .NET via task sdk:gen (openapi-generator + buf generate); generated source gitignored, configs committed
ts-sdk CI job; gateway 0.5 → 0.7; ADR-0016, reference/sdks.md

3.8 — Framework adapters ✅ #122

agentcontextos.integrations (in sdks/python) — 8 integrations wrapping the SDK Client: LangChain, LlamaIndex, Haystack, DSPy, LangGraph, CrewAI, AutoGen, Semantic Kernel
Shared _common.py runs /v1/query (pack / generate off) → maps Chunk to framework docs / tool strings; each lazy-imports its framework behind an optional extra with a friendly missing_dependency
13 tests vs real framework base classes; integrations CI job; ADR-0017, reference/integrations.md

3.9 — Webhooks ✅ #123

New rag-webhooks package (rag-core + httpx): WebhookDispatcher (tenant fan-out + HMAC signing + backoff retries + span / log), HttpWebhookSender, signing.py (HMAC-SHA256 over <ts>.<body>, verify() with replay tolerance)
rag-core 0.23 → 0.24 webhook_types + SubscriptionStore / WebhookSender SPIs + noops; emission via the WebhookPublisher seam (no emitter depends on rag-webhooks); ingest ingest.completed, PolicyWriter audit.policy_violation; drift.detected / eval.regression types reserved for Phase 5
Gateway make_webhooks_router() (CRUD + /test), secret returned once then masked; WebhooksConfig in rag.yaml; ragctl webhooks demo; ~50 tests; ADR-0018, reference/webhooks.md

3.10 — Admin UI v0 ✅ #124

Next.js 14 (App Router) + TS + Tailwind operator console (apps/admin-ui) to a committed design handoff; hand-rolled shadcn-style primitives (Radix-free, no CLI) on shadcn CSS-var tokens + Geist + lucide — deterministic offline builds
App shell (collapsible sidebar, tenant switcher, theme toggle) + 9 pages: Dashboard, Corpora, Connectors, Glossary, Webhooks (one-time secret reveal + send-test), Audit Log (filters + pagination), API Keys (Preview), Tenants (Preview), Config Viewer
Live-vs-seed hybrid (lib/use-live.ts): Corpora + Webhooks hit the gateway when NEXT_PUBLIC_GATEWAY_URL is set, else seed data + a "Demo data" badge
14 vitest + RTL; tsc / lint / build clean; admin-ui CI job; ADR-0019, reference/admin-ui.md

3.11 — Status & Metrics GUI (MVP) ✅ #125

rag-observability read-side: MetricsCollector (counter / gauge / histogram, reservoir p50/p95/p99, snapshot()) + LogTail + RingBufferLogHandler (poll by seq cursor)
rag_gateway.status (make_status_router): REST /v1/status/health (per-component roll-up from real error rates) / metrics / logs / connectors/status; SSE logs stream; WebSocket health + metrics push (ADR-0020)
Request-timing middleware feeds the metrics (status surface excluded); CORS for the console; build_app injectables; gateway 0.7 → 0.8
Admin Observability nav: Live Status (WS) / Metrics (WS) / Logs (SSE) + Dashboard banner; 45 tests; ADR-0020, reference/status-api.md · full 12-page GUI is Step 5.6

Phase 4 — Reliability (Weeks 18–22) ✅

Step	Title	Status	PR
4.1	Semantic cache (L0/L1/L2)	✅	#126
4.2	Fallback chain	✅	#127
4.3	Hallucination guard	✅	#128
4.4	Circuit breakers	✅	#129
4.5	Quotas & rate limiting	✅	#130
4.6	Latency tuning & load test	✅	#131

4.1 — Semantic cache (L0/L1/L2) ✅ #126

New rag-cache package: TtlLruCache + L0 (InMemory{Retrieval,Answer,Embedding}Cache), L2 similarity (Semantic{Retrieval,Answer}Cache + QuerySimilarityIndex), Tiered{Retrieval,Answer}Cache (read-through + promotion, write-through, cache.* events + TierStats)
L1 Redis{Retrieval,Answer}Cache in rag-backends (tag-set invalidation), injected; reliability: L1 failures degrade, never raise
Gateway /v1/{query,retrieve} consult the cache (served_from_cache); CacheConfig tiered knobs; cross-tenant red-team suite
ADR-0021, reference/cache.md

4.2 — Fallback chain ✅ #127

rag_retrieval.fallback.FallbackChain wrapping RetrievalRouter — degradation ladder hybrid → BM25-only → keyword → "no answer"
Triggers: proactive budget (CostEstimator start-tier, ADR-0008) + reactive RetrievalError / empty-below-min_results; graceful no_answer (HTTP 200, no_answer_on_exhaustion toggle); keyword rung relaxes caller filters only (ACL pushdown preserved)
SupportsRoute drop-in (CorpusRouter relaxed) + build_app default wrap; FallbackTier / FallbackTrigger / FallbackResult; retrieve.fallback span + fallback.engaged event
ragctl fallback; 39 tests; architecture/fallback-chain.md, reference/fallback.md

4.3 — Hallucination guard ✅ #128

New core-only rag-guard package: HallucinationGuard (claim split → per-claim NLI vs. evidence → per-tenant threshold → annotate / redact / block); NLIScorer SPI (mirrors Reranker) + NoopNLIScorer + dependency-free LexicalNLIScorer default
Frozen types NLILabel / NLIScore / GuardVerdict / GuardAction / ClaimVerdict / GuardResult + GuardError; runs after egress_text on /v1/query + OpenAI chat (sync block ⇒ content_filter; stream advisory) + MCP query; Answer.guard + RagAnnotations.guard
Disabled by default; reliability degrade-never-raise (guard.scorer_degraded); guard.check span + PII-safe guard.claim_blocked event
cfg.guard; ragctl guard; ADR-0022, reference/guard.md

4.4 — Circuit breakers ✅ #129

New core-only rag-breaker package: CircuitBreaker (3-state closed / open / half-open, consecutive-failure trip + timed half-open probe, injected clock), BreakerRegistry (per-backend, force-close)
Breaker{Vector,Keyword,Graph}RetrievalBackend SPI wrappers — an open breaker raises CircuitOpenError, which HybridRetriever's gather(return_exceptions=True) already drops (zero fan-out changes); generalises the 2.10 health tracker
CircuitState / BreakerSnapshot + CircuitOpenError; PII-free breaker.opened event; no per-call span; on by default
Gateway GET /v1/status/breakers + POST .../{name}/force-close + health roll-up; admin-ui Live Status breakers card; cfg.breakers; ragctl breaker; ADR-0023, reference/breaker.md

4.5 — Quotas & rate limiting ✅ #130

New rag-quota package enforcing per-tenant quotas through the PolicyEngine PDP (ADR-0005): QuotaPolicyEngine (answers rate_limit / quota_check) over a QuotaEnforcer (5 dims — QPS + monthly tokens / cost / queries + storage gauge; micro-dollar cost; snapshot / reset)
QuotaStore SPI — weighted two-slot sliding-window InMemoryQuotaStore + atomic-Lua RedisQuotaStore (rag-backends, injected) + NoopQuotaStore; QuotaVerdict / QuotaSnapshot / QuotaDimension + QuotaExceededError(RateLimitError)
Wired on every surface via build_app(quota_enforcer=) + quota_guard (REST query / retrieve, OpenAI chat + embeddings, ingest storage); 429 + Retry-After pre-check, post-response metering. Disabled by default (can reject); fail-open by default (quota.store_degraded)
GET /v1/status/quotas + reset + health component; admin-ui Quotas card; ragctl quota; ~82 tests + 2 vitest; ADR-0024, reference/quota.md

4.6 — Latency tuning & load test ✅ #131

Gateway overhead p99 ≤ 30 ms gate (default noop backends → per-request wall time is the gateway's own cost; distinct from the backend-bound end-to-end target); gate signal is the server-side gateway.request_duration_ms from the 3.11 MetricsCollector
Driven in-process via httpx ASGITransport, sequentially, retry-tolerant; dedicated single-runner perf-gate CI job (excluded from the cross-OS sweep via -m "not perf")
One reusable engine rag_gateway.perf (measure_gateway_overhead + profile_gateway) powers gate + harness + ragctl perf; tests/contract/budgets.py made real (drift-guarded); load test = in-process eval/gateway_load_v0 + Locust locustfile.py
~14 tests; ADR-0025, reference/perf.md, architecture/latency.md

Phase 5 — Eval & Observability (Weeks 22–28) ✅

Step	Title	Status	PR
5.1	Per-query tracing & provenance	✅	#132
5.2	Offline eval harness (golden set)	✅	#133
5.3	CI eval gate	✅	#134
5.4	Online metrics & feedback	✅	#135
5.5	Drift monitors	✅	#136
5.6	Status & Metrics GUI (full build)	✅	#137–#142
5.6a	— Drift & Feedback console cards	✅	#137
5.6b	— Query Trace viewer	✅	#138
5.6c	— Cost anomaly	✅	#139
5.6d	— Regression bisector	✅	#140
5.6e	— Grafana dashboards	✅	#141
5.6f	— Cross-links + close-out	✅	#142
5.7	A/B testing & shadow mode	✅	#143–#147
5.7a	— A/B analyzer + tracker + dashboard	✅	#143
5.7b	— Shadow mode	✅	#145
5.7c	— A/B routing	✅	#146
5.7d	— Console + close-out	✅	#147

5.1 — Per-query tracing & provenance ✅ #132

Stable span attributes: rag.schema_version stamped at the span_from_trace_context choke point + a telemetry_attrs registry/contract
HMAC-signed ProvenanceRecord — new rag-provenance package, privacy-by-default hashes, degrade-open
GET /v1/query/{id}/trace — signed record + TraceCollector-captured span tree, tenant-scoped
ADR-0026, reference/provenance.md, architecture/per-query-tracing.md

5.2 — Offline eval harness (golden set) ✅ #133

500 queries / 5 domains (committed tests/eval/golden/, drift-gated) run through the real HybridRetriever over a synthetic corpus (noop SPIs + HashingEmbedder, deterministic)
Metrics: Recall@k / MRR / nDCG@k / Faithfulness (dependency-free lexical_faithfulness, RAGAS optional) / Citation Precision — overall + per-domain + per-difficulty
render_html_report HTML + report.json / report.md; eval/golden_set_v0/ harness + --check threshold floor; ragctl eval run --html
ADR-0027, reference/eval-harness.md, architecture/golden-set-eval.md

5.3 — CI eval gate ✅ #134

Standalone always-run eval-gate.yml runs the harness, compares vs committed tests/eval/baselines/main.json (compact byte-stable EvalBaseline)
Enforces thresholds.yaml floors + regression deltas (recall@10 / mrr / faithfulness gate; acl / pii are the red-team suites'); sticky-marker PR comment with a diff table (even on failure, fork-PR tolerant)
Comparison in rag_config.eval (compare_to_baseline / render_gate_comment); result types in rag_core.eval; runner python -m eval.golden_set_v0.gate
ADR-0028, architecture/ci-eval-gate.md

5.4 — Online metrics & feedback ✅ #135

New rag-feedback package: FeedbackRecorder (normalise signal → [-1,1] score → PII-redact comment via an injected PIIDetector → tenant-scoped FeedbackStore.put → feedback.recorded; degrade-open) + pure aggregate_feedback → FeedbackStats
One polymorphic POST /v1/feedback (explicit + implicit FeedbackSignal, body identity, redact-don't-hash, acks stored=false when inert / degraded) + per-tenant GET /v1/status/feedback dashboard
FeedbackRecord / FeedbackStats + FeedbackRequest / FeedbackAck; cfg.feedback; ragctl feedback
ADR-0029, reference/feedback.md, architecture/online-feedback.md

5.5 — Drift monitors ✅ #136

New rag-drift package: population_stability_index (pure binned PSI) + DriftMonitor (ref / current windows, PSI or mean-drop, tri-state verdict, rebaseline) + DriftMonitorRegistry (five monitors)
Monitors: query distribution / embedding PSI / retrieval score (via PSI) + citation-clickthrough / faithfulness (via mean-drop); infra-scoped, fed via observe from the query + feedback paths
evaluate emits drift.detected (event + Step 3.9 webhook) on the transition into drift; DriftSnapshot / DriftReport; cfg.drift; GET /v1/status/drift + POST .../{metric}/rebaseline; ragctl drift
ADR-0030, reference/drift.md, architecture/drift-monitors.md

5.6 — Status & Metrics GUI (full build) ✅ #137–#142

Delivered in six vertical slices (5.6a–f), detailed below: Drift / Feedback + Cost cards on Live Status, the Query Trace viewer page, the regression bisector, drift + cost Grafana dashboards, observability cross-links
Deferred (documented): feedback / breaker / quota Grafana export, a Loki-events dashboard, per-domain eval gating

5.6a — Drift & Feedback console cards ✅ #137

Surface GET /v1/status/drift (per-monitor verdict + one-click rebaseline) and GET /v1/status/feedback (mean score, sentiment split, signal counts) as two Live Status cards (useLive live-vs-seed)
New TS types + fetchDrift / rebaselineDrift / fetchFeedback + seeds; 3 vitest; browser-verified

5.6b — Query Trace viewer ✅ #138

New Query Trace page (/trace): paste a request_id → GET /v1/query/{id}/trace renders the signed provenance record, an HMAC verification pill, evidence citations, and the captured span tree as an offset / duration waterfall
Input-driven fetch + seed fallback + 404 path; QueryTrace / SpanRecord / Provenance* types + fetchTrace; 3 vitest

5.6c — Cost anomaly ✅ #139

New CostTracker (rag-observability) — bounded per-tenant rolling window of recent per-request token costs; two-gate (ratio + z-score) tri-state verdict; scale-free on tokens (decoupled from quota pricing); fed O(1) from record_request_usage, independent of quotas
Pull-based GET /v1/status/cost (no per-request span / event); CostSnapshot a dataclass → no rag-core / dist/schemas churn; admin Cost-anomaly card; 16 tests + 1 vitest
ADR-0031, reference/cost.md, architecture/cost-anomaly.md

5.6d — Regression bisector ✅ #140

python -m eval.golden_set_v0.bisect --good <ref> --bad HEAD --metric <m> --threshold <floor> — binary-search good..bad (O(log n) harness runs) for the first commit that dropped a metric
Each candidate runs in a throwaway git worktree with that commit's packages/*/src on PYTHONPATH; pure bisect_commits → BisectResult / BisectStep (additive); 7 unit tests + live worktree smoke
reference/eval-harness.md

5.6e — Grafana dashboards ✅ #141

rag_observability.register_platform_metrics registers OTel observable gauges — rag.drift.{statistic,threshold,drifted} (from report()) + rag.cost.{ratio,elevated,recent_mean_tokens} (from CostTracker.snapshot_all) — on the existing OTel → collector → Prometheus pipeline; registered once by build_app_from_config
Provisioned rag-quality-cost Grafana dashboard; proven by an OTel InMemoryMetricReader test + a dashboard-validation gate (tests/infra/test_grafana_dashboards.py)
guides/grafana-dashboards.md

5.6f — Cross-links + close-out ✅ #142

Wires the observability surfaces together: the Trace viewer's Logs button → /logs?trace=<id> (clearable violet trace-chip), a log detail drawer → View this trace's logs, a Trace quick-link on Live Status
Admin-ui only; 1 vitest cross-link spec; closes Step 5.6

5.7 — A/B testing & shadow mode ✅

Final Phase-5 step, delivered in vertical slices 5.7a–d: analyzer + tracker + dashboard (5.7a ✅), shadow mode (5.7b ✅), A/B routing (5.7c ✅), console + close-out (5.7d ✅)
Shadow N% of live queries; A/B variant routing; statistical analyzer with lift + confidence interval — all delivered; Phase 5 complete

5.7a — A/B analyzer + tracker + dashboard ✅ #143

Pure analyze_ab_experiment (rag_config.eval) — lift + normal-approx Welch CI + significance via stdlib statistics.NormalDist (no numpy / scipy) → ABAnalysisResult (rag_core.eval, additive, not in dist/schemas)
ABExperimentTracker (rag-observability, beside CostTracker) — a pure per-(experiment, variant) sample holder, decoupled from the analyzer (observability keeps no rag_config dep); the gateway composes the two at GET /v1/status/experiments
cfg.experiments (opt-in, since A/B routing can change responses); wired inert by default; observe-only — no query-path change yet (shadow / routing are 5.7b/c); ragctl experiments; 17 tests
ADR-0032, reference/experiments.md

5.7b — Shadow mode ✅ #145

New ShadowRunner (rag_gateway.experiments) runs a candidate retriever alongside the served control path on a sampled fraction of live queries — observe-only — and feeds both variants' outcome_metric (mean retrieval score) into the 5.7a ABExperimentTracker
Never delays the response: scheduled as a FastAPI BackgroundTask (runs after the response is sent); degrade-open (experiment.shadow_failed); skipped on a retrieval-cache hit
Deterministic request_id-hash sampling; candidate = any SupportsRoute (config builds a RetrievalRouter differing only in shadow_candidate RRF weights; production injects one via build_app(shadow_runner=…)); same read_chunk PDP — no coverage-linter entry
cfg.experiments gains shadow_enabled / shadow_sample_rate / shadow_experiment / shadow_candidate; doubly opt-in (enabled and shadow_enabled); ragctl shadow; 27 tests
ADR-0032, reference/experiments.md, architecture/ab-shadow-mode.md

5.7c — A/B routing ✅ #146

New ABRouter (rag_gateway.experiments, beside ShadowRunner) — deterministic hash(request_id) < routing_sample_rate variant assignment that serves the candidate retrieval config to the assigned fraction of users (the first slice that can change a response); assign / is_candidate / candidate / record / tag
Served-path wiring in /v1/query + /v1/retrieve: the candidate arm is routed inline via _route_query (one retrieval, no added latency); not degrade-open (a candidate failure → 502, never a silent fallback to control that would corrupt the experiment); the served arm's outcome_metric is recorded once per fresh retrieval (skipped on a cache hit)
Variant-partitioned retrieval cache — compute_plan_hash / compute_params_hash fold in the assigned variant, so control and candidate never share a cache entry (byte-identical key when routing is off)
New frozen ExperimentAssignment (experiment / variant / is_candidate) on the QueryResponse / RetrieveResponse experiment field (additive, dist/schemas + dist/openapi regenerated; REST-only — gRPC proto mirror deferred like corpus_decision); served variant also on the gateway.query_complete log
Same read_chunk PDP site (no coverage-linter entry); gated separately by cfg.experiments.routing_enabled (requires enabled) — independent of shadow_enabled — since routing can change a response; ShadowCandidateConfig generalised to CandidateConfig (shared by shadow_candidate + new routing_candidate); build_app(ab_router=…) inject seam; ragctl ab drives assign → serve → record → analyze; ~25 new tests; all gates green (ruff, mypy --strict, RAG001, schema/openapi-drift, proto-compat, policy-coverage)
ADR-0032 (5.7c update), reference/experiments.md, architecture/ab-routing.md

5.7d — Console + close-out ✅ #147

New A/B experiments card on the admin Live Status page (apps/admin-ui) surfacing GET /v1/status/experiments — one row per running experiment with the lift (relative, colour-coded), a verdict badge (candidate wins / candidate worse / no change / collecting), control→candidate means, the confidence interval, and per-arm sample counts; live-vs-seed via useLive (fetchExperiments + seedExperiments), demo badge when no gateway
ABAnalysis TS type mirrors rag_core.eval.ABAnalysisResult; admin-ui gates green (tsc / eslint / 34 vitest / next build); browser-verified both the significant-win + insufficient-data states render
Phase-5 close-out: Step 5.7 ✅ → Phase 5 complete (7 / 7); deferred items remain documented (per-tenant drift / per-dimension embedding PSI; feedback/breaker/quota Grafana export + Loki-events dashboard; gRPC proto mirror of corpus_decision + experiment; sequential / multi-metric experiments)
reference/experiments.md, reference/admin-ui.md

Phase 6 — Governance & Tenancy (Weeks 28–34) ✅

Step	Title	Status	Planned deliverables
6.1	Logical multi-tenancy	✅	#148 — `TenantResolver` → `TenantSettings` (namespace / PII / ACL labels) applied at the boundary; `RequestContext.namespace`; `GET /v1/status/tenant`; `ragctl tenant`
6.2	Physical tenancy (dedicated index)	✅	#149 — `dedicated_index` → `ctx.physical_index` → backend `<base>-<key>` (Noop/Pinecone/Qdrant, lazy create); cross-tenant probe gate
6.3	ACL push-down at retrieval	✅	#150 — opt-in `AclPolicyEngine` And-merges `any_in("acl_labels", …)` into every `read_chunk` push-down (overlap, fail-closed); `acl.egress_denied`
6.4	ACL egress verifier	✅	#151 — `AclEgressVerifier` re-checks returned chunks at the gateway router boundary (defense-in-depth above the 6.3 push-down); `acl.egress_violation`; zero-violation-rate red-team gate
6.5	PII policies	✅	#152 — `PiiPolicyEngine` egress_text decorator (allow / redact / mask / block per tenant) over answer + context; reuses Step 1.7 detector; `pii.egress_blocked`
6.6	Immutable audit log	✅	6.6a #153 — read API `GET /v1/audit` + `GET /v1/audit/verify` + shared store + `cfg.audit`. 6.6b #154 — `AuditExporter` signed WORM bundle + `POST /v1/audit/export` + `ragctl audit`
6.7	BYOK (Bring Your Own Key)	✅	6.7a ✅ #155 — `KeyManager` SPI + envelope encryption (`LocalKeyManager`) + `EncryptingStorage` + sealing/isolation/tamper. 6.7b ✅ #156 — `cfg.kms` + per-tenant key + `build_key_manager_from_config` factory + `AwsKmsKeyManager`. 6.7c ✅ #157 — `GcpKmsKeyManager` / `AzureKeyVaultKeyManager` / `VaultKeyManager` behind `[kms-]` extras. 6.7d* ✅ #158 — `RotatingKeyManager` zero-downtime rotation + `rewrap`
6.8	SSO / SCIM	✅	#159 — `rag-sso`: `FederatedAuth` (`Auth` SPI backend over per-tenant `OidcProvider` / `SamlProvider`) federates a token / assertion → `Principal` at the existing `authenticate` seam (groups → `acl_labels`); stdlib HS256 + `defusedxml` defaults, asymmetric OIDC / XML-DSig behind `[oidc]` / `[saml]` extras; SCIM 2.0 `ScimService` + `ScimStore` SPI → `/scim/v2/*` (per-tenant bearer); `tenants[].sso` + `cfg.sso` / `cfg.scim`; `GET /v1/status/sso`; `ragctl sso` / `scim`
6.9	Air-gapped install bundle	✅	#161 — `ragctl airgap build`/`verify`/`install`: signed `.tar.gz` of all images (`docker save`) + Helm chart + config + standalone installer; `SHA256SUMS` content-hash (offline-verifiable) + optional cosign; `infra/airgap/images.txt`; `install.{sh,ps1}`; `task airgap:*`; `release-airgap.yml`
6.10	Compliance posture	✅	#162 — `rag-compliance`: `RetentionEnforcer` (data retention + GDPR erasure via `purge_` SPI capability, `dry_run`) + `POST /v1/compliance/erase`; per-tenant `data_region` enforced at ingest (403); `GET /v1/status/compliance` posture; `cfg.compliance`; PII-free `compliance.`; `ragctl compliance`; SOC 2 + GDPR mapping docs

6.1 — Logical multi-tenancy ✅ #148

Makes per-tenant rag.yaml config actually drive requests (it was a passive registry before; only quotas were per-tenant). New TenantResolver (rag_config.tenancy) maps a tenant id → frozen TenantSettings (rag_core.types: namespace / pii_policy / acl_labels), merging the tenants: entry with safe defaults
Resolved once, at the boundary: the request-context middleware (build_gateway_context) applies the resolved settings — namespace + pii_policy onto the RequestContext, the tenant's acl_labels unioned into the principal's; the resolved view is also exposed on request.state.gateway.tenant_settings
Unknown tenants resolve to safe defaults (namespace = the id, default PII redact, no labels) — isolated, never privileged
New RequestContext.namespace (defaults to tenant_id via a before-validator, so every pre-6.1 call site is unchanged); it's a backend-partition primitive — Pinecone now partitions on ctx.namespace — not a chunk field, so filter_pushdown is unchanged (chunks carry tenant_id)
TenantConfig gains namespace + acl_labels (→ rag.schema); TenantSettings joins dist/schemas + RequestContext regenerated; GET /v1/status/tenant (→ dist/openapi) surfaces the resolved view; ragctl tenant list / resolve <id> print it; inert by default in build_app, built from cfg.tenants by build_app_from_config
Scope: resolution + threading only — ACL push-down enforcement is 6.3, PII egress is 6.5, physical tenancy is 6.2; ~21 new tests (resolver, config, gateway, cross-tenant red-team, CLI); all gates green (ruff, mypy --strict 293 files, RAG001, schema/openapi-drift, proto-compat, policy-coverage)
ADR-0033, reference/tenancy.md, architecture/multi-tenancy.md

6.2 — Physical tenancy (dedicated index) ✅ #149

Logical tenancy (6.1) shares one index + a tenant_id filter; physical tenancy gives a tenant its own vector index/collection, so its data is separate even if a filter were bypassed. Opt-in per tenant via TenantConfig.dedicated_index (+ optional dedicated_index_name)
Same resolve-once-at-the-boundary pattern as 6.1: the resolver computes a physical_index key (dedicated_index_name or tenant_id, else None) on TenantSettings, threaded onto new RequestContext.physical_index; the gateway middleware applies it
Backends read only ctx (graph is backends→core, never rag-config), so the decision arrives via ctx.physical_index; each namespaces its base under the key (<base>-<key>) and lazily creates the dedicated index/collection on first use, falling back to the base index when None. Implemented for Noop (CI oracle, partitioned by physical_index) + Pinecone (dedicated index) + Qdrant (dedicated collection); pgvector/ES/Weaviate follow later
Cross-tenant probe gate (tests/redteam/test_cross_tenant_dedicated_index.py, CI, Noop) proves a dedicated tenant's data is invisible to another independent of the tenant filter (a matching tenant_id pointed at a different index still returns nothing); live backends are integration-tested
One instance + per-tenant derivation (no per-tenant backend instances, no SPI change); GET /v1/status/tenant + ragctl tenant resolve report dedicated_index / physical_index; TenantConfig fields → rag.schema, TenantSettings + RequestContext regenerated, /v1/status/tenant → dist/openapi; ~30 new/updated tests; all gates green
ADR-0034, reference/tenancy.md, architecture/multi-tenancy.md

6.3 — ACL push-down at retrieval ✅ #150

Enforces label-based ACLs at retrieval: 6.1 resolved each tenant's acl_labels onto the principal, but nothing enforced them. New AclPolicyEngine (rag-policy) — a decorator like QuotaPolicyEngine — And-merges any_in("acl_labels", sorted(principal.acl_labels)) into every read_chunk push-down; a chunk is retrievable only when its labels overlap the principal's
Overlap via the existing AnyIn predicate — the semantics the codebase already assumed and every backend translator already speaks (pgvector &&, Qdrant MatchAny, Cypher ANY(), noop evaluate), so zero new predicate / translator changes. Injected at the canonical HybridRetriever read_chunk PDP site (already merges filter_pushdown), so no new call site / coverage-linter entry
Fail-closed: the clause is uniform — any_in([]) matches nothing, so a label-less principal retrieves nothing (model "public" as a shared label granted to all); each label-less request emits one PII-free acl.egress_denied event (pre-registered EVT_ACL_DENIED)
Opt-in via new cfg.acl.enabled (default off → pre-6.3 behaviour, tenant scoping only); build_app(acl_enabled=…) wraps the engine, build_app_from_config from config. Decorates without absorbing the inner engine (composes with a production PDP)
AclConfig → rag.schema; ~17 new tests incl. an end-to-end ACL red-team through HybridRetriever (overlap / disjoint / fail-closed / public-label / no-tenant-bypass) + engine unit + event + wiring; all gates green (ruff, mypy --strict, RAG001, schema-drift, policy-coverage, log-schema)
Scope: push-down enforcement only — post-retrieval egress re-verification is 6.4, graph edge ACLs deferred. ADR-0035, architecture/policy-engine.md, reference/tenancy.md

6.4 — ACL egress verifier ✅ #151

The 6.3 push-down filters ACLs at the source (one layer, re-implemented by each backend translator); 6.4 adds the defense-in-depth second layer that the tests/policy/coverage.py linter docstring already anticipated. New AclEgressVerifier (rag_policy.egress) — verify(ctx, refs) keeps only the returned ChunkRefs whose acl_labels overlap the principal's, dropping the rest — so a filter-translation bug, a backend that ignores the predicate, or a path wired without the policy engine cannot leak an over-privileged chunk past the boundary
Identical overlap semantics to the push-down (same ∩ ≠ ∅, same fail-closed / "public is a shared label" model), so it's a no-op on a correctly-filtered result and only ever acts on a genuine leak. Reads ChunkRef.acl_labels (which every backend populates regardless of the filter applied) → no re-hydration, an O(results) set-intersection per call (hot-path-safe)
Independent of the PDP — it consults only ctx.principal.acl_labels, so it backstops the push-down even when the push-down isn't wired. Layered at the retrieval router boundary: a thin SupportsRoute wrapper (AclEgressVerifyingRouter, in the gateway so rag-retrieval keeps its no-rag-policy property) decorates app.state.retrieval_router — the single attribute every surface reads — so query / retrieve / corpus / OpenAI / agent all inherit it, a layer above the HybridRetriever where the push-down merges
Opt-in, on-by-default behind ACL: new cfg.acl.verify_egress (default true) gated on cfg.acl.enabled — turning ACLs on gives both layers; set it false to run the push-down alone (the strict drop is meaningless without the label model, so it never fires for a non-ACL deployment). build_app(acl_verify_egress=…) / build_app_from_config wire it; app.state.acl_verify_egress reports the effective state
acl.egress_violation (pre-registered EVT_ACL_EGRESS_VIOLATION, error level, PII-free: tenant / principal / counts / dropped chunk ids only) fires once per call that drops ≥1 ref — the unexpected push-down failure, distinct from 6.3's expected acl.egress_denied; a clean pass is silent
Scope: gateway retrieval surfaces (everything reading app.state.retrieval_router); trusts the labels the backend reports on each ChunkRef (catching a mislabelling backend needs authoritative re-hydration — deferred), per-tenant/per-label violation metrics deferred to the 6.x governance dashboards. AclConfig → rag.schema regenerated; ~21 new tests incl. a red-team zero-violation-rate gate (tests/redteam/test_acl_egress_verifier.py — bypassed push-down + leaky backend → verifier drops every violation) + verifier unit + gateway wiring + event-schema; all gates green (ruff, mypy --strict 296 files, RAG001, schema-drift, policy-coverage, log-schema). ADR-0036, architecture/policy-engine.md, reference/tenancy.md

6.5 — PII policies ✅ #152

Step 1.7 detects + redacts PII at ingest; Step 6.1 resolved each tenant's pii_policy onto RequestContext.pii_policy; but nothing enforced it at egress — the egress_text PDP decision existed and the gateway already called it at every generation boundary, yet the default engine answered allow. New PiiPolicyEngine (rag-pii) — a decorator like QuotaPolicyEngine / AclPolicyEngine — answers egress_text and applies the per-tenant action: allow delegates, redact / mask → transform, block → deny
Handles both subject shapes the gateway already passes — the retrieved context (list[Chunk] → the LLM on /v1/query?generate / /v1/chat/completions / gRPC Converse) and the agent's final answer (str) — so enforcement lands at all four existing egress_text call sites with no route change and no coverage-linter entry (the existing deny→drop / transform→substitute handling consumes the result unchanged)
Reuses Step 1.7 machinery: detection via the injected PIIDetector SPI (default the dependency-free RegexPIIDetector; production injects Presidio), rewriting via the same redact_spans / mask_spans, and the same min_score + entities filter as the ingest PiiProcessor — so the two stages never disagree, and a clean scan is a no-op. block on a chunk list is all-or-nothing (any PII denies the whole context); redact/mask rewrite each affected chunk's content (clean chunks pass through, same object)
Lives in rag-pii (which gains a rag-policy dep, exactly as rag-quota does for QuotaPolicyEngine; graph stays acyclic — rag-policy never imports rag-pii). filter_pushdown / health / non-egress_text decisions delegate inward, so it composes with the ACL + quota PDP
Opt-in via new cfg.pii.enabled (default off — it can redact / withhold content); build_app(pii_enabled=…, pii_detector=…) wraps the engine after the acl wrap, build_app_from_config from config; app.state.pii_enabled reports the state. Events are PII-free: pii.egress_blocked (block, the alertable denial — the reserved event from Step 0.7b now emitted) / pii.detected (redact/mask) carry entity types + counts + surface, never the matched values
Scope: enforces at the existing egress_text sites (context for query/OpenAI/gRPC, answer for the agent). Deferred: a post-generation answer re-check for the query/OpenAI/gRPC generate paths (they sanitise the context pre-LLM) and a PII check on retrieval-only citations — stored chunks are already ingest-sanitised (1.7). PiiConfig → rag.schema regenerated; ~24 new tests (engine unit: str/chunks × allow/redact/mask/block, min_score/entities, events, delegation; gateway wiring + composition with ACL + behavioral redact/block); all gates green (ruff, mypy --strict 297 files, RAG001, schema-drift, policy-coverage, log-schema/PII gate). ADR-0037, architecture/pii.md, architecture/policy-engine.md, reference/pii.md

6.6 — Immutable audit log 🚧 (sliced 6.6a + 6.6b)

Step 0.7c shipped the foundation — an append-only AuditStore SPI, a NoopAuditStore that links events into a SHA-256 hash chain, and an AuditWriter facade — but the log was write-only (only the corpus router wrote corpus.route, into a store nothing could read or verify). Step 6.6 makes it usable + provably intact, delivered as two vertical slices.

6.6a — Audit read API + chain verification ✅ #153

One shared audit store. build_app now creates a single AuditWriter over a hash-chain store, exposes app.state.audit_store / audit_writer / audit_enabled, and hands the same writer to the corpus router — so the events the writers append land in exactly the store the read API serves (before this, the corpus router held a private NoopAuditStore the gateway couldn't read). New AuditWriter.store property is the read accessor; default store is the in-memory NoopAuditStore (creds-free chain), production injects a durable one
GET /v1/audit — tenant-scoped (a principal sees only its own tenant's events, filtered by ctx.tenant_id at the boundary — same pattern as GET /v1/query/{id}/trace), newest-first, bounded by limit (clamped [1,1000]), optional action / outcome filters. The AuditListResponse carries chain_verified so one read both returns the events and attests the log is untampered
GET /v1/audit/verify — whole-log hash-chain integrity (the chain is one global sequence across tenants), returning {ok, event_count} — a boolean + total, no event content, so global verification leaks nothing cross-tenant
The hash chain is the tamper-evidence mechanism (no second scheme): verify_chain() recomputes every link and fails if any event or stored hash was altered. Immutability at rest (preventing deletion/replacement of the store) is the 6.6b WORM export's job
Read API on by default: new cfg.audit.enabled (default true) — unlike the behaviour-changing ACL / PII toggles, the audit log is a passive, tenant-scoped compliance record, so exposing it out of the box is the expected enterprise default; off → endpoints 404 (AuditNotFoundError), events still recorded. New AuditListResponse / AuditVerifyResponse wire types (rag_core.gateway_types); AuditError / AuditNotFoundError (→ 404); dist/openapi + dist/rag.schema regenerated
Scope: read + verify only (WORM signed export is 6.6b); today the populated event is corpus.route (every query) — expanding audit coverage (ACL / PII / ingest decisions) is a follow-up, the surface + chain are in place for it. ~10 gateway tests (tenant-scoped list, cross-tenant isolation, verify ok + tamper detection, filters/pagination, disabled→404, no-auth→401, shared-store wiring); all gates green (ruff, mypy --strict 298 files, RAG001, schema/openapi-drift, policy-coverage, log-schema). ADR-0038, architecture/audit-log.md, reference/audit.md

6.6b — WORM signed export ✅ #154

The hash chain (6.6a) gives tamper-evidence for the live log; 6.6b adds immutability at rest via a self-verifying signed export. New AuditExporter (rag_core.audit_export, pure stdlib) builds an AuditExport bundle with two independent integrity checks: a content_hash (SHA-256 over the canonical serialisation of the events — pins the exact set + order, recomputed on verify so any add/remove/edit is caught even without the secret) and a signature (HMAC-SHA256 over f"{timestamp}.{content_hash}", mirroring the Step 5.1 ProvenanceSigner scheme). verify() returns {signed, content_ok, verified, reason} with reason ∈ ok / unsigned / no_secret / content_mismatch / signature_mismatch; chain_verified attests the source store's whole-log chain was intact at export time
POST /v1/audit/export is tenant-scoped (the caller's events only — same isolation as GET /v1/audit); ragctl audit exports the whole log in-process. Because integrity is content_hash + HMAC over whatever events are included (not the global chain head), the same verify() works for both — a tenant subset isn't a contiguous chain. The bundle verifies offline (no live store), so it's the artifact an operator archives to immutable storage (S3 Object Lock)
Opt-in signing: new cfg.audit.export_secret (env-interpolated, mirrors provenance.signing_secret); empty (default) → content-hashed but unsigned bundle (verify → unsigned), set it to enable signature verification. build_app(audit_export_secret=…) → app.state.audit_exporter; build_app_from_config from config
New AuditExport / AuditExportSignature / AuditExportVerification core types (dist/schemas regenerated); POST /v1/audit/export → dist/openapi; ragctl audit (seed → export → verify round-trip, --out writes the bundle, --verify FILE checks one offline). ~23 new tests (exporter unit: content_hash determinism, sign→verify, unsigned, no_secret, wrong-secret, content tamper → content_mismatch, signature tamper → signature_mismatch, empty; gateway: tenant-scoped + signed + offline-verifies + cross-tenant isolation + unsigned + disabled→404 + no-auth→401; ragctl round-trip + file verify). All gates green (ruff, mypy --strict 299 files, RAG001, schema/openapi-drift, policy-coverage, log-schema). ADR-0038, architecture/audit-log.md, reference/audit.md
Deferred: a durable live-store AuditStore backend (Postgres / append-only ledger) and expanding what gets audited beyond corpus.route.

6.7 — BYOK (Bring Your Own Key) 🚧 (sliced 6.7a + 6.7b + 6.7c)

New ground — the only prior crypto was HMAC signing. The V1 plan calls for envelope encryption with per-tenant data keys wrapped by a customer-controlled KMS KEK, zero-downtime rotation, and sealing (KEK unavailable → tenant data unreadable). Delivered in slices.

6.7a — Envelope encryption library + local KMS ✅ #155

What's encrypted: chunk content / blobs at rest — not embedding vectors (ANN search reads vectors directly, so encrypting them would break retrieval; vectors carry no raw text). A deliberate, documented constraint
KeyManager SPI (rag-core) over opaque bytes — encrypt(ctx, plaintext) → bytes / decrypt(ctx, ciphertext) → bytes; ciphertext is a self-describing envelope. NoopKeyManager (passthrough) for wiring tests. EncryptingStorage (rag-core, crypto-free) is a Storage decorator that encrypts on put / decrypts on get — the application seam over any backend. EncryptionError + KeyUnavailableError (sealing)
EnvelopeKeyManager base (rag-backends, cryptography) does the data-key half once for every provider: a fresh AES-256-GCM DEK per payload (with ctx.tenant_id bound as AAD), the small DEK handed to a subclass to wrap/unwrap with the KEK. LocalKeyManager wraps with an in-process per-tenant KEK (dev / tests / air-gapped); cloud providers (6.7b) subclass and wrap/unwrap via the KMS API so the KEK never leaves the customer
Per-tenant isolation via KEK-per-tenant and tenant-bound AAD (even a shared KEK can't read another tenant's blob); sealing via KeyUnavailableError (a tenant with no KEK is denied, others unaffected); tamper-evidence via the GCM auth tag. cryptography in rag-backends; rag-core stays crypto-free (SPI + decorator + noop)
Scope: library + local KMS only. Deferred: cloud KMS providers behind [kms-*] extras + wiring EncryptingStorage into the ingest/storage path + per-tenant key config (6.7b); zero-downtime key rotation (6.7c); vector encryption is out of scope by design. ~19 new tests (KeyManager contract over noop + local; envelope round-trip / non-determinism / tamper / malformed / cross-tenant / shared-KEK-still-blocked / sealing on encrypt+decrypt / wrong-size KEK / per-tenant non-impact / EncryptingStorage stores ciphertext + delegates key ops) + ragctl kms smoke; all gates green (ruff, mypy --strict 305 files, RAG001, policy-coverage, log-schema; no dist drift). ADR-0039, architecture/byok.md, reference/encryption.md

6.7b — Cloud KMS (AWS) + config / factory / wiring ✅ #156

Makes BYOK configurable + provider-selectable. New cfg.kms (enabled / provider ∈ noop/local/aws / default_key_id / region / local_key) + tenants[].kms_key_id (per-tenant key reference — a KMS ARN for aws, a hex 32-byte KEK for local). build_key_manager_from_config(cfg) factory (in the gateway, lazy rag-backends imports so rag-backends stays config-free) selects the provider; build_app(key_manager=…) exposes app.state.key_manager + app.state.kms_enabled (passthrough NoopKeyManager when disabled)
AwsKmsKeyManager (rag-backends/kms/aws.py) subclasses EnvelopeKeyManager and wraps/unwraps the DEK via AWS KMS Encrypt/Decrypt for the tenant's customer-managed key (over the already-present aioboto3 — no new dep). Per-tenant CMK from the key-id map (+ default_key_id); any KMS failure (revoked / denied / unreachable) or a missing key id → KeyUnavailableError (sealing). An injectable client seam makes it fully unit-testable with a fake KMS — no AWS creds / network
Scope: AWS provider + config + factory + the app.state.key_manager seam. Deferred: GCP KMS / Azure Key Vault / HashiCorp Vault providers + key rotation (6.7c); wiring EncryptingStorage into the ingest path (blocked on tiered-storage plumbing — chunk content is inline today). ~14 new tests (AWS provider over a fake KMS: round-trip / KMS-key-bound cross-tenant block / sealing / default-key / KMS-error mapping; config defaults; factory noop/local/aws; gateway app.state.key_manager + kms_enabled + a local round-trip). KmsConfig → rag.schema regenerated; all gates green (ruff, mypy --strict 306 files, RAG001, policy-coverage, log-schema). ADR-0039, reference/encryption.md, architecture/byok.md

6.7c — GCP / Azure / Vault KMS providers ✅ #157

Completes the four-provider KMS matrix — each subclasses EnvelopeKeyManager (inheriting the DEK + AES-GCM + tenant-AAD + envelope framing) and only wraps/unwraps the DEK via its KMS API: GcpKmsKeyManager (google-cloud-kms encrypt/decrypt), AzureKeyVaultKeyManager (azure-keyvault-keys wrap_key/unwrap_key, RSA-OAEP-256; a CryptographyClient per key via a cached factory since Azure binds a client to one key), VaultKeyManager (HashiCorp Vault Transit encrypt_data/decrypt_data, sync hvac run in a thread, mount_point)
Each is behind a [kms-gcp] / [kms-azure] / [kms-vault] extra (the SDKs are lazy-imported, so the modules import without the SDK and selecting a provider without its extra raises a clear ImportError); an injectable client / client_factory seam makes every provider fully unit-testable with a fake KMS — no cloud creds / network. Connection uses each SDK's standard credential discovery (GCP ADC, Azure DefaultAzureCredential, Vault VAULT_ADDR/VAULT_TOKEN); per-tenant key resolution + sealing (KeyUnavailableError) are uniform with AWS
KmsProvider enum + build_key_manager_from_config factory + KmsConfig.vault_mount extended. mypy overrides added for the three SDK module trees (absent at lint time). Scope: providers only — key rotation is 6.7d; wiring EncryptingStorage into the ingest path stays deferred (tiered storage). ~18 new tests (each provider over a fake client: round-trip / cross-tenant block / sealing / SDK-error mapping; factory cloud-provider-requires-extra contract). KmsConfig → rag.schema regenerated; all gates green (ruff, mypy --strict 309 files, RAG001, policy-coverage, log-schema). ADR-0039, reference/encryption.md, architecture/byok.md

6.7d — Zero-downtime key rotation ✅ #158

Closes Step 6.7. New RotatingKeyManager (rag-core, a crypto-free KeyManager decorator) makes rotating a tenant's KEK seamless: encrypt always uses the current key; decrypt tries the current key then each non-expired retired key (RetiredKey(key_manager, expires_at)). The try-all decrypt is safe, not a guess — AES-GCM authenticates the DEK, so a wrong KEK can't yield a valid key (only the KEK that wrapped a blob decrypts it)
rewrap(ctx, ct) is the background-migration primitive (decrypt with whatever key still works → re-encrypt under the current key); once all blobs are migrated a retired key is dropped. An expired retired key is skipped, so old un-rewrapped data is sealed (EncryptionError) — "retain decrypt-only until expiry" — with no impact on current-key data. Injectable clock for deterministic expiry tests
Composes with every provider (local + all four cloud KMS) since it orchestrates only the KeyManager SPI. ragctl kms --rotate demos the full flow (old + new decrypt, rewrap, expiry seals old data)
Scope: the rotation mechanism (satisfies the planning rotation acceptance — rotate → old + new both decrypt; expired key seals old data). Deferred: config-driven per-tenant multi-generation rotation + the storage-side background re-encryption job land with the EncryptingStorage ingest wiring (tiered storage). ~11 new tests (rotate → old+new decrypt; rewrap migration; expired-key sealing; tamper; no-key sealing; multi-generation; clock-controlled expiry; health) + ragctl kms --rotate. No dist drift; all gates green (ruff, mypy --strict 310 files, RAG001, policy-coverage, log-schema). ADR-0039, reference/encryption.md, architecture/byok.md

6.8 — SSO / SCIM ✅ #159

The seam is the Auth SPI. The gateway middleware already calls auth.authenticate(bearer_token, tenant_id) → Principal at the boundary (the one SPI method that runs before a RequestContext exists). New FederatedAuth (rag-sso) implements it, dispatching to a per-tenant OidcProvider / SamlProvider, so wiring it as the gateway's auth backend is the entire integration — no middleware change. The returned principal's acl_labels come from the IdP's group claims, so Step 6.3 ACL push-down + 6.5 PII egress govern federated users unchanged (authorize stays a coarse allow — federation establishes who, the PolicyEngine decides what)
Dependency-free defaults, heavy crypto behind extras (mirrors BYOK / NLI). OIDC verification is a real stdlib HS256 JWT verifier (verify_jwt: split → header alg allowlist check → constant-time hmac.compare_digest → exp / nbf / iss / aud with leeway); asymmetric RS256 / ES256 delegates to PyJWT behind the [oidc] extra against a configured public key. SAML parses through defusedxml (a core dep — XXE / billion-laughs safe), validates Issuer / Conditions / AudienceRestriction, and injects XML-DSig verification (signxml_verifier, [saml] extra) — require_signature on with no verifier fails closed
Algorithm-confusion designed out: the alg allowlist rejects alg:none and an RS256 token replayed as HS256; symmetric vs asymmetric take different key material (hmac_secret vs public_key)
Per-tenant IdP config on tenants[].sso (reuses the Step 6.1 mechanism; ${ENV}-interpolated secrets). A tenant with no sso block has no provider → its bearer tokens are rejected (fail-closed) while header-identity dev flows still work. build_federated_auth_from_config builds one provider per tenant; NoopAuth when cfg.sso.enabled is off (pre-6.8 behaviour)
SCIM 2.0 is a separate surface with its own auth. New tenant-scoped ScimStore SPI + NoopScimStore (CRUD for ScimUser / ScimGroup, isolation = the store key) + ScimService (uniqueness, server id + meta, the IdP deactivation PATCH, attr eq "value" filter, PII-free scim.* events). /scim/v2/{Users,Groups} + discovery endpoints authenticate a per-tenant SCIM bearer token (cfg.scim.tokens, constant-time compare — not a user JWT), return SCIM-shaped errors (RFC 7644), and 404 when disabled (checked before auth, so the surface is hidden). No new governed SPI call → the PolicyEngine coverage linter passes with no allowlist entry
New core types FederatedIdentity / SsoProtocol / ScimUser / ScimGroup (+ nested) + SsoError (401) / ScimError (400) / ScimNotFoundError (404) / ScimConflictError (409); wire types ScimListResponse / ScimPatchOp / ScimErrorBody / SsoStatusResponse (dist/schemas + dist/openapi regenerated); cfg.sso / cfg.scim / tenants[].sso (dist/rag.schema); PII-free sso.* / scim.* events (subject hashed, never email / userName); ragctl sso (list + in-process OIDC demo) + ragctl scim (in-process provisioning demo)
Scope: verification + provisioning at the boundary. Deferred: remote JWKS discovery + rotation (configured static keys only), SAML SP-initiated redirect + metadata, SCIM bulk / /Me / ETag, directory-backed deprovisioning at authenticate-time, the admin-console SSO/SCIM card. ~80 new tests (jwt / oidc / saml / federated-auth / scim-service unit; ScimStore contract suite; gateway SCIM CRUD + token guard + isolation + disabled-404; SSO status + JWT-through-middleware; ragctl). All gates green (ruff, mypy --strict, RAG001, schema/openapi/config-drift, policy-coverage, log-schema/event-registry/PII). ADR-0040, reference/sso.md, architecture/sso-scim.md

6.9 — Air-gapped install bundle ✅ #161

Ships the platform as one signed, self-contained artifact for networks with no registry / internet: all runtime container images (docker save), the packaged Helm chart (helm package), an example rag.yaml, and a standalone offline installer. Packages what already exists — the gateway image (built from ./Dockerfile, signed keyless in docker.yml) + the chart (0.9) + the runtime image set (docker-compose)
Integrity = a standard SHA256SUMS + optional cosign (mirrors the 6.6b WORM export). A content_hash (SHA-256 over the canonical SHA256SUMS) is always present for tamper-evidence and is verifiable with nothing but sha256sum -c — no network, no cosign — and is pinned in manifest.json. A cosign signature over SHA256SUMS adds authenticity when present. The same SHA256SUMS drives both the Python verifier and the standalone shell installer, so they can never disagree; manifest/SHA256SUMS/*.sig/*.pem are excluded from the hashed payload
Standalone install.{sh,ps1} need no Python workspace — an air-gap host has bash/pwsh + docker + helm (+ maybe cosign / sha256sum), not uv or the repo. Shipped inside every bundle, they verify → docker load images.tar → helm upgrade --install; ragctl airgap install is the same flow for operators who have the toolchain. --verify-only checks integrity without installing
Build/verify logic in ragctl.airgap (under packages/, so mypy --strict + unit-tested): a pure core (BundleManifest model, render_sha256sums / content_hash / build_manifest / verify_bundle) cleanly separated from the subprocess seam (docker / helm / cosign through one _run chokepoint). A --dry-run produces a complete, verifiable bundle minus the image blobs, so the whole packaging + verification path is testable with no Docker
Digest-pinned, manifest-driven image set: infra/airgap/images.txt lists the third-party images (tags); the build resolves each to its digest into manifest.json. The gateway image is added from the chart's image.repository + --version, so the app image is always release-versioned. Key-based cosign is the air-gap recommendation (keyless needs Rekor + a deployment-specific identity policy); the release path (release-airgap.yml on v* tags) signs keyless and attaches the tarball to the GitHub Release
Scope: docker-based image load + the gateway chart. Deferred: ctr/podman load, registry re-tag/push, multi-arch selection, bundling the backend charts (chart deploys the gateway; backends stay external), TUF-rooted offline keyless verify. task airgap:build / build-dry / verify; ~17 tests (manifest / hashing / tamper detection / tarball round-trip / signature paths via stubbed cosign / CLI on a dry-run bundle), shell installer syntax-checked + exercised via --verify-only. All gates green (ruff, mypy --strict, RAG001, schema/openapi/config-drift, policy-coverage, log-schema). ADR-0041, reference/airgap.md, architecture/airgap-bundle.md, guides/airgap-install.md

6.10 — Compliance posture ✅ #162

Phase 6 capstone. The platform already shipped the controls an auditor maps to (audit / ACL / PII / BYOK / SSO / quotas / breakers / guard); 6.10 adds the three missing pieces — data retention, GDPR erasure / residency, and a machine-readable posture that backs the SOC 2 / GDPR mapping
New rag-compliance package (deps rag-core + rag-observability only, config-free — like rag-feedback / rag-drift): RetentionEnforcer drives tenant-scoped purge_* (purge = per-class windows, erase_tenant = GDPR right-to-erasure); compliance_posture / residency_ok are pure
Retention is a capability on the existing stores, not a new SPI: non-abstract purge_before(ctx, cutoff, *, dry_run) + purge_tenant(ctx, *, dry_run) on FeedbackStore + ProvenanceStore (default no-op → no backend breaks). dry_run lives in the SPI so a preview counts without deleting uniformly (ProvenanceStore has no list). The audit chain is never purged in place (it would break tamper-evidence) — audit retention is the Step 6.6b WORM export; RetentionPolicy.audit_days is advisory
Right-to-erasure — POST /v1/compliance/erase erases the calling tenant's data (scope from the principal, never the body); always available (a tenant can always erase its own data), dry-run by default, real delete needs dry_run=false and confirm=true
Data residency — tenants[].data_region resolves onto TenantSettings (reuses the 6.1 resolver) and is enforced at ingest against cfg.compliance.region; a mismatch raises ResidencyViolationError (→ 403) + compliance.residency_violation. Opt-in (cfg.compliance.enabled), lenient when no region declared — a single-deployment assertion + guard, not multi-region routing
The SOC 2 / GDPR mapping is a doc backed by a live posture — GET /v1/status/compliance serves a CompliancePosture (controls + region + retention) computed from the build_app flags; the mapping docs map each Trust Service Criterion / GDPR article to a control, and the posture reports whether it is on in this deployment (so the mapping is checkable, not just prose)
New core types DataClass / RetentionPolicy / ErasureResult / CompliancePosture + ComplianceError (400) / ResidencyViolationError (403); ComplianceEraseRequest wire type; cfg.compliance + tenants[].data_region / retention_days (dist/rag.schema); PII-free compliance.* events; ragctl compliance report (posture from cfg) + compliance demo. Scope: tenant-level erasure + feedback/provenance retention. Deferred: subject-level chunk/document erasure (needs an index delete-by-document path), an admin retention-sweep endpoint (today the sweep is ragctl/cron; erasure is the API), multi-region routing, automated audit-evidence bundles. ~30 tests (enforcer purge/erase/dry-run + posture + residency unit; purge_* contract suite; gateway posture + erase + residency-403; ragctl). All gates green (ruff, mypy --strict, RAG001, schema/openapi/config-drift, policy-coverage, log-schema/event-registry/PII). ADR-0042, reference/compliance.md, architecture/compliance.md, compliance/soc2-control-mapping.md, compliance/gdpr-mapping.md

Phase 7 — Pilot, Harden, GA (Weeks 34–40) ⏳

Step	Title	Status	Planned deliverables
7.1	Load testing	✅	#164 — chaos-under-load CI gate (`eval/gateway_chaos_v0` fault backends + harness; `tests/perf/test_chaos_under_load.py`; graceful degradation: no 5xx + breaker opens); Locust v1 suite (varied-query mix + ramp shape); acceptance targets + runbook
7.2	Chaos engineering	✅	#165 — in-process kill-matrix gate (kill each hot-path backend → no 5xx, breaker opens, expected degraded shape); two new gateway degrade-open guards (retrieval cache + reranker); LitmusChaos cluster manifests (`infra/chaos/`) + `task chaos-kill`
7.3	Red-team	✅	#166 — adversarial probe gate (injection / PII / ACL / tenant-escape) under a `redteam` marker + `redteam-gate` CI job; new `rag-injection` guard (≥ 95 % block, no untrusted chunk in a system-trust position) wired on every answer surface; PII-egress probe; `task redteam`
7.4	Design partner onboarding	✅	#174 — repo-backed pilot program: operator runbook + 5 templates (`docs/pilots/`) + ADR-0046; customer-support/KB kit (corpus w/ PII + planted injection probe; domain criteria); `ragctl pilot` `onboard` + `report` (weekly KPIs + PASS/FAIL verdict from feedback/drift/cost signal components, KPIs pulled not self-reported); worked case study (real `pilot report` → PASS). Signing partners = external GTM item
7.5	Documentation site	✅	#178 — Docusaurus app (`website/`) serving `docs/` in place; generated + drift-gated REST API reference from `dist/openapi.json`; `tests/docs/` quickstart-honesty suite + `lychee`/`codespell`; `docs` CI workflow; `task docs:api`/`docs:build`
7.6	Marketplace listings	✅	#179 — `marketplace/` listing artifacts (AWS/Azure/GCP); one canonical `pricing.yaml` (dims = 7.9 metering signals) + shared listing copy mapped to all three; reuse Helm/AMI/airgap/GHCR delivery; submission checklists + procurement test; approval = external process
7.7	Packaging & distribution	✅	#180 — `rag-platform` PyPI meta-package; `release.yml` fans one tag out to PyPI/npm/GHCR(admin-ui)/Helm-OCI (cosign-keyless + SPDX SBOM + OIDC/provenance) alongside `docker.yml`+`release-airgap.yml`; `apps/admin-ui/Dockerfile`; channels matrix + guide
7.8	Support & SLA	✅	#181 — support tiers + SLA targets (= `pricing.yaml`); `docs/runbooks/` with a runbook per alert type (tied to real events / `/v1/status/*` / SLOs); incident-response + postmortem template; status page from measured health; PagerDuty via the Step 3.9 webhook system (no new code)
7.9	Billing integration	✅	#182 — `rag_observability.billing`: observe-only `UsageMeter` (5 dimensions) + `GET /v1/billing/usage`; pure `generate_invoice` priced from `pricing.yaml` (reconciles ±0.5%); `BillingProvider` Stripe/marketplace seam; `cfg.billing`; `ragctl billing`; mirrors 5.6c (no `dist/schemas` churn)
7.10	GA cutover	✅	#183 — v1.0.0: SemVer + deprecation policy (ADR-0052); release notes; GA readiness checklist (all exit gates, external items flagged); `rag-platform` meta → 1.0.0; `v1.0.0` tag fans out to every channel

7.1 — Load + chaos testing ✅ #164

Builds on Step 4.6 (ADR-0025: the in-process p99-overhead gate + Locust v0). 7.1 adds the load + resilience side the GA bar needs, split into a deterministic CI gate + a cluster runbook
Chaos-under-load is a deterministic CI gate. New eval/gateway_chaos_v0/ drives the in-process gateway (httpx ASGITransport, no socket / no cluster) under concurrent load while injecting backend faults, and asserts the gateway degrades gracefully — no 5xx, 100% success, the failing backend's circuit breaker opens. The asserted property is resilience, not throughput, so it is timing-independent and runs in the perf CI job (tests/perf/test_chaos_under_load.py); task chaos-test runs the harness --check
Reuses the Phase-4 machinery — it validates it. FaultSpec (seeded failure probability + injected latency) + Chaos{Vector,Keyword,Graph}RetrievalBackend are SPI wrappers exactly like the rag-breaker wrappers (allowlisted in the policy-coverage linter for the same reason: they forward the already-policy-merged retrieve_ids / expand), wired behind real breakers + the fallback ladder. Under 100% vector failure the vector breaker trips and the fallback fuses the survivors; under all-backends-fail it degrades to graceful empty answers — the client never sees a 5xx
The query is varied per request (a unique nonce) in both the chaos harness and the Locust suite — identical queries hit the retrieval cache and short-circuit before retrieval, so a constant-query load test would measure the cache, not the pipeline (and the chaos backends would never be exercised)
The 1000-QPS / p99 < 500 ms acceptance is a cluster runbook, not a CI gate (hardware/backends-bound — same reasoning as ADR-0025). Extended Locust v1 suite (eval/gateway_load_v0/locustfile.py): a weighted read/write mix (/v1/retrieve · /v1/query · /v1/feedback · /v1/status/metrics · /healthz) + a LoadTestShape that ramps → holds a plateau → ramps down (env-tunable peak / hold for distributed workers). Acceptance targets + the runbook in guides/load-testing.md
Scope: in-process chaos gate (resilience) + the Locust suite + runbook for the throughput acceptance. Deferred: distributed-Locust-in-CI against an ephemeral cluster, latency-based breaker tripping (timeouts), storage/LLM-layer fault injection, soak/endurance tests (cluster-level chaos with LitmusChaos is Step 7.2). ~4 chaos perf tests; all gates green (ruff, mypy --strict, RAG001, policy-coverage, schema/openapi-drift, log-schema). ADR-0043, guides/load-testing.md, reference/perf.md

7.2 — Chaos engineering ✅ #165

Builds on Step 7.1 (ADR-0043). 7.1 fault-injected the three retrieval backends under load; 7.2 closes out the deliverable — "kill each backend; verify the fallback chain holds" — and the LitmusChaos piece 7.1 deferred. The GA bar is a sharp invariant: no single backend failure can 5xx the gateway
The kill-matrix is a deterministic in-process CI gate. eval/gateway_chaos_v0/kill_matrix.py walks the full hot-path backend set — vector / keyword / graph / embedder / retrieval_cache / reranker / llm — kills one at a time (100% unavailable) behind the real Phase-4 breakers + fallback + the gateway degrade guards, drives /v1/query, and asserts no 5xx, 100% success, the killed on-path retrieval breaker opens, and the expected degraded shape (tests/perf/test_chaos_kill_matrix.py, perf-marked + timing-independent; task chaos-kill). A tiny seeded keyword corpus + the store's real hydrate make rerank/pack/generate actually run, so every kill exercises a genuine degrade path
Chaos fixes what it finds. Running the matrix exposed two ways to 5xx: a down retrieval cache (get_best/put_with_query) and a down reranker (reranker.rerank) propagated unhandled. Both gained minimal degrade-open guards at the call site (matching the existing LLM guard): the cache treats an outage as a miss (gateway.cache.degraded, on /v1/query + /v1/retrieve); the reranker falls back to retrieval-only (gateway.rerank.degraded, honouring RerankPipeline's documented "caller decides" contract — the reranker is also the hydration stage, so the degraded response carries no citations, same shape as rerank=false). The LLM (answer-generation guard) + embedder (understanding's per-component catch) already degraded
Graph is off the default read path. RetrievalRouter.decide only sets use_graph when the caller supplies graph seeds; a standard query never does, so killing graph cannot affect it (breaker stays closed) — the matrix records this honestly (no 5xx asserted, breaker-open not required) rather than forcing a synthetic graph query
Cluster chaos is a runbook, not a CI gate (topology-bound). infra/chaos/ LitmusChaos manifests: gateway pod-delete (PDB-protected instance loss) + pod-network-loss/pod-network-latency toward a backend host (the cluster way to "kill a backend", since backends are reached by host), each with an embedded httpProbe asserting the gateway stays 200 throughout
Scope: kill-matrix gate + the two degrade-open hardenings + the LitmusChaos runbook. No new core/wire types, SPI methods, config, or events (dist/ untouched; the degrade kinds are log-only like gateway.answer.failed; the kill wrappers are pure-raise so the policy-coverage linter needs no new rule). Deferred: latency-based breaker tripping, multi-backend simultaneous kills as a gate, Litmus-in-CI against an ephemeral cluster, soak chaos. 12 kill-matrix perf tests; all gates green (ruff, mypy --strict 325 files, RAG001, policy-coverage, schema/openapi-drift, log-schema). ADR-0044, guides/chaos-engineering.md, reference/perf.md

7.3 — Red-team / security ✅ #166

The governance stack (ACL 6.3/6.4, PII 6.5, tenancy 6.1/6.2, hallucination guard 4.3) becomes an adversarial probe gate across four classes. ACL bypass + tenant escape were already gated (tests/redteam/test_acl_*, test_cross_tenant_*); 7.3 adds prompt injection + PII egress and consolidates all four under a redteam marker (a tests/redteam/conftest.py auto-marks the directory — scoped to its own subtree so it can't mark the whole suite) + a first-class redteam-gate CI job (task redteam)
The injection gap was real. trust_level was stored + filterable everywhere but nothing enforced it at the LLM adapter — a retrieved document could carry "ignore your instructions, you are now…" straight to the model, and the OpenAI-chat surface assembled context as a role="system" message (an attacker-controlled system-trust position, which PROBLEM-TRACEABILITY forbids)
New rag-injection package (deps rag-core + rag-observability, mirroring rag-guard/rag-pii): a pluggable InjectionDetector (dependency-free HeuristicInjectionDetector — a regex library anchored on the grammar of an attack: an imperative verb aimed at the model + its instructions/role/safety, within a bounded sentence-local gap, so benign prose mentioning "system"/"prompt"/"instructions" isn't flagged) + PromptInjectionGuard.inspect(ctx, chunks) that drops chunks scoring ≥ a per-tenant threshold (degrade-open; PII-free injection.blocked). Paired with INJECTION_RESISTANT_SYSTEM_PROMPT + build_user_message so retrieved content can only ever appear as fenced untrusted data in the user turn — never a system turn. Wired into every answer surface (/v1/query, /v1/chat/completions, MCP); off by default (cfg.injection)
Deterministic CI gate, honest numbers. eval/redteam_v0/injection_corpus.py — 568 known + 630 generated payloads across all categories + a benign control set. tests/redteam/test_prompt_injection.py asserts 96.6 % block (≥ 95 % bar), 0 false positives (≤ 5 % bar), and the no-system-position invariant end-to-end with a capturing LLM on both surfaces. Building the corpus hardened the detector — it exposed ~20 missed phrasings (e.g. "ignore your previous instructions"), the point of red-teaming. The corpus is broader than the detector's patterns (carriers, morphology, a few evasive payloads) so the rate is honest, not a circular 100 %
PII-egress probe (test_pii_egress.py): synthetic high-PII corpora through the 6.5 PiiPolicyEngine — block denies, redact/mask strip every span (verified by an independent second detector), over answer-text + list[Chunk] shapes
Scope: the injection guard + the four-class probe gate + task redteam + the redteam-gate CI job. Dependency-scan CVE gating is the existing pip-audit audit job; the external pentest is a documented human-process item (guides/red-team.md), not code. Injection types are package-local + internal (no attacker signal on the wire; dist/schemas/openapi untouched). Deferred: a real ML injection classifier behind the seam, per-tenant pattern tuning, nightly staging runs. ~23 new tests (12 package-local + 11 red-team) on top of the existing suite; all gates green (ruff, mypy --strict 329 files, RAG001, policy-coverage, schema/openapi-drift, log-schema/event-registry). ADR-0045, guides/red-team.md, reference/injection.md

7.4 — Design-partner pilots ✅ #174

Step 7.4 is a GTM/process step made checkable: signing 3 referenceable design partners is an external human deliverable (like the 7.3 external pentest), so the repo ships the machine — a repeatable program whose KPIs are pulled from the platform's own signals, not self-reported. Delivered as one PR sliced 7.4a–d.
7.4a — framework. The operator runbook (guides/design-partner-pilots.md): pilot lifecycle (qualify → onboard → run → review → graduate), roles, onboarding-as-configured-deployment, the KPI→signal mapping, the intake → triage → incorporate → close feedback loop, acceptance (signed criteria green ≥ 3 consecutive weeks). Five fill-in templates under docs/pilots/templates/ (onboarding-checklist / success-criteria / weekly-kpi / feedback-log / case-study). ADR-0046.
7.4b — customer-support/KB kit. docs/pilots/customer-support/ — a 3-doc sample corpus calibrated to exercise each criterion: an HR handbook carrying synthetic PII (egress demo), a product FAQ (grounded answers), and a security overview hiding a planted prompt-injection payload (the 7.3 guard must drop it). Domain-calibrated success criteria (faithfulness 0.92, a deflection KPI); framework stays vertical-extensible (legal / engineering kits deferred).
7.4c — ragctl pilot tooling. pilot onboard renders the per-tenant rag.yaml block + onboarding checklist; pilot report seeds a representative stream into the platform's own signal components (rag_feedback aggregate / rag_drift registry / rag_observability.CostTracker — the same ones behind GET /v1/status/{feedback,drift,cost}) and prints the weekly-KPI dashboard + a PASS/FAIL verdict (satisfaction ≥ +0.5, zero drifting monitors, cost not elevated); quality + latency cross-ref ragctl eval / perf. Seed-then-report (no infra), the same shape as ragctl feedback / drift. 3 new ragctl tests. reference/pilot.md.
7.4d — worked case study. docs/pilots/customer-support/case-study.md runs the framework end-to-end on the kit with real ragctl pilot report output (satisfaction +0.733, 0/5 monitors drifting, cost ok → PASS) + the PII-redaction + injection-blocked security demonstration — framed honestly as a representative pilot (a real partner substitutes their corpus + users).
Scope: the framework + the first vertical kit + the ragctl pilot reader + the worked case study. No new package / core or wire type / governed SPI call / dist change — pilot is a pure ragctl reader over existing signal components, so the policy-coverage + schema/openapi-drift gates need no new entry. Deferred / external: sourcing + signing the real referenceable partners; the legal + internal-engineering kits; per-pilot golden-set generation tooling; a pilot KPI card in the admin console. All gates green (ruff, mypy --strict, RAG001, link-check). ADR-0046, guides/design-partner-pilots.md, reference/pilot.md.

7.5 — Documentation site ✅ #178

A public docs site that can't drift from the repo: a Docusaurus 3 app under website/ whose docs plugin points at ../docs (routeBasePath: '/'), so the PR-reviewed docs/ tree is the site — no copy step. markdown.format: 'detect' renders hand-written .md as CommonMark (no MDX escaping of < / {).
The REST API reference is a generated, drift-gated artifact. scripts/gen_api_reference.py (pure stdlib) renders the 43-path dist/openapi.json → docs/reference/rest-api.md; the docs CI workflow regenerates + diffs it, exactly like dist/schemas / the proto stubs — so the published API docs always match the gateway contract. task docs:api regenerates.
Doc honesty is tested, not asserted. tests/docs/ extracts every ragctl <subcommand> and /v1/... path from the quickstart guides and fails if any isn't a registered CLI command / a real OpenAPI path — the lightest deterministic form of "every quickstart is exercised", no services needed (building it caught + fixed a newline-spanning regex false-positive).
New docs CI workflow (.github/workflows/docs.yml): API-reference drift · quickstart doc-tests · codespell · lychee --offline link-check · full npm run build. task docs:dev / docs:build run/build locally.
Scope: the site scaffold + generated reference + doc-tests + CI + task targets. No code / dist change beyond the new generated reference file. Deferred: Algolia DocSearch, versioned docs aligned to releases, the deploy workflow + custom domain, gRPC/MCP reference auto-generation, executable (vs existence-checked) snippets. Verified locally (the npm site build is a CI step; the Python generator + the 3 doc-tests run green). ADR-0047, guides/documentation-site.md.

7.6 — Marketplace listings ✅ #179

Listing artifacts for the three clouds under a new marketplace/. As with the 7.3 pentest + 7.4 partners, approval + procurement integration are an external per-cloud process — the repo ships the submission inputs, not a faked "live" status.
One pricing model, three listings. marketplace/pricing.yaml is canonical (Community / Pro / Business / Enterprise tiers + usage-metered dimensions); its dimension ids match the Step 7.9 metering signals one-to-one, so prices reconcile with metered usage and a price change is a one-file edit. marketplace/listing-content.md is the shared copy (title / features / SLA / support / terms) the per-cloud files reference — listings can't drift from each other.
Reuse existing delivery — no new packaging. AWS (SaaS + AMI), Azure (SaaS + managed app), GCP (SaaS + K8s app) each map the model to that cloud's billing primitives and deploy via the existing Helm chart (infra/) + air-gap bundle (6.9) + GHCR/OCI artifacts (7.7). The per-cloud Marketplace wrappers (packer AMI, Azure ARM, GCP schema.yaml) are thin shells, skeletoned.
Scope: marketplace/ (canonical pricing + shared copy + three per-cloud specs + checklists) + the process guide. No code / dist change. Deferred: the Marketplace wrapper artifacts, the procurement/metering API integrations, and the actual submissions/approvals (external). Pure docs/manifests, link-checked. ADR-0048, guides/marketplace-listings.md.

7.7 — Packaging & distribution ✅ #180

The acceptance bar — one-command install from every channel, all artifacts signed + SBOM-attested — met by adding a server meta-package + a release path that fans one tag out to every channel with the same signing posture the gateway image already had.
rag-platform PyPI meta-package (packaging/rag-platform/) pins the workspace component distributions, so pip install rag-platform==X resolves a reproducible server stack (optional backends + cloud KMS as extras). The agentcontextos SDK stays a separate, lighter dist; the existing agentcontextos / @agentcontextos/sdk scope is kept — no rename (the plan's aspirational ragplatform names would break existing consumers).
One tag → every channel. A vX.Y.Z tag triggers docker.yml (gateway image, cosign + SBOM, already shipped), the new release.yml (PyPI Trusted Publishing for the meta + SDK · npm with provenance · the admin-ui image via a new apps/admin-ui/Dockerfile, cosign + SBOM · the Helm chart pushed to OCI + cosign-signed), and release-airgap.yml (signed offline bundle). Kept as separate workflows so each re-runs independently.
Signed + SBOM, keyless. cosign keyless (GitHub OIDC) for images + chart, SPDX SBOM per image, OIDC Trusted Publishing for PyPI, --provenance for npm, cosign-over-SHA256SUMS for the air-gap bundle — no long-lived registry tokens.
Scope: the meta-package + release.yml + the admin-ui Dockerfile + a channels-matrix packaging/README.md + the guide. Deferred: publishing the generated Go/Java/.NET SDKs; separate worker/eval-runner images (single gateway process today; core is a library, not an image); the one-time trusted-publisher / GHCR setup + first publish (external). ADR-0049, guides/packaging-distribution.md.

7.8 — Support, SLAs & on-call ✅ #181

The support + ops layer, grounded in the signals the platform already emits rather than a new alerting subsystem.
Support tiers + SLA (support-sla.md): Community / Pro / Business / Enterprise with response targets; per-tier availability + latency SLAs that match marketplace/pricing.yaml (sold = promised = measured); the follow-the-sun on-call + escalation.
A runbook for every alert type (docs/runbooks/alerts.md): acl/tenant escape · breaker.opened · drift.detected · cost anomaly · quota.exceeded · injection spike · gateway degraded · latency/availability SLO · eval.regression · ingest failures — each names the exact event / /v1/status/* surface / SLO that raises it, a diagnosis path, a mitigation (often the existing self-healing), and an escalation.
Incident response + postmortem (incident-response.md + postmortem-template.md): severity ladder, IC/Comms/Ops roles, the detect→resolve flow, and a status page that reflects measured health (/v1/status/health + Grafana SLOs, Step 3.11/5.6e) — not manual toggles.
Paging reuses the Step 3.9 webhook system — PagerDuty is a subscriber for audit.policy_violation / drift.detected / eval.regression (Events API v2), no new code.
Scope: the guide + the docs/runbooks/ area + this ADR. No code/dist change. Deferred: the hosted status-page deploy, the PagerDuty account + routing key, staffing the rotation (people process). ADR-0050, runbooks/.

7.9 — Billing & metering ✅ #182

Per-tenant usage accounting for money — a third concern beside cost-anomaly detection (5.6c) and quota enforcement (4.5), mirroring the 5.6c shape to avoid churn.
Dataclasses in rag-observability, no dist/schemas churn. rag_observability.billing holds a UsageMeter (thread-safe per-tenant, per-dimension counters — queries / docs_ingested / storage_gb / reranker_calls / llm_tokens), UsageSnapshot, and the pricing/invoice types; the gateway wraps the snapshot in a gateway-local Pydantic BillingUsageResponse (like CostStatusResponse). Only dist/openapi (the new endpoint) + dist/rag.schema (the new cfg.billing block) regenerate.
Observe-only metering; enforcement stays quotas. Fed O(1) from record_request_usage (queries + tokens), independently of quotas, making no governed SPI call (the policy-coverage linter needs no entry — same as cost/drift/feedback). GET /v1/billing/usage serves the per-tenant usage dashboard; inert by default (cfg.billing).
Invoicing is offline, pure, priced from config. generate_invoice(snapshot, pricing, tier) is pure; PricingModel/load_pricing read marketplace/pricing.yaml, so a price change is config (the shared 7.6/7.9 bar). Each metered line item's quantity equals the metered overage, so bills reconcile with usage ±0.5% by construction (the acceptance) — ragctl billing prints the invoice + the reconciliation check.
External billers are a Protocol seam. BillingProvider.report_usage (Stripe usage-records / marketplace metering APIs) with a NoopBillingProvider default, called degrade-open — no rag_core.spi ceremony.
Scope: the billing module + the usage endpoint + cfg.billing + wiring + ragctl billing + tests (unit reconciliation + gateway endpoint + CLI). All gates green (ruff, mypy --strict 330 files, config/openapi-drift regenerated). Deferred: the concrete Stripe/marketplace adapters, the docs/reranker/storage feed points, an admin usage card, subject-level invoicing. ADR-0051, reference/billing.md.

7.10 — GA cutover ✅ #183

General Availability — v1.0.0 — the close of an 8-phase, 84-step build (Phase 0 foundations → Phase 7 GA).
SemVer from GA. MAJOR/MINOR/PATCH over the drift-gated public contracts (REST/OpenAPI · gRPC rag.proto · rag.yaml schema · SDKs · ragctl CLI), so a breaking change is caught in CI and forces a deliberate MAJOR bump; a deprecation cycle of ≥ 1 MINOR precedes removal (ADR-0052). The rag-platform meta-package carries the platform version (bumped to 1.0.0).
The GA bar is a checklist of gates, not a declaration. ga-readiness-checklist.md maps every Phase 7 exit gate to a CI gate or an in-repo artifact; inherently-external items — external pentest, SOC 2 Type II audit, the first publish to each registry, signing 3 referenceable customers, on-call staffing + the hosted status page, and the launch announcement — are marked (external) and ride on top, the same split used throughout Phase 7.
One tag, every channel. v1.0.0 is tagged on main; the existing release workflows (docker.yml + release.yml + release-airgap.yml, Step 7.7) fan it out to PyPI / npm / GHCR / Helm-OCI / the air-gap bundle, all cosign-signed + SBOM-attested.
Scope: the GA release notes + readiness checklist + ADR-0052 + the meta version bump + the v1.0.0 tag. Deferred / external: the checklist's (external) items + the V1.1 backlog (no-code agent builder, multimodal outputs, on-prem GPU autoscaling, marketplace partner connectors, domain packs). release-notes/v1.0.0, ga/ga-readiness-checklist.

PR & Branch History

Complete log of every PR. Routine Dependabot bumps are grouped; everything else is listed individually.

PR	Merged	Title
#1	2026-05-21	Problems To Start With
#2	2026-05-21	Add V1 execution plan, logging standard, GUI spec, and doc restructure
#3	2026-05-21	feat: Phase 0 Step 0.1 — monorepo and build system
#4	2026-05-22	Phase 0 Step 0.2 — Core domain types & errors
#5	Closed	Duplicate merge of #4 (closed, no-op)
#6	2026-05-22	feat(tooling): Phase 0 Step 0.2c — cross-platform support
#7	2026-05-22	feat(core): Phase 0 Step 0.3 — Plugin SPI interfaces
#8	2026-05-22	feat(config): Phase 0 Step 0.4 — rag.yaml schema & loader
#9	2026-05-22	feat(ci): Phase 0 Step 0.5 — CI/CD pipeline hardening
#10–#24	2026-05-22	Dependabot dependency bumps — GitHub Actions / npm / pip (15 PRs)
#25	2026-05-23	chore: release main
#26	2026-05-23	feat(dev): Phase 0 Step 0.6 — local dev stack
#27	2026-05-23	feat(core): OTel tracing + SPI metrics foundation (Step 0.7)
#28	Open	chore: release main
#29	2026-05-23	feat(core): structured logging foundation (Step 0.7b)
#30	2026-05-23	refactor(observability): extract packages/observability with full log schema (Step 0.7b follow-up)
#31	2026-05-23	feat(core): audit log skeleton — AuditStore SPI + hash-chain noop + AuditWriter (Step 0.7c)
#32	2026-05-23	docs: add CLAUDE.md with project context and per-PR documentation requirement
#33	2026-05-23	feat(eval): eval skeleton — golden-set schema, metrics, RAGAS spike, ragctl eval (Step 0.8)
#34	2026-05-23	feat(infra): IaC foundation — Terraform modules + Helm chart skeleton (Step 0.9)
#35	2026-05-23	chore(tracker): sync PR links for steps 0.7b–0.9 and history #28–#34
#36	2026-05-23	feat(ragctl): consolidated control-plane CLI scaffold (Step 0.10)
#37	2026-05-23	fix(logging): allowlist rag_core/audit.py for RAG001
#38	2026-05-23	ci: wire RAG001 logging check into ci.yml on all OSes
#39	2026-05-23	chore(tracker): backfill PR links for #35, #36, #38
#40	2026-05-23	feat(backends): storage backend plugins — pgvector, Qdrant, Redis, S3 (Step 1.1)
#41	2026-05-23	docs(planning): Phase 1 architecture-refactor window (Steps 1.1a–1.1f) + ADRs 0005–0009
#42	2026-05-23	chore(tracker): log PR #41 in history table
#43	2026-05-23	chore(claude): add /nxt slash command to report next tracker item
#44	2026-05-23	feat(core): RequestContext + ctx-threaded SPI (Step 1.1a)
#45	2026-05-24	feat(core): SPI split — RetrievalBackend / IndexBackend (Step 1.1b)
#46	2026-05-24	feat(policy): rag-policy package — PolicyEngine PDP + coverage linter (Step 1.1c)
#47	2026-05-24	chore(schemas): track generated JSON Schemas + add drift CI gate
#48	2026-05-24	chore(ci): lint scripts/ + ignore S603/S607 in scripts/**
#49	2026-05-24	chore(tracker): sync PR links for steps 1.1a/1.1c + log #46–#48
#50	2026-05-24	feat(core): Pipeline + Batcher primitives (Step 1.1d)
#51	2026-05-24	feat(core,observability): Cache SPI split + async telemetry (Step 1.1e)
#52	2026-05-24	docs(adr,architecture): finalize ADRs 0005–0009 + extend reviewer checklist (Step 1.1f)
#53	2026-05-24	chore(tracker): sync Step 1.1f → ✅ and log PR #52
#54	2026-05-24	chore(tracker): log PR #53 in history
#55	2026-05-24	feat(core,backends): connectors framework — SPI evolution + filesystem/S3/GCS (Step 1.2)
#56	2026-05-24	chore(tracker): sync Step 1.2 → ✅ and log PR #55
#57	2026-05-24	feat(core,parsers): document parsers — PDF/DOCX/PPTX/XLSX/HTML/MD/text/JSON/CSV/YAML (Step 1.3)
#58	2026-05-25	chore(tracker): sync Step 1.3 → ✅ and log PRs #56, #57
#59	2026-05-25	fix(parsers): normalize CRLF/CR to LF in text & markdown parsers
#60	2026-05-25	feat(core,ocr): OCR pipeline — region-aware SPI + Tesseract + PaddleOCR (Step 1.4)
#61	2026-05-25	chore(tracker): sync Step 1.4 → ✅ and log PRs #58, #59, #60
#62	2026-05-25	feat(core,chunker): structure-aware chunker (Step 1.5)
#63	2026-05-25	chore(tracker): sync Step 1.5 → ✅ and log PRs #61, #62
#64	2026-05-25	feat(core,enricher): metadata enricher (Step 1.6)
#65	2026-05-25	chore(tracker): sync Step 1.6 → ✅ and log PRs #63, #64
#66	2026-05-25	feat(core,pii): PII detection + per-tenant policy enforcement (Step 1.7)
#67	2026-05-25	chore(tracker): sync Step 1.7 → ✅ and log PRs #65, #66
#68	2026-05-25	feat(embedders): embedder pipeline — OpenAI / Cohere / BGE / E5 (Step 1.8)
#69	2026-05-25	chore(tracker): sync Step 1.8 → ✅ and log PRs #67, #68; advance to 1.10
#70	2026-05-25	feat(backends): CDC connectors + Document.fingerprint dedup (Step 1.9)
#71	2026-05-25	chore(tracker): sync Step 1.9 → ✅ and log PRs #69, #70
#72	2026-05-25	feat(ingest,gateway): write path + POST /v1/ingest (Step 1.10 — CLI demo milestone)
#73	2026-05-25	chore(tracker): mark Phase 1 complete (Step 1.10 → ✅; log PRs #71, #72)
#74	2026-05-25	feat(core,backends): knowledge store read layer + FilterExpr push-down (Step 2.1)
#75	2026-05-25	chore(tracker): sync Step 2.1 → ✅ and log PRs #73, #74
#76	2026-05-25	feat(backends): Weaviate, Pinecone, Elasticsearch vector backends + IndexHint wiring (Step 2.2)
#77	2026-05-26	chore(tracker): sync Step 2.2 → log PR #76 + history rows #75, #76
#82	2026-05-26	feat(backends): BM25 keyword retrieval — Elasticsearch + tantivy (Step 2.3)
#83	2026-05-26	chore(tracker): sync Step 2.3 → log PR #82 + history rows #77, #82
#84	2026-05-26	feat(core,backends): graph retrieval — Neo4j + Memgraph + NetworkX + expand() (Step 2.4)
#85	2026-05-26	chore(tracker): sync Step 2.4 → log PR #84 + history rows #83, #84
#86	2026-05-26	feat(retrieval): hybrid RRF fusion — rrf_fuse + HybridRetriever (Step 2.5)
#87	2026-05-26	fix(tooling): unblock mypy by removing duplicate package-local tests/init.py
#88	2026-05-26	fix(retrieval): cast policy_filter to restore FilterExpr across lazy boundary
#89	2026-05-26	chore(tracker): sync Step 2.5 → ✅ and log PRs #85–#88
#90	2026-05-26	feat(query): rag-query — query understanding (rewriter, decomposer, HyDE, glossary)
#91	2026-05-26	chore(tracker): sync Step 2.6 → ✅ and log PR #90
#92	2026-05-26	feat(reranker): rag-reranker — two-stage cross-encoder + MMR (Step 2.7)
#93	2026-05-26	chore(tracker): sync Step 2.7 → ✅ and log PR #92
#94	2026-05-26	feat(packer): rag-packer — context packer (dedup + reorder + budget + conflicts) (Step 2.8)
#95	2026-05-26	chore(tracker): sync Step 2.8 → ✅ and log PR #94
#96	2026-05-26	feat(graphrag): rag-graphrag — community detection + summarisation + graph-aware retrieval (Step 2.9)
#97	2026-05-26	chore(tracker): sync Step 2.9 → ✅ and log PR #96
#98	2026-05-26	feat(retrieval): retrieval router — query-shape classifier + health tracker + BM25-only fallback (Step 2.10)
#99	2026-05-26	chore(tracker): sync Step 2.10 → ✅ and log PR #98
#100	2026-05-26	feat(retrieval): agent-loop validation spike — AgentLoopV0 + 50-query harness + gap list (Step 2.11)
#101	2026-05-26	refactor(retrieval): split RetrievalRouter.route into decide + execute (G-02)
#102	2026-05-26	fix(core): reject zero Budget caps at construction (G-03)
#103	2026-05-26	feat(query): cache HyDE embeddings + rewriter outputs (G-06)
#104	2026-05-26	feat(retrieval): semantic embedding cache + ADR-0010 (G-01)
#105	2026-05-26	feat(retrieval): per-stage cost estimator + ADR-0008 backlink (G-04)
#106	2026-05-26	chore(tracker): sync Phase 2 + pre-3.1 fixes complete; log PRs #101–#105
#107	2026-05-27	feat(gateway): Curl-able RAG — /v1/query + /v1/retrieve + /v1/corpora (Step 3.1)
#108	2026-05-27	chore(tracker): sync PR #107 link for Step 3.1
#109	2026-05-27	feat(gateway): gRPC service — RagService over rag.gateway.v1 (Step 3.2)
#110	2026-05-27	chore(tracker): sync PR #109 link for Step 3.2
#111	2026-06-01	feat(gateway): MCP server — query/retrieve/ingest tools (Step 3.3)
#112	2026-06-01	chore(tracker): sync PR #111 link for Step 3.3
#113	2026-06-01	feat(gateway): OpenAI-compatible endpoints with retrieval pre-fetch (Step 3.4)
#114	2026-06-01	chore(tracker): sync PR #113 link for Step 3.4
#115	2026-06-03	feat(retrieval): corpus router with static rules, learned classifier & federated fan-out (Step 3.5)
#119	2026-06-03	chore(tracker): sync PR #115 link for Step 3.5
#120	2026-06-04	feat(agent): production agent runtime over REST SSE + gRPC Converse (Step 3.6)
#121	2026-06-04	feat(sdks): official Python/TypeScript/Go/Java/.NET SDKs (Step 3.7)
#122	2026-06-04	feat(integrations): framework adapters for 8 agent/RAG frameworks (Step 3.8)
#123	2026-06-04	feat(webhooks): signed outbound webhook delivery (Step 3.9)
#124	2026-06-04	feat(admin-ui): Next.js 14 operator console (Step 3.10)
#125	2026-06-04	feat(gateway): status & metrics surface + operator console pages (Step 3.11)
#126	2026-06-04	feat(cache): tiered semantic cache L0/L1/L2 (Step 4.1)
#127	2026-06-04	feat(retrieval): fallback chain (Step 4.2)
#128	2026-06-04	feat(guard): hallucination guard (Step 4.3)
#129	2026-06-04	feat(breaker): per-backend circuit breakers (Step 4.4)
#130	2026-06-04	feat(quota): per-tenant quotas & rate limiting (Step 4.5)
#131	2026-06-04	feat(perf): gateway latency gate, load test & profiling (Step 4.6)
#132	2026-06-04	feat(provenance): per-query tracing & HMAC-signed provenance (Step 5.1)
#133	2026-06-05	feat(eval): offline golden-set eval harness (Step 5.2)
#134	2026-06-05	feat(eval): CI eval gate (Step 5.3)
#135	2026-06-05	feat(feedback): online metrics & feedback (Step 5.4)
#136	2026-06-05	feat(drift): drift monitors (Step 5.5)
#137	2026-06-05	feat(admin-ui): drift & feedback Live Status cards (Step 5.6a)
#138	2026-06-05	feat(admin-ui): query trace & provenance viewer (Step 5.6b)
#139	2026-06-05	feat(cost): per-tenant cost-anomaly detection + console card (Step 5.6c)
#140	2026-06-05	feat(eval): regression bisector over git history (Step 5.6d)
#141	2026-06-05	feat(observability): drift + cost Prometheus metrics + Grafana dashboard (Step 5.6e)
#142	2026-06-05	feat(admin-ui): observability cross-links + Step 5.6 close-out (5.6f)
#143	2026-06-05	feat(experiments): A/B analyzer + tracker + dashboard (Step 5.7a)
#144	2026-06-05	docs(tracker): restructure for readability + complete PR history
#145	2026-06-05	feat(experiments): shadow mode — observe-only candidate fan-out (Step 5.7b)
#146	2026-06-07	feat(experiments): A/B routing — serve the candidate to a fraction of users (Step 5.7c)
#147	2026-06-07	feat(admin-ui): A/B experiments console card + Phase-5 close-out (Step 5.7d)
#148	2026-06-08	feat(tenancy): logical multi-tenancy — per-tenant config resolution (Step 6.1)
#149	2026-06-08	feat(tenancy): physical tenancy — dedicated index per tenant (Step 6.2)
#150	2026-06-08	feat(policy): ACL push-down at retrieval — AclPolicyEngine (Step 6.3)
#151	2026-06-08	feat(policy): ACL egress verifier — defense-in-depth re-check (Step 6.4)
#152	2026-06-08	feat(pii): PII egress policies — PiiPolicyEngine egress_text decorator (Step 6.5)
#153	2026-06-08	feat(gateway): audit read API + chain verification (Step 6.6a)
#154	2026-06-08	feat(audit): WORM signed export — AuditExporter + POST /v1/audit/export (Step 6.6b)
#155	2026-06-08	feat(crypto): BYOK envelope encryption library + local KMS (Step 6.7a)
#156	2026-06-08	feat(crypto): cfg.kms + key-manager factory + AWS KMS provider (Step 6.7b)
#157	2026-06-08	feat(crypto): GCP / Azure / Vault KMS providers (Step 6.7c)
#158	2026-06-08	feat(crypto): zero-downtime key rotation — RotatingKeyManager (Step 6.7d)
#165	2026-06-08	test(perf): chaos kill-matrix gate + cache/rerank degrade-open + LitmusChaos (Step 7.2)
#166	2026-06-08	feat(security): prompt-injection guard (rag-injection) + red-team probe gate (Step 7.3)
#174	2026-06-10	feat(pilot): design-partner pilot program — framework + customer-support kit + ragctl pilot + case study (Step 7.4)
#175	2026-06-10	fix(core): carry corpus_id on Embedding for vector corpus scoping (ADR-0004 §3)
#176	2026-06-10	fix(gateway): scope answers to the question's subject on both answer prompts
#177	2026-06-10	feat(admin-ui): wire dashboard + audit page to live data; clearer trace help
#178	2026-06-10	feat(docs): Docusaurus documentation site + generated API reference + doc-tests (Step 7.5)
#179	2026-06-10	docs(marketplace): AWS/Azure/GCP listing artifacts + canonical pricing model (Step 7.6)
#180	2026-06-10	feat(release): rag-platform PyPI meta + release.yml (PyPI/npm/Helm-OCI/admin-ui image) (Step 7.7)
#181	2026-06-10	docs(support): support tiers + SLA + per-alert runbooks + incident response (Step 7.8)
#182	2026-06-10	feat(billing): per-tenant usage metering + usage API + invoice generation (Step 7.9)
#183	2026-06-10	chore(release): GA cutover — v1.0.0 release notes + readiness checklist + SemVer policy (Step 7.10)
#184	2026-06-10	docs: GA polish — site publishing scope, HLD v1.2, ragctl reference, port + link fixes
#185	2026-06-10	fix(gateway): v1.0.1 — container/Helm serve path: config-driven entrypoint + /readyz + valid chart config
#189	2026-06-11	feat(gateway,admin-ui): corpus create/delete admin CRUD (Step 3.10)
#78–#80, #116–#118	Open	Dependabot bumps — awaiting merge
#81	Closed	Dependabot bump — superseded

Branch Naming Convention

build/phase-{N}/step-{N}.{M}-{short-slug}

Examples:

build/phase-0/step-0.2-core-types
build/phase-1/step-1.5-chunker
build/phase-3/step-3.1-gateway-rest

FilesExpand file tree

TRACKER.md

Latest commit

History