feat(docs): Docusaurus documentation site + generated API reference + doc tests (Step 7.5)

.github/workflows/docs.yml

@@ -0,0 +1,84 @@
+name: Docs
+
+on:
+  pull_request:
+    paths:
+      - "docs/**"
+      - "website/**"
+      - "dist/openapi.json"
+      - "scripts/gen_api_reference.py"
+      - "tests/docs/**"
+      - ".github/workflows/docs.yml"
+  push:
+    branches: [main]
+    paths:
+      - "docs/**"
+      - "website/**"
+      - "dist/openapi.json"
+      - "scripts/gen_api_reference.py"
+
+permissions:
+  contents: read
+
+jobs:
+  api-reference-drift:
+    name: API reference drift
+    runs-on: ubuntu-22.04
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Regenerate + diff rest-api.md
+        run: |
+          python scripts/gen_api_reference.py
+          git diff --exit-code docs/reference/rest-api.md \
+            || { echo "::error::docs/reference/rest-api.md is stale — run 'task docs:api'"; exit 1; }
+
+  quickstart-tests:
+    name: Quickstart doc tests
+    runs-on: ubuntu-22.04
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v6
+      - run: uv run pytest tests/docs -q
+
+  spell-check:
+    name: Spell check
+    runs-on: ubuntu-22.04
+    steps:
+      - uses: actions/checkout@v4
+      - uses: codespell-project/actions-codespell@v2
+        with:
+          path: docs
+          skip: "*.svg,*.json,*.html,./dist,./docs/research"
+          ignore_words_list: "ba,nin,te,ist,fo,ot,assertIn,iif"
+
+  link-check:
+    name: Link check (offline)
+    runs-on: ubuntu-22.04
+    steps:
+      - uses: actions/checkout@v4
+      - uses: lycheeverse/lychee-action@v2
+        with:
+          args: "--offline --include-fragments docs/**/*.md website/README.md README.md"
+          fail: true
+
+  build:
+    name: Docusaurus build
+    runs-on: ubuntu-22.04
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 20
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Generate API reference
+        run: python scripts/gen_api_reference.py
+      - name: Build site
+        run: |
+          cd website
+          npm install
+          npm run build

TRACKER.md

@@ -14,12 +14,13 @@
 | | |
 |---|---|
 | **Last updated** | 2026-06-10 |
-| **Current phase** | Phase 7 — Pilot, Harden, GA (**4 / 10 steps**) — Phases 0–6 ✅ complete |
-| **Overall** | **78 / 84 steps** — Phases 0–6 complete |
-| **Next action** | **Step 7.5 — Documentation site**: Docusaurus / MkDocs site; API reference generated from OpenAPI; quickstart + concept + how-to + ops guides; versioned docs; link-check + spell-check in CI. |
+| **Current phase** | Phase 7 — Pilot, Harden, GA (**5 / 10 steps**) — Phases 0–6 ✅ complete |
+| **Overall** | **79 / 84 steps** — Phases 0–6 complete |
+| **Next action** | **Step 7.6 — Marketplace listings**: AWS / Azure / GCP Marketplace listings (SaaS offer + self-hosted AMI/Helm); each with pricing, SLA, support tiers, terms, integration guide; documented procurement path. |
 
 **Recently shipped**
 
+- **7.5** ✅ Documentation site — a **Docusaurus 3** app (`website/`) that serves the repo `docs/` tree **in place** (single source of truth; `markdown.format: 'detect'` renders hand-written `.md` as CommonMark).  The **REST API reference** (`docs/reference/rest-api.md`) is **generated + drift-gated** from `dist/openapi.json` by `scripts/gen_api_reference.py` (`task docs:api`) — never hand-edited.  Doc honesty is a `tests/docs/` suite (every `ragctl` / `/v1/` reference in a quickstart must be real) + `lychee` offline link-check + `codespell`, wired into a new `docs` CI workflow.  Deferred: Algolia search, versioned docs, the deploy workflow.  [ADR-0047](docs/adr/ADR-0047-documentation-site.md)  [#178](https://github.com/officialCodeWork/AgentContextOS/pull/178)
 - **7.4** ✅ Design-partner pilots — the pilot program shipped as a **repo-backed machine** (signing the referenceable partners is the external GTM deliverable, like the 7.3 pentest): an operator runbook + five fill-in templates (`docs/pilots/`), a **customer-support/KB kit** (`docs/pilots/customer-support/` — sample corpus with synthetic PII + a planted injection probe; domain-calibrated criteria), and **`ragctl pilot`** (`onboard` renders the per-tenant config + checklist; `report` assembles a weekly-KPI dashboard + **PASS/FAIL** verdict from the platform's own feedback / drift / cost signal components — the same ones behind `GET /v1/status/*`).  KPIs are **pulled, not self-reported**.  A worked case study runs it end-to-end on the kit with real `ragctl pilot report` output (satisfaction **+0.733**, **0/5** drift, cost **ok** → **PASS**) + the PII-redaction + injection-blocked security demo.  No `dist`/SPI/config change.  [#174](https://github.com/officialCodeWork/AgentContextOS/pull/174)
 - **7.3** ✅ Red-team / security — turns the governance stack into an **adversarial probe gate** across four classes (prompt injection / PII egress / ACL bypass / tenant escape) under a `redteam` marker + a first-class **`redteam-gate`** CI job (`task redteam`).  Closes the verified injection gap: new **`rag-injection`** package — a pluggable `InjectionDetector` (dependency-free `HeuristicInjectionDetector`, regexes anchored on attack *grammar* so benign prose isn't flagged) + `PromptInjectionGuard.inspect` that drops hijack chunks **before the LLM**, paired with `INJECTION_RESISTANT_SYSTEM_PROMPT` + `build_user_message` so untrusted context is **fenced data in the user turn, never a system-trust position** (fixes the OpenAI-chat surface that put context in a `system` message); wired on `/v1/query` + `/v1/chat/completions` + MCP; off by default (`cfg.injection`); PII-free `injection.blocked` event.  Deterministic gate: a ≥ 500 known + ≥ 500 generated corpus (`eval/redteam_v0/`) hits **96.6 % block** (≥ 95 % bar) at **0 false positives**, plus the no-system-position invariant end-to-end — building the corpus *hardened the detector* (~20 missed phrasings).  New PII-egress probe over `PiiPolicyEngine` (zero leakage, second-detector verified); `pip-audit` is the CVE gate; the external pentest is a documented process item.  Injection types stay internal (no attacker signal, `dist/schemas`/`openapi` untouched).  [#166](https://github.com/officialCodeWork/AgentContextOS/pull/166)
 - **7.2** ✅ Chaos engineering — a deterministic in-process **kill-matrix gate** (`eval/gateway_chaos_v0/kill_matrix.py` + `tests/perf/test_chaos_kill_matrix.py`) that extends 7.1 from the three retrieval backends to the **full hot-path set** (vector/keyword/graph/embedder/retrieval_cache/reranker/llm): kill each backend in turn (100% unavailable) behind the real breakers + fallback, drive `/v1/query`, and assert **no single failure 5xx-es the gateway** (no 5xx, on-path retrieval breaker opens, expected degraded shape; a seeded keyword corpus + real `hydrate` make rerank/generate actually run).  **Chaos fixed what it found** — the matrix exposed that a down **retrieval cache** or **reranker** 5xx-ed, so the gateway gained two minimal **degrade-open** guards (`gateway.cache.degraded` → miss on `/v1/query` + `/v1/retrieve`; `gateway.rerank.degraded` → retrieval-only, honouring `RerankPipeline`'s "caller decides" contract); LLM + embedder already degraded.  **LitmusChaos cluster manifests** (`infra/chaos/`: gateway pod-delete + backend `pod-network-loss`/`latency` with httpProbe acceptance) as the cluster runbook + `task chaos-kill`.  No `dist/`/SPI/config change.  [#165](https://github.com/officialCodeWork/AgentContextOS/pull/165)
@@ -806,7 +807,7 @@ New ground — the only prior crypto was HMAC signing.  The V1 plan calls for en
 | 7.2 | Chaos engineering | ✅ | [#165](https://github.com/officialCodeWork/AgentContextOS/pull/165) — in-process **kill-matrix** gate (kill each hot-path backend → no 5xx, breaker opens, expected degraded shape); two new gateway **degrade-open** guards (retrieval cache + reranker); LitmusChaos cluster manifests (`infra/chaos/`) + `task chaos-kill` |
 | 7.3 | Red-team | ✅ | [#166](https://github.com/officialCodeWork/AgentContextOS/pull/166) — adversarial probe gate (injection / PII / ACL / tenant-escape) under a `redteam` marker + `redteam-gate` CI job; new **`rag-injection`** guard (≥ 95 % block, no untrusted chunk in a system-trust position) wired on every answer surface; PII-egress probe; `task redteam` |
 | 7.4 | Design partner onboarding | ✅ | [#174](https://github.com/officialCodeWork/AgentContextOS/pull/174) — repo-backed pilot program: operator runbook + 5 templates (`docs/pilots/`) + ADR-0046; **customer-support/KB kit** (corpus w/ PII + planted injection probe; domain criteria); **`ragctl pilot`** `onboard` + `report` (weekly KPIs + PASS/FAIL verdict from feedback/drift/cost signal components, KPIs pulled not self-reported); worked case study (real `pilot report` → PASS). Signing partners = external GTM item |
-| 7.5 | Documentation site | ⏳ | Docusaurus / MkDocs site; API reference generated from OpenAPI; quickstart guides |
+| 7.5 | Documentation site | ✅ | [#178](https://github.com/officialCodeWork/AgentContextOS/pull/178) — Docusaurus app (`website/`) serving `docs/` in place; **generated + drift-gated REST API reference** from `dist/openapi.json`; `tests/docs/` quickstart-honesty suite + `lychee`/`codespell`; `docs` CI workflow; `task docs:api`/`docs:build` |
 | 7.6 | Marketplace listings | ⏳ | AWS / Azure / GCP Marketplace AMI / Helm listings |
 | 7.7 | Packaging & distribution | ⏳ | PyPI publish (`rag-platform`), npm publish (`@ragplatform/sdk`), Docker Hub images |
 | 7.8 | Support & SLA | ⏳ | Support tiers defined; SLA dashboards; PagerDuty integration; runbooks |
@@ -849,6 +850,14 @@ New ground — the only prior crypto was HMAC signing.  The V1 plan calls for en
 - **7.4d — worked case study.**  [docs/pilots/customer-support/case-study.md](docs/pilots/customer-support/case-study.md) runs the framework end-to-end on the kit with **real `ragctl pilot report` output** (satisfaction **+0.733**, **0/5** monitors drifting, cost **ok** → **PASS**) + the PII-redaction + injection-blocked security demonstration — framed honestly as a representative pilot (a real partner substitutes their corpus + users).
 - **Scope:** the framework + the first vertical kit + the `ragctl pilot` reader + the worked case study.  **No new package / core or wire type / governed SPI call / `dist` change** — `pilot` is a pure `ragctl` reader over existing signal components, so the policy-coverage + schema/openapi-drift gates need no new entry.  **Deferred / external:** sourcing + signing the real referenceable partners; the legal + internal-engineering kits; per-pilot golden-set *generation* tooling; a pilot KPI card in the admin console.  All gates green (ruff, mypy --strict, RAG001, link-check).  [ADR-0046](docs/adr/ADR-0046-design-partner-pilots.md), [guides/design-partner-pilots.md](docs/guides/design-partner-pilots.md), [reference/pilot.md](docs/reference/pilot.md).
 
+### 7.5 — Documentation site ✅ [#178](https://github.com/officialCodeWork/AgentContextOS/pull/178)
+
+- A public docs site that **can't drift from the repo**: a **Docusaurus 3** app under `website/` whose docs plugin points at `../docs` (`routeBasePath: '/'`), so the PR-reviewed `docs/` tree *is* the site — no copy step.  `markdown.format: 'detect'` renders hand-written `.md` as CommonMark (no MDX escaping of `<` / `{`).
+- **The REST API reference is a generated, drift-gated artifact.**  `scripts/gen_api_reference.py` (pure stdlib) renders the 43-path `dist/openapi.json` → `docs/reference/rest-api.md`; the `docs` CI workflow regenerates + diffs it, exactly like `dist/schemas` / the proto stubs — so the published API docs always match the gateway contract.  `task docs:api` regenerates.
+- **Doc honesty is tested, not asserted.**  `tests/docs/` extracts every `ragctl <subcommand>` and `/v1/...` path from the quickstart guides and fails if any isn't a registered CLI command / a real OpenAPI path — the lightest deterministic form of "every quickstart is exercised", no services needed (building it caught + fixed a newline-spanning regex false-positive).
+- **New `docs` CI workflow** (`.github/workflows/docs.yml`): API-reference drift · quickstart doc-tests · `codespell` · `lychee --offline` link-check · full `npm run build`.  `task docs:dev` / `docs:build` run/build locally.
+- **Scope:** the site scaffold + generated reference + doc-tests + CI + `task` targets.  **No code / `dist` change** beyond the new generated reference file.  **Deferred:** Algolia DocSearch, versioned docs aligned to releases, the deploy workflow + custom domain, gRPC/MCP reference auto-generation, executable (vs existence-checked) snippets.  Verified locally (the npm site build is a CI step; the Python generator + the 3 doc-tests run green).  [ADR-0047](docs/adr/ADR-0047-documentation-site.md), [guides/documentation-site.md](docs/guides/documentation-site.md).
+
 ---
 
 ## PR & Branch History
@@ -997,6 +1006,10 @@ Complete log of every PR. Routine Dependabot bumps are grouped; everything else
 | [#165](https://github.com/officialCodeWork/AgentContextOS/pull/165) | 2026-06-08 | test(perf): chaos kill-matrix gate + cache/rerank degrade-open + LitmusChaos (Step 7.2) |
 | [#166](https://github.com/officialCodeWork/AgentContextOS/pull/166) | 2026-06-08 | feat(security): prompt-injection guard (rag-injection) + red-team probe gate (Step 7.3) |
 | [#174](https://github.com/officialCodeWork/AgentContextOS/pull/174) | 2026-06-10 | feat(pilot): design-partner pilot program — framework + customer-support kit + ragctl pilot + case study (Step 7.4) |
+| [#175](https://github.com/officialCodeWork/AgentContextOS/pull/175) | 2026-06-10 | fix(core): carry corpus_id on Embedding for vector corpus scoping (ADR-0004 §3) |
+| [#176](https://github.com/officialCodeWork/AgentContextOS/pull/176) | 2026-06-10 | fix(gateway): scope answers to the question's subject on both answer prompts |
+| [#177](https://github.com/officialCodeWork/AgentContextOS/pull/177) | 2026-06-10 | feat(admin-ui): wire dashboard + audit page to live data; clearer trace help |
+| [#178](https://github.com/officialCodeWork/AgentContextOS/pull/178) | 2026-06-10 | feat(docs): Docusaurus documentation site + generated API reference + doc-tests (Step 7.5) |
 | #78–#80, #116–#118 | Open | Dependabot bumps — awaiting merge |
 | #81 | Closed | Dependabot bump — superseded |
 

Taskfile.yml

@@ -184,6 +184,28 @@ tasks:
     cmds:
       - "{{.PYTHON}} scripts/check_openapi_drift.py"
 
+  # ---------------------------------------------------------------------------
+  # Documentation site (Step 7.5) — Docusaurus under website/, sourcing docs/
+  # ---------------------------------------------------------------------------
+  docs:api:
+    desc: "Regenerate docs/reference/rest-api.md from dist/openapi.json"
+    cmds:
+      - "{{.PYTHON}} scripts/gen_api_reference.py"
+
+  docs:dev:
+    desc: "Run the documentation site locally (http://localhost:3000)"
+    dir: website
+    cmds:
+      - npm install
+      - npm run dev
+
+  docs:build:
+    desc: "Build the static documentation site → website/build/"
+    dir: website
+    cmds:
+      - npm install
+      - npm run build
+
   sdk:gen:
     desc: "Generate all SDK artifacts (proto stubs + OpenAPI clients) into sdks/*/"
     cmds: