feat(gateway): audit read API + chain verification (Step 6.6a)

TRACKER.md

@@ -16,10 +16,11 @@
 | **Last updated** | 2026-06-08 |
 | **Current phase** | Phase 6 — Governance & Tenancy (**5 / 10 steps**) |
 | **Overall** | **69 / 84 steps** — Phases 0–5 complete |
-| **Next action** | **Step 6.6 — Immutable audit log**: hash-chain audit log with tamper-evident verification + WORM export; `GET /v1/audit`. Builds on the Step 0.7c `AuditStore` SPI + `AuditWriter` (SHA-256 hash chain) already in `rag-core`. |
+| **Next action** | **Step 6.6b — WORM signed audit export**: an HMAC-signed export bundle over the events + chain head (reusing the `ProvenanceSigner` pattern), `POST /v1/audit/export` + `ragctl audit export/verify` → immutability at rest (write to S3 Object Lock). Completes Step 6.6 (6.6a — the read API + chain verification — shipped). |
 
 **Recently shipped**
 
+- **6.6a** 🚧 Audit read API + chain verification — shared `AuditWriter`/store on `app.state`; `GET /v1/audit` (tenant-scoped, newest-first, `chain_verified`) + `GET /v1/audit/verify` (whole-log integrity); `cfg.audit.enabled` (default on); first slice of Step 6.6 (WORM export is 6.6b) — [#153](https://github.com/officialCodeWork/AgentContextOS/pull/153)
 - **6.5** ✅ PII egress policies — `PiiPolicyEngine` (`rag-pii`) answers `egress_text` over the context (`list[Chunk]`) + agent answer (`str`) the gateway already passes, applying the per-tenant `pii_policy` (allow / redact / mask / block); reuses the Step 1.7 detector + rewriters; opt-in `cfg.pii.enabled`; emits `pii.egress_blocked` — [#152](https://github.com/officialCodeWork/AgentContextOS/pull/152)
 - **6.4** ✅ ACL egress verifier — `AclEgressVerifier` re-checks returned `ChunkRef`s against the principal's labels at the gateway router boundary (defense-in-depth, same overlap semantics, independent of the PDP); `acl.egress_violation` event; red-team zero-violation-rate gate — [#151](https://github.com/officialCodeWork/AgentContextOS/pull/151)
 - **6.3** ✅ ACL push-down — opt-in `AclPolicyEngine` And-merges `any_in("acl_labels", principal.acl_labels)` into every `read_chunk` query (overlap, fail-closed); `acl.egress_denied` event — [#150](https://github.com/officialCodeWork/AgentContextOS/pull/150)
@@ -653,7 +654,7 @@
 | 6.3 | ACL push-down at retrieval | ✅ | [#150](https://github.com/officialCodeWork/AgentContextOS/pull/150) — opt-in `AclPolicyEngine` And-merges `any_in("acl_labels", …)` into every `read_chunk` push-down (overlap, fail-closed); `acl.egress_denied` |
 | 6.4 | ACL egress verifier | ✅ | [#151](https://github.com/officialCodeWork/AgentContextOS/pull/151) — `AclEgressVerifier` re-checks returned chunks at the gateway router boundary (defense-in-depth above the 6.3 push-down); `acl.egress_violation`; zero-violation-rate red-team gate |
 | 6.5 | PII policies | ✅ | [#152](https://github.com/officialCodeWork/AgentContextOS/pull/152) — `PiiPolicyEngine` egress_text decorator (allow / redact / mask / block per tenant) over answer + context; reuses Step 1.7 detector; `pii.egress_blocked` |
-| 6.6 | Immutable audit log | ⏳ | Hash-chain audit log; WORM export; tamper-evident verification; `GET /v1/audit` |
+| 6.6 | Immutable audit log | 🚧 | **6.6a** ✅ [#153](https://github.com/officialCodeWork/AgentContextOS/pull/153) — read API `GET /v1/audit` (tenant-scoped) + `GET /v1/audit/verify` (whole-log chain) + shared store wiring + `cfg.audit`.  **6.6b** ⏳ — WORM signed export |
 | 6.7 | BYOK (Bring Your Own Key) | ⏳ | KMS integration (AWS KMS, GCP KMS, HashiCorp Vault); envelope encryption for embeddings |
 | 6.8 | SSO / SCIM | ⏳ | OIDC + SAML IdP federation; SCIM 2.0 user provisioning; per-tenant IdP config |
 | 6.9 | Air-gapped install bundle | ⏳ | Signed tarball with all images + Helm chart; offline bootstrap; cosign verification |
@@ -705,6 +706,19 @@
 - **Opt-in** via new `cfg.pii.enabled` (default off — it can redact / withhold content); `build_app(pii_enabled=…, pii_detector=…)` wraps the engine after the acl wrap, `build_app_from_config` from config; `app.state.pii_enabled` reports the state.  Events are **PII-free**: `pii.egress_blocked` (block, the alertable denial — the reserved event from Step 0.7b now emitted) / `pii.detected` (redact/mask) carry entity *types* + counts + surface, never the matched values
 - **Scope:** enforces at the existing `egress_text` sites (context for query/OpenAI/gRPC, answer for the agent).  **Deferred:** a post-generation answer re-check for the query/OpenAI/gRPC generate paths (they sanitise the *context* pre-LLM) and a PII check on retrieval-only citations — stored chunks are already ingest-sanitised (1.7).  `PiiConfig` → `rag.schema` regenerated; ~24 new tests (engine unit: str/chunks × allow/redact/mask/block, min_score/entities, events, delegation; gateway wiring + composition with ACL + behavioral redact/block); all gates green (ruff, mypy --strict 297 files, RAG001, schema-drift, policy-coverage, log-schema/PII gate).  [ADR-0037](docs/adr/ADR-0037-pii-egress-policies.md), [architecture/pii.md](docs/architecture/pii.md), [architecture/policy-engine.md](docs/architecture/policy-engine.md), [reference/pii.md](docs/reference/pii.md)
 
+### 6.6 — Immutable audit log 🚧 (sliced 6.6a + 6.6b)
+
+Step 0.7c shipped the foundation — an append-only `AuditStore` SPI, a `NoopAuditStore` that links events into a **SHA-256 hash chain**, and an `AuditWriter` facade — but the log was **write-only** (only the corpus router wrote `corpus.route`, into a store nothing could read or verify).  Step 6.6 makes it usable + provably intact, delivered as two vertical slices.
+
+#### 6.6a — Audit read API + chain verification ✅ [#153](https://github.com/officialCodeWork/AgentContextOS/pull/153)
+
+- **One shared audit store.**  `build_app` now creates a single `AuditWriter` over a hash-chain store, exposes **`app.state.audit_store` / `audit_writer` / `audit_enabled`**, and hands the *same* writer to the corpus router — so the events the writers append land in exactly the store the read API serves (before this, the corpus router held a private `NoopAuditStore` the gateway couldn't read).  New `AuditWriter.store` property is the read accessor; default store is the in-memory `NoopAuditStore` (creds-free chain), production injects a durable one
+- **`GET /v1/audit`** — **tenant-scoped** (a principal sees only its own tenant's events, filtered by `ctx.tenant_id` at the boundary — same pattern as `GET /v1/query/{id}/trace`), newest-first, bounded by `limit` (clamped `[1,1000]`), optional `action` / `outcome` filters.  The `AuditListResponse` carries **`chain_verified`** so one read both returns the events and attests the log is untampered
+- **`GET /v1/audit/verify`** — **whole-log** hash-chain integrity (the chain is one global sequence across tenants), returning `{ok, event_count}` — a boolean + total, no event content, so global verification leaks nothing cross-tenant
+- **The hash chain is the tamper-*evidence* mechanism** (no second scheme): `verify_chain()` recomputes every link and fails if any event or stored hash was altered.  Immutability *at rest* (preventing deletion/replacement of the store) is the 6.6b WORM export's job
+- **Read API on by default**: new `cfg.audit.enabled` (default **true**) — unlike the behaviour-changing ACL / PII toggles, the audit log is a passive, tenant-scoped compliance record, so exposing it out of the box is the expected enterprise default; off → endpoints 404 (`AuditNotFoundError`), events still recorded.  New `AuditListResponse` / `AuditVerifyResponse` wire types (`rag_core.gateway_types`); `AuditError` / `AuditNotFoundError` (→ 404); `dist/openapi` + `dist/rag.schema` regenerated
+- **Scope:** read + verify only (WORM signed export is 6.6b); today the populated event is `corpus.route` (every query) — expanding audit coverage (ACL / PII / ingest decisions) is a follow-up, the surface + chain are in place for it.  ~10 gateway tests (tenant-scoped list, **cross-tenant isolation**, verify ok + **tamper detection**, filters/pagination, disabled→404, no-auth→401, shared-store wiring); all gates green (ruff, mypy --strict 298 files, RAG001, schema/openapi-drift, policy-coverage, log-schema).  [ADR-0038](docs/adr/ADR-0038-immutable-audit-log.md), [architecture/audit-log.md](docs/architecture/audit-log.md), [reference/audit.md](docs/reference/audit.md)
+
 ---
 
 ## Phase 7 — Pilot, Harden, GA (Weeks 34–40) ⏳
@@ -861,6 +875,7 @@ Complete log of every PR. Routine Dependabot bumps are grouped; everything else
 | [#150](https://github.com/officialCodeWork/AgentContextOS/pull/150) | 2026-06-08 | feat(policy): ACL push-down at retrieval — AclPolicyEngine (Step 6.3) |
 | [#151](https://github.com/officialCodeWork/AgentContextOS/pull/151) | 2026-06-08 | feat(policy): ACL egress verifier — defense-in-depth re-check (Step 6.4) |
 | [#152](https://github.com/officialCodeWork/AgentContextOS/pull/152) | 2026-06-08 | feat(pii): PII egress policies — PiiPolicyEngine egress_text decorator (Step 6.5) |
+| [#153](https://github.com/officialCodeWork/AgentContextOS/pull/153) | 2026-06-08 | feat(gateway): audit read API + chain verification (Step 6.6a) |
 | #78–#80, #116–#118 | Open | Dependabot bumps — awaiting merge |
 | #81 | Closed | Dependabot bump — superseded |
 

apps/gateway/src/rag_gateway/app.py

@@ -113,6 +113,7 @@
 from rag_webhooks import build_default_dispatcher, ingest_completed_event
 
 from rag_gateway.agent import build_default_agent_loop, make_agent_router
+from rag_gateway.audit import make_audit_router
 from rag_gateway.corpora import make_corpora_router
 from rag_gateway.feedback import make_feedback_router
 from rag_gateway.middleware import install_request_context_middleware
@@ -372,6 +373,7 @@ def build_app(
     acl_verify_egress: bool = True,
     pii_enabled: bool = False,
     pii_detector: Any | None = None,
+    audit_enabled: bool = True,
     enable_cors: bool = True,
     default_tenant_id: TenantId | None = None,
 ) -> FastAPI:
@@ -464,9 +466,20 @@ def build_app(
     app.state.llm = llm if llm is not None else NoopLLM()
     app.state.embedder = embedder or build_default_embedder()
     app.state.corpus_store = corpus_store or NoopCorpusStore()
+    # Immutable audit log (Step 6.6) — one AuditWriter over a tamper-evident
+    # SHA-256 hash-chain store, shared by the writers (the corpus router today)
+    # and the GET /v1/audit read API.  Defaults to an in-memory NoopAuditStore so
+    # the chain is exercised creds-free; production injects a durable store.
+    if audit_writer is None:
+        audit_writer = AuditWriter(NoopAuditStore())
+    app.state.audit_writer = audit_writer
+    app.state.audit_store = audit_writer.store
+    app.state.audit_enabled = audit_enabled
     # Corpus router (Step 3.5) sits above the retrieval router; with the
     # default empty corpus store it degrades to a single unconstrained
-    # retrieval, preserving the pre-3.5 demo behaviour.
+    # retrieval, preserving the pre-3.5 demo behaviour.  It shares the audit
+    # writer above so its ``corpus.route`` events land in the same chain the
+    # read API serves.
     app.state.corpus_router = corpus_router or build_default_corpus_router(
         corpus_store=app.state.corpus_store,
         retrieval_router=app.state.retrieval_router,
@@ -701,6 +714,8 @@ async def info() -> dict[str, Any]:
                 "GET  /v1/status/drift",
                 "GET  /v1/status/logs",
                 "GET  /v1/status/logs/stream",
+                "GET  /v1/audit",
+                "GET  /v1/audit/verify",
                 "WS   /v1/status/ws",
                 "GET  /v1/connectors/status",
                 "GET  /healthz",
@@ -786,6 +801,7 @@ async def ingest_document(
     app.include_router(make_webhooks_router())
     app.include_router(make_feedback_router())
     app.include_router(make_status_router())
+    app.include_router(make_audit_router())
 
     return app
 

apps/gateway/src/rag_gateway/audit.py

@@ -0,0 +1,115 @@
+"""Audit-log read API — ``GET /v1/audit`` + ``/v1/audit/verify`` (Step 6.6).
+
+Exposes the tamper-evident SHA-256 hash-chain audit store (Step 0.7c) over HTTP:
+
+* ``GET /v1/audit`` — the **calling tenant's own** audit events, newest-first,
+  bounded by ``limit`` and optionally filtered by ``action`` / ``outcome``.  A
+  tenant never sees another tenant's events.  The response carries
+  ``chain_verified`` so a single read both returns the events and attests that
+  the underlying log is untampered.
+* ``GET /v1/audit/verify`` — whole-log hash-chain integrity (the chain links all
+  events into one sequence, so verification is inherently global).
+
+Both require tenant identity (``Authorization`` / ``X-Tenant-Id`` headers) and
+return 404 when the read surface is disabled (``cfg.audit.enabled = false``).
+The store is read off ``app.state.audit_store`` — the same instance the writers
+(the corpus router today) append to.  The signed WORM export is Step 6.6b.
+"""
+
+from __future__ import annotations
+
+from fastapi import APIRouter, Request
+from rag_core.errors import AuditNotFoundError, AuthError
+from rag_core.gateway_types import AuditListResponse, AuditVerifyResponse, GatewayError
+from rag_core.spi.audit_store import AuditStore
+from rag_core.types import AuditEvent, RequestContext
+
+_MAX_LIMIT = 1000
+
+__all__ = ["make_audit_router"]
+
+
+def _require_ctx(request: Request) -> RequestContext:
+    """Pull the per-request RequestContext off ``request.state.gateway``."""
+    gateway = getattr(request.state, "gateway", None)
+    if gateway is None:
+        raise AuthError(
+            "request did not pass through the gateway middleware — server misconfigured"
+        )
+    if gateway.auth_error is not None:
+        raise gateway.auth_error
+    ctx: RequestContext | None = gateway.ctx
+    if ctx is None:
+        raise AuthError("missing tenant / principal credentials")
+    return ctx
+
+
+def _require_store(request: Request) -> AuditStore:
+    """Return the wired audit store, or 404 when the read surface is disabled."""
+    state = request.app.state
+    if not getattr(state, "audit_enabled", False):
+        raise AuditNotFoundError("the audit-log read API is disabled on this gateway")
+    store: AuditStore | None = getattr(state, "audit_store", None)
+    if store is None:
+        raise AuditNotFoundError("no audit store is wired on this gateway")
+    return store
+
+
+def make_audit_router() -> APIRouter:
+    """Build the audit read-API router (Step 6.6)."""
+    router = APIRouter(tags=["audit"])
+
+    @router.get(
+        "/v1/audit",
+        response_model=AuditListResponse,
+        responses={
+            401: {"model": GatewayError, "description": "Missing or invalid auth"},
+            404: {"model": GatewayError, "description": "Audit read API disabled"},
+        },
+        summary="The tenant's own audit events, newest-first (Step 6.6)",
+    )
+    async def list_audit(
+        request: Request,
+        limit: int = 100,
+        action: str | None = None,
+        outcome: str | None = None,
+    ) -> AuditListResponse:
+        """Return the calling tenant's audit events (tamper-evident hash chain)."""
+        ctx = _require_ctx(request)
+        store = _require_store(request)
+
+        # Tenant isolation: only the caller's own events ever leave the boundary.
+        mine = [e for e in store.events() if e.tenant_id == ctx.tenant_id]
+        total = len(mine)
+        filtered: list[AuditEvent] = [
+            e
+            for e in mine
+            if (action is None or e.action == action)
+            and (outcome is None or e.outcome.value == outcome)
+        ]
+        bounded = max(1, min(limit, _MAX_LIMIT))
+        newest_first = list(reversed(filtered))[:bounded]
+        return AuditListResponse(
+            tenant_id=ctx.tenant_id,
+            events=newest_first,
+            returned=len(newest_first),
+            total=total,
+            chain_verified=store.verify_chain(),
+        )
+
+    @router.get(
+        "/v1/audit/verify",
+        response_model=AuditVerifyResponse,
+        responses={
+            401: {"model": GatewayError, "description": "Missing or invalid auth"},
+            404: {"model": GatewayError, "description": "Audit read API disabled"},
+        },
+        summary="Whole-log hash-chain integrity check (Step 6.6)",
+    )
+    async def verify_audit(request: Request) -> AuditVerifyResponse:
+        """Verify the whole-log SHA-256 hash chain — True iff no link was altered."""
+        _require_ctx(request)
+        store = _require_store(request)
+        return AuditVerifyResponse(ok=store.verify_chain(), event_count=len(store.events()))
+
+    return router

apps/gateway/src/rag_gateway/query.py

@@ -34,6 +34,7 @@
 from rag_core import get_logger
 from rag_core.errors import (
     ACLDeniedError,
+    AuditNotFoundError,
     AuthError,
     ProvenanceNotFoundError,
     RagError,
@@ -103,6 +104,9 @@ def _http_status_for(exc: BaseException) -> int:
     if isinstance(exc, ProvenanceNotFoundError):
         # No provenance record for this query id (unknown / evicted / other tenant).
         return 404
+    if isinstance(exc, AuditNotFoundError):
+        # Audit read API disabled / no store wired (Step 6.6).
+        return 404
     if isinstance(exc, RetrievalError):
         # Retrieval errors are bad-gateway because they indicate a
         # downstream backend failure, not bad input.

apps/gateway/src/rag_gateway/wiring.py

@@ -710,6 +710,10 @@ def build_app_from_config(cfg: RagConfig, **overrides: Any) -> FastAPI:
     # Per-tenant PII enforcement at egress (Step 6.5) — opt-in; off by default.
     pii_enabled = overrides.pop("pii_enabled", cfg.pii.enabled)
 
+    # Immutable audit-log read API (Step 6.6) — on by default (tenant-scoped,
+    # passive compliance record); flip off to withhold the HTTP read surface.
+    audit_enabled = overrides.pop("audit_enabled", cfg.audit.enabled)
+
     return build_app(
         corpus_store=corpus_store,
         retrieval_router=retrieval_router,
@@ -722,6 +726,7 @@ def build_app_from_config(cfg: RagConfig, **overrides: Any) -> FastAPI:
         acl_enabled=acl_enabled,
         acl_verify_egress=acl_verify_egress,
         pii_enabled=pii_enabled,
+        audit_enabled=audit_enabled,
         **overrides,
     )