feat(pii): PII egress policies — PiiPolicyEngine egress_text decorator (Step 6.5)

TRACKER.md

@@ -14,12 +14,13 @@
 | | |
 |---|---|
 | **Last updated** | 2026-06-08 |
-| **Current phase** | Phase 6 — Governance & Tenancy (**4 / 10 steps**) |
-| **Overall** | **68 / 84 steps** — Phases 0–5 complete |
-| **Next action** | **Step 6.5 — PII policies**: per-tenant PII enforcement at egress (block / redact / allow) through the PolicyEngine `egress_text` decision; `pii.egress_blocked` event. Builds on the per-tenant `pii_policy` already resolved onto `RequestContext` in 6.1. |
+| **Current phase** | Phase 6 — Governance & Tenancy (**5 / 10 steps**) |
+| **Overall** | **69 / 84 steps** — Phases 0–5 complete |
+| **Next action** | **Step 6.6 — Immutable audit log**: hash-chain audit log with tamper-evident verification + WORM export; `GET /v1/audit`. Builds on the Step 0.7c `AuditStore` SPI + `AuditWriter` (SHA-256 hash chain) already in `rag-core`. |
 
 **Recently shipped**
 
+- **6.5** ✅ PII egress policies — `PiiPolicyEngine` (`rag-pii`) answers `egress_text` over the context (`list[Chunk]`) + agent answer (`str`) the gateway already passes, applying the per-tenant `pii_policy` (allow / redact / mask / block); reuses the Step 1.7 detector + rewriters; opt-in `cfg.pii.enabled`; emits `pii.egress_blocked` — [#152](https://github.com/officialCodeWork/AgentContextOS/pull/152)
 - **6.4** ✅ ACL egress verifier — `AclEgressVerifier` re-checks returned `ChunkRef`s against the principal's labels at the gateway router boundary (defense-in-depth, same overlap semantics, independent of the PDP); `acl.egress_violation` event; red-team zero-violation-rate gate — [#151](https://github.com/officialCodeWork/AgentContextOS/pull/151)
 - **6.3** ✅ ACL push-down — opt-in `AclPolicyEngine` And-merges `any_in("acl_labels", principal.acl_labels)` into every `read_chunk` query (overlap, fail-closed); `acl.egress_denied` event — [#150](https://github.com/officialCodeWork/AgentContextOS/pull/150)
 - **6.2** ✅ Physical tenancy — per-tenant *dedicated* vector index/collection; `TenantConfig.dedicated_index` → `ctx.physical_index` → backend `<base>-<key>` (Noop/Pinecone/Qdrant); cross-tenant probe gate — [#149](https://github.com/officialCodeWork/AgentContextOS/pull/149)
@@ -58,9 +59,9 @@
 | 3 | Gateway & Agent Runtime | 11 | **11** | 0 |
 | 4 | Reliability | 6 | **6** | 0 |
 | 5 | Eval & Observability | 7 | **7** | 0 |
-| 6 | Governance & Tenancy | 10 | **4** | 6 |
+| 6 | Governance & Tenancy | 10 | **5** | 5 |
 | 7 | Pilot, Harden, GA | 10 | 0 | 10 |
-| **Total** | | **84** | **68** | **16** |
+| **Total** | | **84** | **69** | **15** |
 
 ---
 
@@ -651,7 +652,7 @@
 | 6.2 | Physical tenancy (dedicated index) | ✅ | [#149](https://github.com/officialCodeWork/AgentContextOS/pull/149) — `dedicated_index` → `ctx.physical_index` → backend `<base>-<key>` (Noop/Pinecone/Qdrant, lazy create); cross-tenant probe gate |
 | 6.3 | ACL push-down at retrieval | ✅ | [#150](https://github.com/officialCodeWork/AgentContextOS/pull/150) — opt-in `AclPolicyEngine` And-merges `any_in("acl_labels", …)` into every `read_chunk` push-down (overlap, fail-closed); `acl.egress_denied` |
 | 6.4 | ACL egress verifier | ✅ | [#151](https://github.com/officialCodeWork/AgentContextOS/pull/151) — `AclEgressVerifier` re-checks returned chunks at the gateway router boundary (defense-in-depth above the 6.3 push-down); `acl.egress_violation`; zero-violation-rate red-team gate |
-| 6.5 | PII policies | ⏳ | Per-tenant PII enforcement (block / redact / allow); egress redaction; `pii.egress_blocked` event |
+| 6.5 | PII policies | ✅ | [#152](https://github.com/officialCodeWork/AgentContextOS/pull/152) — `PiiPolicyEngine` egress_text decorator (allow / redact / mask / block per tenant) over answer + context; reuses Step 1.7 detector; `pii.egress_blocked` |
 | 6.6 | Immutable audit log | ⏳ | Hash-chain audit log; WORM export; tamper-evident verification; `GET /v1/audit` |
 | 6.7 | BYOK (Bring Your Own Key) | ⏳ | KMS integration (AWS KMS, GCP KMS, HashiCorp Vault); envelope encryption for embeddings |
 | 6.8 | SSO / SCIM | ⏳ | OIDC + SAML IdP federation; SCIM 2.0 user provisioning; per-tenant IdP config |
@@ -695,6 +696,15 @@
 - **`acl.egress_violation`** (pre-registered `EVT_ACL_EGRESS_VIOLATION`, `error` level, PII-free: tenant / principal / counts / dropped chunk **ids** only) fires once per call that drops ≥1 ref — the *unexpected* push-down failure, distinct from 6.3's *expected* `acl.egress_denied`; a clean pass is silent
 - **Scope:** gateway retrieval surfaces (everything reading `app.state.retrieval_router`); trusts the labels the backend reports on each `ChunkRef` (catching a *mislabelling* backend needs authoritative re-hydration — deferred), per-tenant/per-label violation metrics deferred to the 6.x governance dashboards.  `AclConfig` → `rag.schema` regenerated; ~21 new tests incl. a **red-team zero-violation-rate gate** (`tests/redteam/test_acl_egress_verifier.py` — bypassed push-down + leaky backend → verifier drops every violation) + verifier unit + gateway wiring + event-schema; all gates green (ruff, mypy --strict 296 files, RAG001, schema-drift, policy-coverage, log-schema).  [ADR-0036](docs/adr/ADR-0036-acl-egress-verifier.md), [architecture/policy-engine.md](docs/architecture/policy-engine.md), [reference/tenancy.md](docs/reference/tenancy.md)
 
+### 6.5 — PII policies ✅ [#152](https://github.com/officialCodeWork/AgentContextOS/pull/152)
+
+- Step 1.7 detects + redacts PII at *ingest*; Step 6.1 resolved each tenant's `pii_policy` onto `RequestContext.pii_policy`; but nothing enforced it at *egress* — the `egress_text` PDP decision existed and the gateway already called it at every generation boundary, yet the default engine answered `allow`.  New **`PiiPolicyEngine`** (`rag-pii`) — a decorator like `QuotaPolicyEngine` / `AclPolicyEngine` — answers `egress_text` and applies the per-tenant action: **allow** delegates, **redact / mask** → `transform`, **block** → `deny`
+- **Handles both subject shapes the gateway already passes** — the retrieved **context** (`list[Chunk]` → the LLM on `/v1/query?generate` / `/v1/chat/completions` / gRPC `Converse`) and the agent's final **answer** (`str`) — so enforcement lands at **all four existing `egress_text` call sites with no route change and no coverage-linter entry** (the existing deny→drop / transform→substitute handling consumes the result unchanged)
+- **Reuses Step 1.7 machinery**: detection via the injected `PIIDetector` SPI (default the dependency-free `RegexPIIDetector`; production injects Presidio), rewriting via the same `redact_spans` / `mask_spans`, and the **same `min_score` + `entities` filter** as the ingest `PiiProcessor` — so the two stages never disagree, and a clean scan is a **no-op**.  `block` on a chunk list is all-or-nothing (any PII denies the whole context); redact/mask rewrite each affected chunk's `content` (clean chunks pass through, same object)
+- Lives in **`rag-pii`** (which gains a `rag-policy` dep, exactly as `rag-quota` does for `QuotaPolicyEngine`; graph stays acyclic — `rag-policy` never imports `rag-pii`).  `filter_pushdown` / `health` / non-`egress_text` decisions delegate inward, so it **composes with the ACL + quota PDP**
+- **Opt-in** via new `cfg.pii.enabled` (default off — it can redact / withhold content); `build_app(pii_enabled=…, pii_detector=…)` wraps the engine after the acl wrap, `build_app_from_config` from config; `app.state.pii_enabled` reports the state.  Events are **PII-free**: `pii.egress_blocked` (block, the alertable denial — the reserved event from Step 0.7b now emitted) / `pii.detected` (redact/mask) carry entity *types* + counts + surface, never the matched values
+- **Scope:** enforces at the existing `egress_text` sites (context for query/OpenAI/gRPC, answer for the agent).  **Deferred:** a post-generation answer re-check for the query/OpenAI/gRPC generate paths (they sanitise the *context* pre-LLM) and a PII check on retrieval-only citations — stored chunks are already ingest-sanitised (1.7).  `PiiConfig` → `rag.schema` regenerated; ~24 new tests (engine unit: str/chunks × allow/redact/mask/block, min_score/entities, events, delegation; gateway wiring + composition with ACL + behavioral redact/block); all gates green (ruff, mypy --strict 297 files, RAG001, schema-drift, policy-coverage, log-schema/PII gate).  [ADR-0037](docs/adr/ADR-0037-pii-egress-policies.md), [architecture/pii.md](docs/architecture/pii.md), [architecture/policy-engine.md](docs/architecture/policy-engine.md), [reference/pii.md](docs/reference/pii.md)
+
 ---
 
 ## Phase 7 — Pilot, Harden, GA (Weeks 34–40) ⏳
@@ -850,6 +860,7 @@ Complete log of every PR. Routine Dependabot bumps are grouped; everything else
 | [#149](https://github.com/officialCodeWork/AgentContextOS/pull/149) | 2026-06-08 | feat(tenancy): physical tenancy — dedicated index per tenant (Step 6.2) |
 | [#150](https://github.com/officialCodeWork/AgentContextOS/pull/150) | 2026-06-08 | feat(policy): ACL push-down at retrieval — AclPolicyEngine (Step 6.3) |
 | [#151](https://github.com/officialCodeWork/AgentContextOS/pull/151) | 2026-06-08 | feat(policy): ACL egress verifier — defense-in-depth re-check (Step 6.4) |
+| [#152](https://github.com/officialCodeWork/AgentContextOS/pull/152) | 2026-06-08 | feat(pii): PII egress policies — PiiPolicyEngine egress_text decorator (Step 6.5) |
 | #78–#80, #116–#118 | Open | Dependabot bumps — awaiting merge |
 | #81 | Closed | Dependabot bump — superseded |
 

apps/gateway/src/rag_gateway/app.py

@@ -370,6 +370,8 @@ def build_app(
     tenant_resolver: Any | None = None,
     acl_enabled: bool = False,
     acl_verify_egress: bool = True,
+    pii_enabled: bool = False,
+    pii_detector: Any | None = None,
     enable_cors: bool = True,
     default_tenant_id: TenantId | None = None,
 ) -> FastAPI:
@@ -518,8 +520,20 @@ def build_app(
         from rag_policy import AclPolicyEngine
 
         policy_engine = AclPolicyEngine(inner=policy_engine)
+    # Per-tenant PII enforcement at egress (Step 6.5) — when enabled, decorate the
+    # engine so the ``egress_text`` decision scans the context / answer leaving the
+    # system and applies the tenant's ``pii_policy`` (allow / redact / mask / block).
+    # Opt-in (it can redact or withhold content); the default detector is the
+    # dependency-free RegexPIIDetector — production injects Presidio.  Plugs into the
+    # gateway's existing egress_text call sites, so no route change is needed.
+    if pii_enabled:
+        from rag_pii import PiiPolicyEngine, RegexPIIDetector
+
+        detector = pii_detector if pii_detector is not None else RegexPIIDetector()
+        policy_engine = PiiPolicyEngine(inner=policy_engine, detector=detector)
     app.state.policy_engine = policy_engine
     app.state.acl_enabled = acl_enabled
+    app.state.pii_enabled = pii_enabled
     app.state.quota_enforcer = quota_enforcer
 
     # Hallucination guard (Step 4.3) — post-generation faithfulness check over

apps/gateway/src/rag_gateway/wiring.py

@@ -707,6 +707,9 @@ def build_app_from_config(cfg: RagConfig, **overrides: Any) -> FastAPI:
     acl_enabled = overrides.pop("acl_enabled", cfg.acl.enabled)
     acl_verify_egress = overrides.pop("acl_verify_egress", cfg.acl.verify_egress)
 
+    # Per-tenant PII enforcement at egress (Step 6.5) — opt-in; off by default.
+    pii_enabled = overrides.pop("pii_enabled", cfg.pii.enabled)
+
     return build_app(
         corpus_store=corpus_store,
         retrieval_router=retrieval_router,
@@ -718,6 +721,7 @@ def build_app_from_config(cfg: RagConfig, **overrides: Any) -> FastAPI:
         quota_enforcer=quota_enforcer,
         acl_enabled=acl_enabled,
         acl_verify_egress=acl_verify_egress,
+        pii_enabled=pii_enabled,
         **overrides,
     )
 

apps/gateway/tests/test_pii.py

@@ -0,0 +1,104 @@
+"""PII egress policy config + gateway wiring (Step 6.5)."""
+
+from __future__ import annotations
+
+from rag_config import RagConfig
+from rag_config.schema import AclConfig, PiiConfig
+from rag_core.types import (
+    Chunk,
+    ChunkId,
+    CorpusId,
+    DocumentId,
+    PiiAction,
+    PiiPolicy,
+    Principal,
+    PrincipalId,
+    PrincipalKind,
+    RequestContext,
+    TenantId,
+)
+from rag_gateway import build_app
+from rag_gateway.wiring import build_app_from_config
+from rag_pii import PiiPolicyEngine
+from rag_policy import AclPolicyEngine, PolicyDecision
+
+
+def _ctx(action: PiiAction) -> RequestContext:
+    tid = TenantId("acme")
+    return RequestContext(
+        tenant_id=tid,
+        principal=Principal(
+            id=PrincipalId("p"), kind=PrincipalKind.user, display_name="p", tenant_id=tid
+        ),
+        pii_policy=PiiPolicy(action=action, min_score=0.0),
+    )
+
+
+# ---------------------------------------------------------------------------
+# config + wiring
+# ---------------------------------------------------------------------------
+def test_pii_config_defaults_off() -> None:
+    assert RagConfig().pii.enabled is False
+
+
+def test_inert_by_default() -> None:
+    app = build_app()
+    assert app.state.pii_enabled is False
+    assert not isinstance(app.state.policy_engine, PiiPolicyEngine)
+
+
+def test_build_from_config_wraps_when_enabled() -> None:
+    app = build_app_from_config(RagConfig(pii=PiiConfig(enabled=True)))
+    assert app.state.pii_enabled is True
+    assert isinstance(app.state.policy_engine, PiiPolicyEngine)
+
+
+def test_build_from_config_inert_when_disabled() -> None:
+    app = build_app_from_config(RagConfig())
+    assert not isinstance(app.state.policy_engine, PiiPolicyEngine)
+
+
+def test_pii_decorates_without_absorbing_inner() -> None:
+    app = build_app(pii_enabled=True)
+    engine = app.state.policy_engine
+    assert isinstance(engine, PiiPolicyEngine)
+    # the inner engine is preserved, so PII composes with — rather than replaces —
+    # the production PDP.
+    assert engine.inner is not None
+
+
+def test_pii_composes_with_acl() -> None:
+    app = build_app_from_config(RagConfig(acl=AclConfig(enabled=True), pii=PiiConfig(enabled=True)))
+    engine = app.state.policy_engine
+    # PII is the outer decorator; ACL is preserved beneath it.
+    assert isinstance(engine, PiiPolicyEngine)
+    assert isinstance(engine.inner, AclPolicyEngine)
+
+
+# ---------------------------------------------------------------------------
+# behavioral — the wired engine enforces over egress_text with the default detector
+# ---------------------------------------------------------------------------
+async def test_wired_engine_blocks_pii_answer() -> None:
+    app = build_app(pii_enabled=True)
+    res = await app.state.policy_engine.evaluate(
+        _ctx(PiiAction.block), PolicyDecision.egress_text, "email me at bob@corp.com"
+    )
+    assert res.is_deny()
+
+
+async def test_wired_engine_redacts_pii_context() -> None:
+    app = build_app(pii_enabled=True)
+    tid = TenantId("acme")
+    chunk = Chunk(
+        id=ChunkId("c1"),
+        document_id=DocumentId("d1"),
+        tenant_id=tid,
+        corpus_id=CorpusId("c"),
+        content="ssn 123-45-6789 on file",
+        position=0,
+    )
+    res = await app.state.policy_engine.evaluate(
+        _ctx(PiiAction.redact), PolicyDecision.egress_text, [chunk]
+    )
+    assert res.is_transform()
+    assert "123-45-6789" not in res.transformed[0].content

dist/rag.schema.json

@@ -965,6 +965,19 @@
       "title": "PIIPolicy",
       "type": "string"
     },
+    "PiiConfig": {
+      "additionalProperties": false,
+      "description": "PII enforcement at egress (Step 6.5).\n\nWhen ``enabled`` the gateway wraps its PolicyEngine in a ``PiiPolicyEngine``\nthat answers the ``egress_text`` decision: it scans the text about to leave the\nsystem \u2014 the retrieved context sent to the LLM (``/v1/query?generate`` /\n``/v1/chat/completions`` / gRPC ``Converse``) and the agent's final answer \u2014\nand applies the **per-tenant** action resolved onto ``RequestContext.pii_policy``\n(Step 6.1, from ``tenants[].pii_policy``): **allow** passes through, **redact** /\n**mask** sanitise the text (``transform``), and **block** withholds it (``deny``).\n\n**Disabled by default** \u2014 turning it on can redact or withhold answers.  The\ndefault detector is the dependency-free ``RegexPIIDetector`` (email / phone /\nSSN / credit-card / IP); production injects Presidio.  A **block** emits a\nPII-free ``pii.egress_blocked`` event; a redact/mask emits ``pii.detected``.",
+      "properties": {
+        "enabled": {
+          "default": false,
+          "title": "Enabled",
+          "type": "boolean"
+        }
+      },
+      "title": "PiiConfig",
+      "type": "object"
+    },
     "PlatformConfig": {
       "additionalProperties": false,
       "properties": {
@@ -1621,6 +1634,9 @@
     "acl": {
       "$ref": "#/$defs/AclConfig"
     },
+    "pii": {
+      "$ref": "#/$defs/PiiConfig"
+    },
     "webhooks": {
       "$ref": "#/$defs/WebhooksConfig"
     },

dist/rag.schema.yaml

@@ -946,6 +946,44 @@ $defs:
     - allow
     title: PIIPolicy
     type: string
+  PiiConfig:
+    additionalProperties: false
+    description: 'PII enforcement at egress (Step 6.5).
+
+
+      When ``enabled`` the gateway wraps its PolicyEngine in a ``PiiPolicyEngine``
+
+      that answers the ``egress_text`` decision: it scans the text about to leave
+      the
+
+      system — the retrieved context sent to the LLM (``/v1/query?generate`` /
+
+      ``/v1/chat/completions`` / gRPC ``Converse``) and the agent''s final answer
+      —
+
+      and applies the **per-tenant** action resolved onto ``RequestContext.pii_policy``
+
+      (Step 6.1, from ``tenants[].pii_policy``): **allow** passes through, **redact**
+      /
+
+      **mask** sanitise the text (``transform``), and **block** withholds it (``deny``).
+
+
+      **Disabled by default** — turning it on can redact or withhold answers.  The
+
+      default detector is the dependency-free ``RegexPIIDetector`` (email / phone
+      /
+
+      SSN / credit-card / IP); production injects Presidio.  A **block** emits a
+
+      PII-free ``pii.egress_blocked`` event; a redact/mask emits ``pii.detected``.'
+    properties:
+      enabled:
+        default: false
+        title: Enabled
+        type: boolean
+    title: PiiConfig
+    type: object
   PlatformConfig:
     additionalProperties: false
     properties:
@@ -1497,6 +1535,8 @@ properties:
     $ref: '#/$defs/QuotaConfig'
   acl:
     $ref: '#/$defs/AclConfig'
+  pii:
+    $ref: '#/$defs/PiiConfig'
   webhooks:
     $ref: '#/$defs/WebhooksConfig'
   provenance: