feat(sso): OIDC/SAML federation + SCIM 2.0 provisioning (Step 6.8)

TRACKER.md

@@ -14,12 +14,13 @@
 | | |
 |---|---|
 | **Last updated** | 2026-06-08 |
-| **Current phase** | Phase 6 — Governance & Tenancy (**7 / 10 steps**) |
-| **Overall** | **71 / 84 steps** — Phases 0–5 complete |
-| **Next action** | **Step 6.8 — SSO / SCIM**: OIDC + SAML IdP federation; SCIM 2.0 user provisioning; per-tenant IdP config. New ground (the gateway's `Auth` SPI + `NoopAuth` header-identity is the seam this builds on). |
+| **Current phase** | Phase 6 — Governance & Tenancy (**8 / 10 steps**) |
+| **Overall** | **72 / 84 steps** — Phases 0–5 complete |
+| **Next action** | **Step 6.9 — Air-gapped install bundle**: signed tarball with all images + Helm chart; offline bootstrap; cosign verification. |
 
 **Recently shipped**
 
+- **6.8** ✅ SSO / SCIM — `rag-sso`: `FederatedAuth` (an `Auth` SPI backend over per-tenant `OidcProvider` / `SamlProvider`) federates a bearer token / SAML assertion → `Principal` at the existing `authenticate` seam (group claims → `acl_labels`); dependency-free defaults (stdlib HS256 JWT + `defusedxml` SAML), asymmetric OIDC / XML-DSig behind `[oidc]` / `[saml]` extras; algorithm-allowlist downgrade defense; SCIM 2.0 `ScimService` over the new tenant-scoped `ScimStore` SPI driving `/scim/v2/{Users,Groups}` (per-tenant bearer token); per-tenant IdP on `tenants[].sso`; `cfg.sso` / `cfg.scim`; `GET /v1/status/sso`; PII-free `sso.*` / `scim.*` events; `ragctl sso` / `scim` — [#159](https://github.com/officialCodeWork/AgentContextOS/pull/159)
 - **6.7** ✅ BYOK envelope encryption — `KeyManager` SPI + `EncryptingStorage` + `EnvelopeKeyManager` (AES-256-GCM DEK + tenant AAD); `LocalKeyManager` + four cloud KMS providers (`Aws`/`Gcp`/`AzureKeyVault`/`Vault`, behind `[kms-*]` extras); `cfg.kms` + per-tenant key + factory; per-tenant isolation + sealing + tamper-evidence; zero-downtime rotation (`RotatingKeyManager` + `rewrap`); `ragctl kms` — [#155](https://github.com/officialCodeWork/AgentContextOS/pull/155)–[#158](https://github.com/officialCodeWork/AgentContextOS/pull/158)
 - **6.6** ✅ Immutable audit log — **6.6a** shared `AuditWriter`/store on `app.state` + `GET /v1/audit` (tenant-scoped, `chain_verified`) + `GET /v1/audit/verify` (whole-log) + `cfg.audit.enabled` ([#153](https://github.com/officialCodeWork/AgentContextOS/pull/153)); **6.6b** `AuditExporter` self-verifying WORM bundle (SHA-256 `content_hash` + HMAC, offline `verify()`), `POST /v1/audit/export` (tenant-scoped) + `ragctl audit`, `cfg.audit.export_secret` ([#154](https://github.com/officialCodeWork/AgentContextOS/pull/154))
 - **6.5** ✅ PII egress policies — `PiiPolicyEngine` (`rag-pii`) answers `egress_text` over the context (`list[Chunk]`) + agent answer (`str`) the gateway already passes, applying the per-tenant `pii_policy` (allow / redact / mask / block); reuses the Step 1.7 detector + rewriters; opt-in `cfg.pii.enabled`; emits `pii.egress_blocked` — [#152](https://github.com/officialCodeWork/AgentContextOS/pull/152)
@@ -61,9 +62,9 @@
 | 3 | Gateway & Agent Runtime | 11 | **11** | 0 |
 | 4 | Reliability | 6 | **6** | 0 |
 | 5 | Eval & Observability | 7 | **7** | 0 |
-| 6 | Governance & Tenancy | 10 | **7** | 3 |
+| 6 | Governance & Tenancy | 10 | **8** | 2 |
 | 7 | Pilot, Harden, GA | 10 | 0 | 10 |
-| **Total** | | **84** | **71** | **13** |
+| **Total** | | **84** | **72** | **12** |
 
 ---
 
@@ -646,7 +647,7 @@
 - **Phase-5 close-out:** Step 5.7 ✅ → **Phase 5 complete (7 / 7)**; deferred items remain documented (per-tenant drift / per-dimension embedding PSI; feedback/breaker/quota Grafana export + Loki-events dashboard; gRPC proto mirror of `corpus_decision` + `experiment`; sequential / multi-metric experiments)
 - [reference/experiments.md](docs/reference/experiments.md), [reference/admin-ui.md](docs/reference/admin-ui.md)
 
-## Phase 6 — Governance & Tenancy (Weeks 28–34) 🚧 (3 / 10)
+## Phase 6 — Governance & Tenancy (Weeks 28–34) 🚧 (2 / 10)
 
 | Step | Title | Status | Planned deliverables |
 |------|-------|:------:|----------------------|
@@ -657,7 +658,7 @@
 | 6.5 | PII policies | ✅ | [#152](https://github.com/officialCodeWork/AgentContextOS/pull/152) — `PiiPolicyEngine` egress_text decorator (allow / redact / mask / block per tenant) over answer + context; reuses Step 1.7 detector; `pii.egress_blocked` |
 | 6.6 | Immutable audit log | ✅ | **6.6a** [#153](https://github.com/officialCodeWork/AgentContextOS/pull/153) — read API `GET /v1/audit` + `GET /v1/audit/verify` + shared store + `cfg.audit`.  **6.6b** [#154](https://github.com/officialCodeWork/AgentContextOS/pull/154) — `AuditExporter` signed WORM bundle + `POST /v1/audit/export` + `ragctl audit` |
 | 6.7 | BYOK (Bring Your Own Key) | ✅ | **6.7a** ✅ [#155](https://github.com/officialCodeWork/AgentContextOS/pull/155) — `KeyManager` SPI + envelope encryption (`LocalKeyManager`) + `EncryptingStorage` + sealing/isolation/tamper.  **6.7b** ✅ [#156](https://github.com/officialCodeWork/AgentContextOS/pull/156) — `cfg.kms` + per-tenant key + `build_key_manager_from_config` factory + `AwsKmsKeyManager`.  **6.7c** ✅ [#157](https://github.com/officialCodeWork/AgentContextOS/pull/157) — `GcpKmsKeyManager` / `AzureKeyVaultKeyManager` / `VaultKeyManager` behind `[kms-*]` extras.  **6.7d** ✅ [#158](https://github.com/officialCodeWork/AgentContextOS/pull/158) — `RotatingKeyManager` zero-downtime rotation + `rewrap` |
-| 6.8 | SSO / SCIM | ⏳ | OIDC + SAML IdP federation; SCIM 2.0 user provisioning; per-tenant IdP config |
+| 6.8 | SSO / SCIM | ✅ | [#159](https://github.com/officialCodeWork/AgentContextOS/pull/159) — `rag-sso`: `FederatedAuth` (`Auth` SPI backend over per-tenant `OidcProvider` / `SamlProvider`) federates a token / assertion → `Principal` at the existing `authenticate` seam (groups → `acl_labels`); stdlib HS256 + `defusedxml` defaults, asymmetric OIDC / XML-DSig behind `[oidc]` / `[saml]` extras; SCIM 2.0 `ScimService` + `ScimStore` SPI → `/scim/v2/*` (per-tenant bearer); `tenants[].sso` + `cfg.sso` / `cfg.scim`; `GET /v1/status/sso`; `ragctl sso` / `scim` |
 | 6.9 | Air-gapped install bundle | ⏳ | Signed tarball with all images + Helm chart; offline bootstrap; cosign verification |
 | 6.10 | Compliance posture | ⏳ | SOC 2 Type II control mapping; GDPR data-residency config; data-retention policies |
 
@@ -759,6 +760,16 @@ New ground — the only prior crypto was HMAC signing.  The V1 plan calls for en
 - Composes with every provider (local + all four cloud KMS) since it orchestrates only the `KeyManager` SPI.  `ragctl kms --rotate` demos the full flow (old + new decrypt, rewrap, expiry seals old data)
 - **Scope:** the rotation *mechanism* (satisfies the planning rotation acceptance — rotate → old + new both decrypt; expired key seals old data).  **Deferred:** config-driven per-tenant multi-generation rotation + the storage-side background re-encryption job land with the `EncryptingStorage` ingest wiring (tiered storage).  ~11 new tests (rotate → old+new decrypt; rewrap migration; expired-key sealing; tamper; no-key sealing; multi-generation; clock-controlled expiry; health) + `ragctl kms --rotate`.  No dist drift; all gates green (ruff, mypy --strict 310 files, RAG001, policy-coverage, log-schema).  [ADR-0039](docs/adr/ADR-0039-byok-envelope-encryption.md), [reference/encryption.md](docs/reference/encryption.md), [architecture/byok.md](docs/architecture/byok.md)
 
+### 6.8 — SSO / SCIM ✅ [#159](https://github.com/officialCodeWork/AgentContextOS/pull/159)
+
+- **The seam is the `Auth` SPI.**  The gateway middleware already calls `auth.authenticate(bearer_token, tenant_id) → Principal` at the boundary (the one SPI method that runs *before* a `RequestContext` exists).  New **`FederatedAuth`** (`rag-sso`) implements it, dispatching to a per-tenant **`OidcProvider`** / **`SamlProvider`**, so wiring it as the gateway's `auth` backend is the **entire integration — no middleware change**.  The returned principal's **`acl_labels` come from the IdP's group claims**, so Step 6.3 ACL push-down + 6.5 PII egress govern federated users unchanged (`authorize` stays a coarse allow — federation establishes *who*, the PolicyEngine decides *what*)
+- **Dependency-free defaults, heavy crypto behind extras** (mirrors BYOK / NLI).  OIDC verification is a real **stdlib HS256** JWT verifier (`verify_jwt`: split → header `alg` **allowlist** check → constant-time `hmac.compare_digest` → `exp` / `nbf` / `iss` / `aud` with leeway); asymmetric **RS256 / ES256** delegates to PyJWT behind the **`[oidc]`** extra against a configured public key.  SAML parses through **`defusedxml`** (a core dep — XXE / billion-laughs safe), validates Issuer / Conditions / AudienceRestriction, and **injects** XML-DSig verification (`signxml_verifier`, **`[saml]`** extra) — `require_signature` on with no verifier **fails closed**
+- **Algorithm-confusion designed out:** the `alg` allowlist rejects `alg:none` and an RS256 token replayed as HS256; symmetric vs asymmetric take *different* key material (`hmac_secret` vs `public_key`)
+- **Per-tenant IdP config on `tenants[].sso`** (reuses the Step 6.1 mechanism; `${ENV}`-interpolated secrets).  A tenant with no `sso` block has no provider → its bearer tokens are rejected (fail-closed) while header-identity dev flows still work.  `build_federated_auth_from_config` builds one provider per tenant; **`NoopAuth`** when `cfg.sso.enabled` is off (pre-6.8 behaviour)
+- **SCIM 2.0 is a separate surface with its own auth.**  New tenant-scoped **`ScimStore`** SPI + **`NoopScimStore`** (CRUD for `ScimUser` / `ScimGroup`, isolation = the store key) + **`ScimService`** (uniqueness, server id + `meta`, the IdP **deactivation** PATCH, `attr eq "value"` filter, PII-free `scim.*` events).  `/scim/v2/{Users,Groups}` + discovery endpoints authenticate a **per-tenant SCIM bearer token** (`cfg.scim.tokens`, constant-time compare — *not* a user JWT), return SCIM-shaped errors (RFC 7644), and 404 when disabled (checked before auth, so the surface is hidden).  **No new governed SPI call** → the PolicyEngine coverage linter passes with no allowlist entry
+- New core types **`FederatedIdentity` / `SsoProtocol` / `ScimUser` / `ScimGroup`** (+ nested) + **`SsoError`** (401) / **`ScimError`** (400) / **`ScimNotFoundError`** (404) / **`ScimConflictError`** (409); wire types `ScimListResponse` / `ScimPatchOp` / `ScimErrorBody` / `SsoStatusResponse` (`dist/schemas` + `dist/openapi` regenerated); `cfg.sso` / `cfg.scim` / `tenants[].sso` (`dist/rag.schema`); PII-free `sso.*` / `scim.*` events (subject **hashed**, never email / userName); **`ragctl sso`** (list + in-process OIDC demo) + **`ragctl scim`** (in-process provisioning demo)
+- **Scope:** verification + provisioning at the boundary.  **Deferred:** remote JWKS discovery + rotation (configured static keys only), SAML SP-initiated redirect + metadata, SCIM bulk / `/Me` / ETag, directory-backed deprovisioning at authenticate-time, the admin-console SSO/SCIM card.  ~80 new tests (jwt / oidc / saml / federated-auth / scim-service unit; `ScimStore` contract suite; gateway SCIM CRUD + token guard + isolation + disabled-404; SSO status + JWT-through-middleware; `ragctl`).  All gates green (ruff, mypy --strict, RAG001, schema/openapi/config-drift, policy-coverage, log-schema/event-registry/PII).  [ADR-0040](docs/adr/ADR-0040-sso-scim.md), [reference/sso.md](docs/reference/sso.md), [architecture/sso-scim.md](docs/architecture/sso-scim.md)
+
 ---
 
 ## Phase 7 — Pilot, Harden, GA (Weeks 34–40) ⏳

apps/gateway/pyproject.toml

@@ -32,6 +32,7 @@ dependencies = [
     "rag-drift",
     "rag-agent",
     "rag-webhooks",
+    "rag-sso",
     "fastapi>=0.115",
     "uvicorn[standard]>=0.30",
     "python-multipart>=0.0.9",
@@ -78,6 +79,7 @@ rag-feedback = { workspace = true }
 rag-drift = { workspace = true }
 rag-agent = { workspace = true }
 rag-webhooks = { workspace = true }
+rag-sso = { workspace = true }
 
 [tool.hatch.build.targets.wheel]
 packages = ["src/rag_gateway"]

apps/gateway/src/rag_gateway/app.py

@@ -125,6 +125,7 @@
     make_query_router,
 )
 from rag_gateway.quota_guard import enforce_storage_quota
+from rag_gateway.scim import make_scim_router
 from rag_gateway.status import (
     InMemoryConnectorStatusStore,
     install_cors,
@@ -377,6 +378,10 @@ def build_app(
     audit_enabled: bool = True,
     audit_export_secret: str = "",
     key_manager: Any | None = None,
+    scim_store: Any | None = None,
+    scim_enabled: bool = False,
+    scim_tokens: dict[str, str] | None = None,
+    sso_enabled: bool = False,
     enable_cors: bool = True,
     default_tenant_id: TenantId | None = None,
 ) -> FastAPI:
@@ -429,6 +434,13 @@ def build_app(
                     "with an SSE log stream and a WebSocket health/metrics push."
                 ),
             },
+            {
+                "name": "scim",
+                "description": (
+                    "SCIM 2.0 directory provisioning — IdP-driven user / group "
+                    "create / update / deactivate (per-tenant bearer token)."
+                ),
+            },
         ],
     )
 
@@ -528,6 +540,19 @@ def build_app(
     )
     auth_backend = auth or NoopAuth()
     app.state.auth = auth_backend
+    # SSO / SCIM (Step 6.8).  ``sso_enabled`` is a diagnostic flag surfaced on
+    # ``GET /v1/status/sso``; the actual federation is whatever ``auth`` backend is
+    # wired (a ``FederatedAuth`` when build_app_from_config builds one from
+    # cfg.sso).  SCIM 2.0 provisioning (``/scim/v2``) runs over an in-memory
+    # NoopScimStore unless a durable store is injected; it is disabled (404s) until
+    # ``scim_enabled`` + per-tenant bearer ``scim_tokens`` are supplied from cfg.scim.
+    from rag_core.spi.noop import NoopScimStore
+    from rag_sso import ScimService
+
+    app.state.sso_enabled = sso_enabled
+    app.state.scim_enabled = scim_enabled
+    app.state.scim_tokens = dict(scim_tokens or {})
+    app.state.scim_service = ScimService(scim_store or NoopScimStore())
     # PolicyEngine is optional in the gateway — when None, the
     # egress check in answer generation skips.  Production wiring
     # always installs one (NoopPolicyEngine in dev; real PDP in 6.x).
@@ -821,6 +846,7 @@ async def ingest_document(
     app.include_router(make_feedback_router())
     app.include_router(make_status_router())
     app.include_router(make_audit_router())
+    app.include_router(make_scim_router())
 
     return app
 

apps/gateway/src/rag_gateway/query.py

@@ -40,6 +40,9 @@
     RagError,
     RateLimitError,
     RetrievalError,
+    ScimConflictError,
+    ScimError,
+    ScimNotFoundError,
 )
 from rag_core.gateway_types import (
     Answer,
@@ -107,6 +110,15 @@ def _http_status_for(exc: BaseException) -> int:
     if isinstance(exc, AuditNotFoundError):
         # Audit read API disabled / no store wired (Step 6.6).
         return 404
+    if isinstance(exc, ScimNotFoundError):
+        # SCIM resource / surface not found (Step 6.8).
+        return 404
+    if isinstance(exc, ScimConflictError):
+        # SCIM uniqueness violation (duplicate userName / displayName) (Step 6.8).
+        return 409
+    if isinstance(exc, ScimError):
+        # Other SCIM provisioning failure (e.g. unsupported filter) (Step 6.8).
+        return 400
     if isinstance(exc, RetrievalError):
         # Retrieval errors are bad-gateway because they indicate a
         # downstream backend failure, not bad input.