diff --git a/TRACKER.md b/TRACKER.md index 2624da1..f9a6ff9 100644 --- a/TRACKER.md +++ b/TRACKER.md @@ -13,13 +13,14 @@ | | | |---|---| -| **Last updated** | 2026-06-07 | -| **Current phase** | **Phase 5 — Eval & Observability ✅ complete (7 / 7 steps)** → starting Phase 6 | -| **Overall** | **64 / 84 steps** — Phases 0–5 complete | -| **Next action** | **Step 6.1 — Logical multi-tenancy** (Phase 6 — Governance & Tenancy): namespace isolation; per-tenant config, quotas, ACLs in `rag.yaml`. First step of the governance phase. | +| **Last updated** | 2026-06-08 | +| **Current phase** | Phase 6 — Governance & Tenancy (**1 / 10 steps**) | +| **Overall** | **65 / 84 steps** — Phases 0–5 complete | +| **Next action** | **Step 6.2 — Physical tenancy (dedicated index)**: dedicated vector index per tenant; cross-tenant probe gate. Builds on the 6.1 namespace primitive. | **Recently shipped** +- **6.1** ✅ Logical multi-tenancy — `TenantResolver` resolves per-tenant `rag.yaml` config (namespace / PII / ACL labels) → `TenantSettings` applied at the gateway boundary; `RequestContext.namespace`; `GET /v1/status/tenant`; `ragctl tenant` — [#148](https://github.com/officialCodeWork/AgentContextOS/pull/148) - **5.7d** ✅ Experiments console card + Phase-5 close-out — admin Live-Status A/B card over `GET /v1/status/experiments` (lift + CI per experiment) — [#147](https://github.com/officialCodeWork/AgentContextOS/pull/147) - **5.7c** ✅ A/B routing — `ABRouter` deterministically *serves* the candidate to a fraction of users (variant-partitioned cache, `ExperimentAssignment` response tag) — [#146](https://github.com/officialCodeWork/AgentContextOS/pull/146) - **5.7b** ✅ Shadow mode — observe-only candidate fan-out (`ShadowRunner`, background task) feeding the A/B tracker — [#145](https://github.com/officialCodeWork/AgentContextOS/pull/145) @@ -54,9 +55,9 @@ | 3 | Gateway & Agent Runtime | 11 | **11** | 0 | | 4 | Reliability | 6 | **6** | 0 | | 5 | Eval & Observability | 7 | **7** | 0 | -| 6 | Governance & Tenancy | 10 | 0 | 10 | +| 6 | Governance & Tenancy | 10 | **1** | 9 | | 7 | Pilot, Harden, GA | 10 | 0 | 10 | -| **Total** | | **84** | **64** | **20** | +| **Total** | | **84** | **65** | **19** | --- @@ -639,11 +640,11 @@ - **Phase-5 close-out:** Step 5.7 ✅ → **Phase 5 complete (7 / 7)**; deferred items remain documented (per-tenant drift / per-dimension embedding PSI; feedback/breaker/quota Grafana export + Loki-events dashboard; gRPC proto mirror of `corpus_decision` + `experiment`; sequential / multi-metric experiments) - [reference/experiments.md](docs/reference/experiments.md), [reference/admin-ui.md](docs/reference/admin-ui.md) -## Phase 6 — Governance & Tenancy (Weeks 28–34) ⏳ +## Phase 6 — Governance & Tenancy (Weeks 28–34) 🚧 (1 / 10) | Step | Title | Status | Planned deliverables | |------|-------|:------:|----------------------| -| 6.1 | Logical multi-tenancy | ⏳ | Namespace isolation; per-tenant config, quotas, ACLs in `rag.yaml` | +| 6.1 | Logical multi-tenancy | ✅ | [#148](https://github.com/officialCodeWork/AgentContextOS/pull/148) — `TenantResolver` → `TenantSettings` (namespace / PII / ACL labels) applied at the boundary; `RequestContext.namespace`; `GET /v1/status/tenant`; `ragctl tenant` | | 6.2 | Physical tenancy (dedicated index) | ⏳ | Dedicated vector index per tenant; cross-tenant probe gate | | 6.3 | ACL push-down at retrieval | ⏳ | ACL filter injected into every vector / BM25 / graph query; `acl.egress_denied` event | | 6.4 | ACL egress verifier | ⏳ | Post-retrieval re-check; defense-in-depth; zero-ACL-violation-rate gate | @@ -654,6 +655,16 @@ | 6.9 | Air-gapped install bundle | ⏳ | Signed tarball with all images + Helm chart; offline bootstrap; cosign verification | | 6.10 | Compliance posture | ⏳ | SOC 2 Type II control mapping; GDPR data-residency config; data-retention policies | +### 6.1 — Logical multi-tenancy ✅ [#148](https://github.com/officialCodeWork/AgentContextOS/pull/148) + +- Makes per-tenant `rag.yaml` config actually drive requests (it was a passive registry before; only quotas were per-tenant). New `TenantResolver` (`rag_config.tenancy`) maps a tenant id → frozen **`TenantSettings`** (`rag_core.types`: namespace / pii_policy / acl_labels), merging the `tenants:` entry with safe defaults +- **Resolved once, at the boundary:** the request-context middleware (`build_gateway_context`) applies the resolved settings — `namespace` + `pii_policy` onto the `RequestContext`, the tenant's `acl_labels` **unioned** into the principal's; the resolved view is also exposed on `request.state.gateway.tenant_settings` +- **Unknown tenants resolve to safe defaults** (namespace = the id, default PII `redact`, no labels) — isolated, never privileged +- New **`RequestContext.namespace`** (defaults to `tenant_id` via a before-validator, so every pre-6.1 call site is unchanged); it's a **backend-partition** primitive — Pinecone now partitions on `ctx.namespace` — **not** a chunk field, so `filter_pushdown` is unchanged (chunks carry `tenant_id`) +- `TenantConfig` gains `namespace` + `acl_labels` (→ `rag.schema`); `TenantSettings` joins `dist/schemas` + `RequestContext` regenerated; **`GET /v1/status/tenant`** (→ `dist/openapi`) surfaces the resolved view; `ragctl tenant list` / `resolve ` print it; **inert by default** in `build_app`, built from `cfg.tenants` by `build_app_from_config` +- **Scope:** resolution + threading only — ACL push-down enforcement is 6.3, PII egress is 6.5, physical tenancy is 6.2; ~21 new tests (resolver, config, gateway, cross-tenant red-team, CLI); all gates green (ruff, mypy --strict 293 files, RAG001, schema/openapi-drift, proto-compat, policy-coverage) +- [ADR-0033](docs/adr/ADR-0033-logical-multi-tenancy.md), [reference/tenancy.md](docs/reference/tenancy.md), [architecture/multi-tenancy.md](docs/architecture/multi-tenancy.md) + --- ## Phase 7 — Pilot, Harden, GA (Weeks 34–40) ⏳ @@ -805,6 +816,7 @@ Complete log of every PR. Routine Dependabot bumps are grouped; everything else | [#145](https://github.com/officialCodeWork/AgentContextOS/pull/145) | 2026-06-05 | feat(experiments): shadow mode — observe-only candidate fan-out (Step 5.7b) | | [#146](https://github.com/officialCodeWork/AgentContextOS/pull/146) | 2026-06-07 | feat(experiments): A/B routing — serve the candidate to a fraction of users (Step 5.7c) | | [#147](https://github.com/officialCodeWork/AgentContextOS/pull/147) | 2026-06-07 | feat(admin-ui): A/B experiments console card + Phase-5 close-out (Step 5.7d) | +| [#148](https://github.com/officialCodeWork/AgentContextOS/pull/148) | 2026-06-08 | feat(tenancy): logical multi-tenancy — per-tenant config resolution (Step 6.1) | | #78–#80, #116–#118 | Open | Dependabot bumps — awaiting merge | | #81 | Closed | Dependabot bump — superseded | diff --git a/apps/gateway/src/rag_gateway/app.py b/apps/gateway/src/rag_gateway/app.py index 522fa95..5b6b559 100644 --- a/apps/gateway/src/rag_gateway/app.py +++ b/apps/gateway/src/rag_gateway/app.py @@ -367,6 +367,7 @@ def build_app( experiment_tracker: Any | None = None, shadow_runner: Any | None = None, ab_router: Any | None = None, + tenant_resolver: Any | None = None, enable_cors: bool = True, default_tenant_id: TenantId | None = None, ) -> FastAPI: @@ -611,10 +612,16 @@ def build_app( # OpenAI-compatible endpoints) can synthesise an anonymous principal # in dev mode and require real auth when this is None in production. app.state.default_tenant_id = effective_default_tenant + # Per-tenant settings resolver (Step 6.1) — makes per-tenant rag.yaml config + # (namespace / PII policy / ACL labels) drive each request. ``None`` (the + # plain ``build_app`` default) keeps the pre-6.1 behaviour; the config-driven + # wiring builds one from ``cfg.tenants``. + app.state.tenant_resolver = tenant_resolver install_request_context_middleware( app, auth=auth_backend, default_tenant_id=effective_default_tenant, + tenant_resolver=tenant_resolver, ) # Metrics middleware is added after request-context (so it wraps it and times # the full handler). CORS is added last so it is the outermost layer and can diff --git a/apps/gateway/src/rag_gateway/middleware.py b/apps/gateway/src/rag_gateway/middleware.py index b088e32..3b625b9 100644 --- a/apps/gateway/src/rag_gateway/middleware.py +++ b/apps/gateway/src/rag_gateway/middleware.py @@ -35,6 +35,7 @@ import re import uuid from collections.abc import Awaitable, Callable +from typing import TYPE_CHECKING from fastapi import Request, Response from rag_core import get_logger @@ -47,9 +48,13 @@ RequestContext, RequestId, TenantId, + TenantSettings, TraceContext, ) +if TYPE_CHECKING: + from rag_config.tenancy import TenantResolver + _log = get_logger(__name__) # W3C Trace Context — https://www.w3.org/TR/trace-context/ @@ -71,7 +76,7 @@ class GatewayContext: Route handlers decide what to do per-route. """ - __slots__ = ("ctx", "auth_error", "request_id", "trace") + __slots__ = ("ctx", "auth_error", "request_id", "trace", "tenant_settings") def __init__( self, @@ -80,11 +85,15 @@ def __init__( auth_error: AuthError | None, request_id: RequestId, trace: TraceContext, + tenant_settings: TenantSettings | None = None, ) -> None: self.ctx = ctx self.auth_error = auth_error self.request_id = request_id self.trace = trace + # The resolved per-tenant view (Step 6.1) when a resolver is wired; the + # namespace / pii_policy / unioned acl_labels are already on ``ctx``. + self.tenant_settings = tenant_settings async def build_gateway_context( @@ -92,6 +101,7 @@ async def build_gateway_context( *, auth: Auth, default_tenant_id: TenantId | None = None, + tenant_resolver: TenantResolver | None = None, ) -> GatewayContext: """Read headers, resolve principal via ``auth``, return a GatewayContext. @@ -103,6 +113,13 @@ async def build_gateway_context( ``"gateway-anon"`` when the caller didn't supply a tenant; in production this should be ``None`` so a missing tenant header becomes an explicit auth failure. + + ``tenant_resolver`` (Step 6.1) makes per-tenant ``rag.yaml`` config drive the + request: when wired, the tenant's resolved namespace + PII policy land on the + :class:`RequestContext` and the tenant's ACL labels are unioned into the + principal's. ``None`` (the plain ``build_app`` default) leaves the pre-6.1 + behaviour — the namespace falls back to the tenant id and the PII policy to the + default. """ headers = request.headers @@ -143,12 +160,27 @@ async def build_gateway_context( tenant_id=tenant_id, ) + # Per-tenant settings (Step 6.1) — resolve the tenant's namespace / PII policy + # / ACL labels from rag.yaml and apply them once, here at the boundary. + tenant_settings: TenantSettings | None = None ctx: RequestContext | None = None if principal is not None and tenant_id is not None: + namespace = str(tenant_id) + pii_policy = None + if tenant_resolver is not None: + tenant_settings = tenant_resolver.resolve(str(tenant_id)) + namespace = tenant_settings.namespace + pii_policy = tenant_settings.pii_policy + if tenant_settings.acl_labels: + principal = principal.model_copy( + update={"acl_labels": principal.acl_labels | tenant_settings.acl_labels} + ) ctx = RequestContext( request_id=request_id, tenant_id=tenant_id, principal=principal, + namespace=namespace, + **({"pii_policy": pii_policy} if pii_policy is not None else {}), trace=trace, ) @@ -157,6 +189,7 @@ async def build_gateway_context( auth_error=auth_error, request_id=request_id, trace=trace, + tenant_settings=tenant_settings, ) @@ -184,7 +217,11 @@ def _parse_traceparent(raw: str | None) -> TraceContext: def install_request_context_middleware( - app: Any, *, auth: Auth, default_tenant_id: TenantId | None = None + app: Any, + *, + auth: Auth, + default_tenant_id: TenantId | None = None, + tenant_resolver: TenantResolver | None = None, ) -> None: # noqa: F821 """Install the per-request middleware on a FastAPI app. @@ -196,6 +233,10 @@ def install_request_context_middleware( The middleware also stamps the ``X-Request-Id`` header on every response so callers can correlate logs without re-reading the request body. + + ``tenant_resolver`` (Step 6.1) is forwarded to + :func:`build_gateway_context` so per-tenant ``rag.yaml`` config drives the + request; ``None`` keeps the pre-6.1 behaviour. """ @app.middleware("http") # type: ignore[untyped-decorator] @@ -207,6 +248,7 @@ async def _request_context_middleware( request, auth=auth, default_tenant_id=default_tenant_id, + tenant_resolver=tenant_resolver, ) request.state.gateway = gateway response = await call_next(request) diff --git a/apps/gateway/src/rag_gateway/status.py b/apps/gateway/src/rag_gateway/status.py index fbf57da..62ce603 100644 --- a/apps/gateway/src/rag_gateway/status.py +++ b/apps/gateway/src/rag_gateway/status.py @@ -58,6 +58,7 @@ from fastapi.responses import StreamingResponse from pydantic import BaseModel, Field from rag_config.eval import analyze_ab_experiment +from rag_config.tenancy import TenantResolver from rag_core.eval import ABAnalysisResult from rag_core.types import ( BreakerSnapshot, @@ -313,6 +314,26 @@ class CostStatusResponse(BaseModel): baseline_cost_micros: float = 0.0 +class TenantStatusResponse(BaseModel): + """One tenant's resolved logical-tenancy settings (Step 6.1). + + The effective per-tenant view the gateway applies to each request, resolved by + the :class:`~rag_config.tenancy.TenantResolver` from ``cfg.tenants``. ``known`` + is whether the tenant is declared in ``rag.yaml`` — an unknown tenant resolves + to safe defaults (namespace = the tenant id, default PII action, no labels), so + it is isolated rather than privileged. Per-tenant like quotas / cost: the + tenant comes from the ``tenant_id`` query param or the ``X-Tenant-Id`` header. + """ + + model_config = {"frozen": True} + + tenant_id: str + known: bool = False + namespace: str = "" + pii_action: str = "redact" + acl_labels: list[str] = Field(default_factory=list) + + class ExperimentsStatusResponse(BaseModel): """A/B experiment comparisons (Step 5.7). @@ -847,6 +868,38 @@ async def status_experiments(request: Request) -> ExperimentsStatusResponse: ] return ExperimentsStatusResponse(experiments=results, total=len(results)) + @router.get("/v1/status/tenant", response_model=TenantStatusResponse) + async def status_tenant(request: Request, tenant_id: str | None = None) -> TenantStatusResponse: + """Resolved per-tenant logical-tenancy settings (Step 6.1). + + Surfaces what the ``TenantResolver`` resolves for a tenant — its + namespace, PII action, and ACL labels — so an operator can see the + effective config a tenant's requests run under. Tenant resolves from the + ``tenant_id`` query param, else the ``X-Tenant-Id`` header, else the + gateway default. Reports safe defaults (``known=false``) when the tenant + is absent from ``rag.yaml`` or no resolver is wired. + """ + resolver = getattr(request.app.state, "tenant_resolver", None) + tid = str( + tenant_id + or request.headers.get("X-Tenant-Id") + or getattr(request.app.state, "default_tenant_id", None) + or "default" + ) + if resolver is not None: + settings = resolver.resolve(tid) + known = resolver.known(tid) + else: + settings = TenantResolver.default_settings(tid) + known = False + return TenantStatusResponse( + tenant_id=tid, + known=known, + namespace=settings.namespace, + pii_action=settings.pii_policy.action.value, + acl_labels=sorted(settings.acl_labels), + ) + @router.websocket("/v1/status/ws") async def status_ws(websocket: WebSocket) -> None: """Push health + metrics snapshots until the client disconnects. diff --git a/apps/gateway/src/rag_gateway/wiring.py b/apps/gateway/src/rag_gateway/wiring.py index a34a216..ff4cbe1 100644 --- a/apps/gateway/src/rag_gateway/wiring.py +++ b/apps/gateway/src/rag_gateway/wiring.py @@ -37,6 +37,7 @@ CorpusStoreProvider, RagConfig, TenantQuota, + TenantResolver, ) from rag_config import ( CorpusRoutingStrategy as CfgStrategy, @@ -687,6 +688,13 @@ def build_app_from_config(cfg: RagConfig, **overrides: Any) -> FastAPI: cfg, overrides.get("experiment_tracker") ) + # Logical multi-tenancy (Step 6.1) — resolve per-tenant rag.yaml config + # (namespace / PII policy / ACL labels) at the request boundary. Always built + # from ``cfg.tenants`` (an unknown tenant resolves to safe defaults) unless a + # caller injects its own. + if "tenant_resolver" not in overrides: + overrides["tenant_resolver"] = TenantResolver.from_config(cfg) + # Phase-5 quality/cost signals as Prometheus metrics (Step 5.6e) — expose the # drift report + per-tenant cost verdicts as OTel observable gauges on the # same pipeline that carries ``rag.spi.*`` so Grafana can graph them. The diff --git a/apps/gateway/tests/test_tenancy.py b/apps/gateway/tests/test_tenancy.py new file mode 100644 index 0000000..ec8130b --- /dev/null +++ b/apps/gateway/tests/test_tenancy.py @@ -0,0 +1,136 @@ +"""Tests for logical multi-tenancy — per-tenant settings resolution (Step 6.1). + +Covers the gateway boundary applying a tenant's resolved namespace / PII policy / +ACL labels to the :class:`RequestContext` (``build_gateway_context``), the +``GET /v1/status/tenant`` surface, and that the plain ``build_app`` stays inert. +""" + +from __future__ import annotations + +from typing import Any + +from fastapi.testclient import TestClient +from rag_config import RagConfig, TenantConfig, TenantResolver +from rag_config.schema import PIIPolicy +from rag_core.types import PiiAction +from rag_gateway import build_app +from rag_gateway.middleware import build_gateway_context +from rag_gateway.wiring import build_app_from_config +from starlette.requests import Request + + +class _StubAuth: + """Never invoked on the dev (X-Principal-Id) path; satisfies the Auth slot.""" + + async def authenticate(self, token: str, tenant_id: Any) -> Any: # pragma: no cover + raise AssertionError("auth.authenticate should not be called on the dev path") + + +def _request(headers: dict[str, str]) -> Request: + raw = [(k.lower().encode(), v.encode()) for k, v in headers.items()] + return Request({"type": "http", "method": "POST", "path": "/v1/query", "headers": raw}) + + +def _resolver() -> TenantResolver: + return TenantResolver.from_config( + RagConfig( + tenants=[ + TenantConfig( + id="acme", + name="Acme", + pii_policy=PIIPolicy.BLOCK, + namespace="acme-prod", + acl_labels=["region:eu"], + ), + TenantConfig(id="beta", name="Beta", pii_policy=PIIPolicy.ALLOW), + ] + ) + ) + + +# --------------------------------------------------------------------------- +# build_gateway_context applies resolved settings to the RequestContext +# --------------------------------------------------------------------------- +async def test_context_carries_resolved_namespace_pii_and_acls() -> None: + gw = await build_gateway_context( + _request({"X-Tenant-Id": "acme", "X-Principal-Id": "p1"}), + auth=_StubAuth(), # type: ignore[arg-type] + tenant_resolver=_resolver(), + ) + assert gw.ctx is not None + assert gw.ctx.namespace == "acme-prod" + assert gw.ctx.pii_policy.action is PiiAction.block + # The tenant's ACL labels are unioned into the principal's. + assert gw.ctx.principal.acl_labels == frozenset({"region:eu"}) + assert gw.tenant_settings is not None and gw.tenant_settings.namespace == "acme-prod" + + +async def test_context_without_resolver_defaults_namespace_to_tenant() -> None: + gw = await build_gateway_context( + _request({"X-Tenant-Id": "acme", "X-Principal-Id": "p1"}), + auth=_StubAuth(), # type: ignore[arg-type] + ) + assert gw.ctx is not None + assert gw.ctx.namespace == "acme" # falls back to the tenant id + assert gw.ctx.pii_policy.action is PiiAction.redact # the default policy + assert gw.tenant_settings is None + + +async def test_unknown_tenant_gets_safe_defaults() -> None: + gw = await build_gateway_context( + _request({"X-Tenant-Id": "ghost", "X-Principal-Id": "p1"}), + auth=_StubAuth(), # type: ignore[arg-type] + tenant_resolver=_resolver(), + ) + assert gw.ctx is not None + assert gw.ctx.namespace == "ghost" + assert gw.ctx.pii_policy.action is PiiAction.redact + assert gw.ctx.principal.acl_labels == frozenset() + + +# --------------------------------------------------------------------------- +# GET /v1/status/tenant +# --------------------------------------------------------------------------- +def test_status_tenant_known() -> None: + client = TestClient(build_app_from_config(RagConfig.model_validate(_resolver_cfg()))) + body = client.get("/v1/status/tenant", headers={"X-Tenant-Id": "acme"}).json() + assert body == { + "tenant_id": "acme", + "known": True, + "namespace": "acme-prod", + "pii_action": "block", + "acl_labels": ["region:eu"], + } + + +def test_status_tenant_unknown_is_defaulted() -> None: + client = TestClient(build_app_from_config(RagConfig.model_validate(_resolver_cfg()))) + body = client.get("/v1/status/tenant", headers={"X-Tenant-Id": "ghost"}).json() + assert body["known"] is False + assert body["namespace"] == "ghost" + assert body["pii_action"] == "redact" + + +def test_status_tenant_inert_without_resolver() -> None: + # Plain build_app wires no resolver → defaults, known=False. + body = ( + TestClient(build_app()).get("/v1/status/tenant", headers={"X-Tenant-Id": "whoever"}).json() + ) + assert body["known"] is False + assert body["namespace"] == "whoever" + + +def _resolver_cfg() -> dict[str, Any]: + return { + "version": "1", + "tenants": [ + { + "id": "acme", + "name": "Acme", + "pii_policy": "block", + "namespace": "acme-prod", + "acl_labels": ["region:eu"], + }, + {"id": "beta", "name": "Beta", "pii_policy": "allow"}, + ], + } diff --git a/dist/openapi.json b/dist/openapi.json index 37af8da..39563b5 100644 --- a/dist/openapi.json +++ b/dist/openapi.json @@ -3664,6 +3664,42 @@ "title": "SubscriptionList", "type": "object" }, + "TenantStatusResponse": { + "description": "One tenant's resolved logical-tenancy settings (Step 6.1).\n\nThe effective per-tenant view the gateway applies to each request, resolved by\nthe :class:`~rag_config.tenancy.TenantResolver` from ``cfg.tenants``. ``known``\nis whether the tenant is declared in ``rag.yaml`` — an unknown tenant resolves\nto safe defaults (namespace = the tenant id, default PII action, no labels), so\nit is isolated rather than privileged. Per-tenant like quotas / cost: the\ntenant comes from the ``tenant_id`` query param or the ``X-Tenant-Id`` header.", + "properties": { + "acl_labels": { + "items": { + "type": "string" + }, + "title": "Acl Labels", + "type": "array" + }, + "known": { + "default": false, + "title": "Known", + "type": "boolean" + }, + "namespace": { + "default": "", + "title": "Namespace", + "type": "string" + }, + "pii_action": { + "default": "redact", + "title": "Pii Action", + "type": "string" + }, + "tenant_id": { + "title": "Tenant Id", + "type": "string" + } + }, + "required": [ + "tenant_id" + ], + "title": "TenantStatusResponse", + "type": "object" + }, "TraceContext": { "description": "OTel-compatible trace context propagated through every pipeline step.", "properties": { @@ -5307,6 +5343,56 @@ ] } }, + "/v1/status/tenant": { + "get": { + "description": "Resolved per-tenant logical-tenancy settings (Step 6.1).\n\nSurfaces what the ``TenantResolver`` resolves for a tenant — its\nnamespace, PII action, and ACL labels — so an operator can see the\neffective config a tenant's requests run under. Tenant resolves from the\n``tenant_id`` query param, else the ``X-Tenant-Id`` header, else the\ngateway default. Reports safe defaults (``known=false``) when the tenant\nis absent from ``rag.yaml`` or no resolver is wired.", + "operationId": "status_tenant_v1_status_tenant_get", + "parameters": [ + { + "in": "query", + "name": "tenant_id", + "required": false, + "schema": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Tenant Id" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/TenantStatusResponse" + } + } + }, + "description": "Successful Response" + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + }, + "description": "Validation Error" + } + }, + "summary": "Status Tenant", + "tags": [ + "status" + ] + } + }, "/v1/webhooks/subscriptions": { "get": { "operationId": "list_subscriptions_v1_webhooks_subscriptions_get", diff --git a/dist/openapi.yaml b/dist/openapi.yaml index a154299..94eec7f 100644 --- a/dist/openapi.yaml +++ b/dist/openapi.yaml @@ -3232,6 +3232,48 @@ components: type: integer title: SubscriptionList type: object + TenantStatusResponse: + description: 'One tenant''s resolved logical-tenancy settings (Step 6.1). + + + The effective per-tenant view the gateway applies to each request, resolved + by + + the :class:`~rag_config.tenancy.TenantResolver` from ``cfg.tenants``. ``known`` + + is whether the tenant is declared in ``rag.yaml`` — an unknown tenant resolves + + to safe defaults (namespace = the tenant id, default PII action, no labels), + so + + it is isolated rather than privileged. Per-tenant like quotas / cost: the + + tenant comes from the ``tenant_id`` query param or the ``X-Tenant-Id`` header.' + properties: + acl_labels: + items: + type: string + title: Acl Labels + type: array + known: + default: false + title: Known + type: boolean + namespace: + default: '' + title: Namespace + type: string + pii_action: + default: redact + title: Pii Action + type: string + tenant_id: + title: Tenant Id + type: string + required: + - tenant_id + title: TenantStatusResponse + type: object TraceContext: description: OTel-compatible trace context propagated through every pipeline step. @@ -4410,6 +4452,48 @@ paths: summary: Reset Quota tags: - status + /v1/status/tenant: + get: + description: 'Resolved per-tenant logical-tenancy settings (Step 6.1). + + + Surfaces what the ``TenantResolver`` resolves for a tenant — its + + namespace, PII action, and ACL labels — so an operator can see the + + effective config a tenant''s requests run under. Tenant resolves from the + + ``tenant_id`` query param, else the ``X-Tenant-Id`` header, else the + + gateway default. Reports safe defaults (``known=false``) when the tenant + + is absent from ``rag.yaml`` or no resolver is wired.' + operationId: status_tenant_v1_status_tenant_get + parameters: + - in: query + name: tenant_id + required: false + schema: + anyOf: + - type: string + - type: 'null' + title: Tenant Id + responses: + '200': + content: + application/json: + schema: + $ref: '#/components/schemas/TenantStatusResponse' + description: Successful Response + '422': + content: + application/json: + schema: + $ref: '#/components/schemas/HTTPValidationError' + description: Validation Error + summary: Status Tenant + tags: + - status /v1/webhooks/subscriptions: get: operationId: list_subscriptions_v1_webhooks_subscriptions_get diff --git a/dist/rag.schema.json b/dist/rag.schema.json index 235aa14..e40ad74 100644 --- a/dist/rag.schema.json +++ b/dist/rag.schema.json @@ -1280,6 +1280,7 @@ }, "TenantConfig": { "additionalProperties": false, + "description": "Per-tenant configuration entry in ``rag.yaml`` (Step 6.1 logical tenancy).\n\nRegistered under ``tenants:`` and resolved at the gateway boundary by the\n``TenantResolver`` into a ``rag_core.types.TenantSettings`` that drives the\nrequest: ``namespace`` is the logical-isolation key (defaults to ``id``);\n``pii_policy`` becomes the request's PII action; ``acl_labels`` are unioned\ninto the principal's labels; ``quota`` is wired into the Step 4.5 enforcer.", "properties": { "id": { "minLength": 1, @@ -1304,6 +1305,26 @@ }, "title": "Corpus Ids", "type": "array" + }, + "namespace": { + "anyOf": [ + { + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Namespace" + }, + "acl_labels": { + "items": { + "type": "string" + }, + "title": "Acl Labels", + "type": "array" } }, "required": [ diff --git a/dist/rag.schema.yaml b/dist/rag.schema.yaml index 83ad8c9..989a042 100644 --- a/dist/rag.schema.yaml +++ b/dist/rag.schema.yaml @@ -1187,6 +1187,19 @@ $defs: type: string TenantConfig: additionalProperties: false + description: 'Per-tenant configuration entry in ``rag.yaml`` (Step 6.1 logical + tenancy). + + + Registered under ``tenants:`` and resolved at the gateway boundary by the + + ``TenantResolver`` into a ``rag_core.types.TenantSettings`` that drives the + + request: ``namespace`` is the logical-isolation key (defaults to ``id``); + + ``pii_policy`` becomes the request''s PII action; ``acl_labels`` are unioned + + into the principal''s labels; ``quota`` is wired into the Step 4.5 enforcer.' properties: id: minLength: 1 @@ -1206,6 +1219,18 @@ $defs: type: string title: Corpus Ids type: array + namespace: + anyOf: + - minLength: 1 + type: string + - type: 'null' + default: null + title: Namespace + acl_labels: + items: + type: string + title: Acl Labels + type: array required: - id - name diff --git a/dist/schemas/RequestContext.json b/dist/schemas/RequestContext.json index a8213d4..0f0292b 100644 --- a/dist/schemas/RequestContext.json +++ b/dist/schemas/RequestContext.json @@ -201,6 +201,11 @@ "principal": { "$ref": "#/$defs/Principal" }, + "namespace": { + "default": "", + "title": "Namespace", + "type": "string" + }, "pii_policy": { "$ref": "#/$defs/PiiPolicy" }, diff --git a/dist/schemas/TenantSettings.json b/dist/schemas/TenantSettings.json new file mode 100644 index 0000000..a92ebea --- /dev/null +++ b/dist/schemas/TenantSettings.json @@ -0,0 +1,69 @@ +{ + "$defs": { + "PiiAction": { + "description": "Per-tenant PII handling policy.", + "enum": [ + "block", + "redact", + "mask", + "encrypt", + "tag_only", + "allow" + ], + "title": "PiiAction", + "type": "string" + }, + "PiiPolicy": { + "description": "Per-tenant PII enforcement contract attached to a `RequestContext`.", + "properties": { + "action": { + "$ref": "#/$defs/PiiAction", + "default": "redact" + }, + "entities": { + "items": { + "type": "string" + }, + "title": "Entities", + "type": "array", + "uniqueItems": true + }, + "min_score": { + "default": 0.5, + "title": "Min Score", + "type": "number" + } + }, + "title": "PiiPolicy", + "type": "object" + } + }, + "description": "Resolved per-tenant view of config \u2014 the unit of logical tenancy (Step 6.1).\n\nProduced by ``rag_config.tenancy.TenantResolver`` from the ``tenants`` block of\n``rag.yaml``, merged with the global defaults, and applied **once** at the\ngateway boundary: ``namespace`` + ``pii_policy`` land on the\n:class:`RequestContext`, and ``acl_labels`` are unioned into the principal's\nlabels. An unknown tenant resolves to safe defaults (``namespace`` = the\ntenant id, the default PII policy, no extra labels), so a tenant that isn't in\nthe config is isolated, never privileged.\n\nIsolation primitives only \u2014 the resolved behavioural overrides (retrieval,\nguard) are applied by their own components, not carried here. ACL *push-down*\nenforcement is Step 6.3; PII *egress* enforcement is Step 6.5.", + "properties": { + "tenant_id": { + "title": "Tenant Id", + "type": "string" + }, + "namespace": { + "title": "Namespace", + "type": "string" + }, + "pii_policy": { + "$ref": "#/$defs/PiiPolicy" + }, + "acl_labels": { + "items": { + "type": "string" + }, + "title": "Acl Labels", + "type": "array", + "uniqueItems": true + } + }, + "required": [ + "tenant_id", + "namespace" + ], + "title": "TenantSettings", + "type": "object" +} diff --git a/docs/README.md b/docs/README.md index b7a56c3..0bc34c4 100644 --- a/docs/README.md +++ b/docs/README.md @@ -6,7 +6,8 @@ |------|-------------| | [RAG-Platform-HLD.md](architecture/RAG-Platform-HLD.md) | High-Level Design: problem statement, layered architecture, pluggable backends, `rag.yaml` contract, deployment topologies, KPIs, risks, glossary | | [high-level-architecture.svg](architecture/high-level-architecture.svg) | Layered architecture diagram (SVG) | -| [request-context.md](architecture/request-context.md) | `RequestContext` — the per-request envelope threaded through every SPI (tenant, principal, ACLs, PII policy, trace, budget) | +| [request-context.md](architecture/request-context.md) | `RequestContext` — the per-request envelope threaded through every SPI (tenant, principal, namespace, ACLs, PII policy, trace, budget) | +| [multi-tenancy.md](architecture/multi-tenancy.md) | Logical multi-tenancy (Step 6.1): make per-tenant `rag.yaml` config drive requests. `TenantResolver` (`rag_config.tenancy`) maps a tenant id → frozen `TenantSettings` (namespace / pii_policy / acl_labels), applied **once** at the gateway boundary (namespace + pii_policy onto the `RequestContext`, acl_labels unioned into the principal); unknown tenants resolve to safe defaults (namespace = id, default PII, no labels — isolated not privileged); `RequestContext.namespace` defaults to `tenant_id` (Pinecone partitions on it; `filter_pushdown` unchanged); resolves+threads only — ACL push-down is 6.3, PII egress 6.5, physical tenancy 6.2; `GET /v1/status/tenant`; inert in `build_app` | | [policy-engine.md](architecture/policy-engine.md) | `PolicyEngine` (PDP) — single decision point for ACL, PII, quotas, redaction; replaces scattered governance checks | | [caching.md](architecture/caching.md) | Three-cache split: `EmbeddingCache`, `RetrievalCache`, `AnswerCache` — distinct invalidation rules | | [performance.md](architecture/performance.md) | Hot-path discipline, per-SPI p99 budgets, async telemetry, reviewer checklist | @@ -102,6 +103,7 @@ | [agent.md](reference/agent.md) | `rag-agent` + agent surface (Step 3.6) — `AgentLoop` / `AgentConfig`, `Controller` (`Scripted` / `Heuristic` / `LLM`), `Tool` / `ToolRegistry` / `RetrieveTool` / `Retriever`, `CheckpointStore` / `InMemoryCheckpointStore` / `AgentSnapshot`, the `rag_core.agent_types` wire models, `POST /v1/agent` (SSE) + gRPC `Converse` + `ragctl agent`, governance boundary, observability, extension points | | [sdks.md](reference/sdks.md) | Official SDKs (Step 3.7) — Python (`agentcontextos`) + TypeScript (`@agentcontextos/sdk`) hand-written clients, generated Go/Java/.NET, identity model, usage per language, the `task openapi:gen` / `sdk:gen` pipeline, extension points | | [admin-ui.md](reference/admin-ui.md) | Admin console (Step 3.10) — Next.js 14 operator GUI (`apps/admin-ui`); 9 pages (dashboard, corpora, connectors, glossary, webhooks, audit, API keys, tenants, config), live-vs-seed hybrid + `NEXT_PUBLIC_GATEWAY_URL`, header identity, running it, internals (shell/primitives/data layer), extension points | +| [tenancy.md](reference/tenancy.md) | Logical multi-tenancy (Step 6.1) — per-tenant `rag.yaml` config (`namespace` / `acl_labels` / `pii_policy` / `quota`); `TenantResolver.resolve(id) → TenantSettings`; `RequestContext.namespace`; `GET /v1/status/tenant`; `ragctl tenant list` / `resolve`; config table + scope/boundaries (6.2/6.3/6.5) + extension points | | [webhooks.md](reference/webhooks.md) | Outbound webhooks (Step 3.9) — event catalogue (`ingest.completed` / `audit.policy_violation` / `drift.detected` / `eval.regression`), event envelope, HMAC signing + `verify()`, at-least-once delivery, `/v1/webhooks/subscriptions` CRUD + test, `rag.yaml` block, `ragctl webhooks demo`, internals + extension points | | [integrations.md](reference/integrations.md) | Framework adapters (Step 3.8) — `agentcontextos.integrations.*` for LangChain / LlamaIndex / Haystack / DSPy / LangGraph / CrewAI / AutoGen / Semantic Kernel; per-framework extras, shared config + chunk metadata, usage per framework, internals + extension points | | [status-api.md](reference/status-api.md) | Status & Metrics API (Step 3.11) — `/v1/status/health` / `metrics` / `logs` (+ SSE `logs/stream`), `WS /v1/status/ws`, `/v1/connectors/status`; metric catalogue + request-timing middleware, the `MetricsCollector` / `LogTail` read-side, CORS + query-param identity for browser streams, extension points | @@ -169,6 +171,7 @@ broken, and what to fix before committing to the next phase. | [ADR-0026-per-query-tracing-provenance.md](adr/ADR-0026-per-query-tracing-provenance.md) | Decision (Step 5.1): make one query explainable + trustworthy via three composing parts — stamp `rag.schema_version` on **every** span at the single `span_from_trace_context` choke point (with a `telemetry_attrs` registry + contract test) rather than rewriting 21 call sites; capture an HMAC-signed `ProvenanceRecord` (privacy-by-default hashes, degrade-open, signing optional but record not) in a new `rag-provenance` package modeled on the webhook signer; a read-side `TraceCollector` span processor grouping by `rag.trace_id` behind tenant-isolated `GET /v1/query/{id}/trace`; inert-by-default `build_app` wiring (on via config); rejected per-call-site version stamping, storing raw text, request-id span grouping, fail-closed recording | | [ADR-0027-offline-eval-harness.md](adr/ADR-0027-offline-eval-harness.md) | Decision (Step 5.2): complete the Step 0.8 eval skeleton with a deterministic, offline harness — synthetic 5-domain corpus on the noop SPIs (a `HashingEmbedder` gives the zero-vector dense path real signal, fused with `NoopKeywordStore` token overlap via the real `HybridRetriever`); 500-query golden set committed under `tests/eval/golden/` and generated reproducibly from the corpus (drift-gated), 3 passages/concept so nDCG + Citation Precision aren't degenerate; nDCG added + dependency-free `lexical_faithfulness` default (RAGAS optional) keep CI ML-free; harness in `eval/golden_set_v0/` (not `rag_config`) so config stays light; additive report fields + self-contained HTML; `--check` enforces the threshold floor now (regression-delta is 5.3); rejected real backends in CI, RAGAS-as-default, hand-authored queries | | [ADR-0032-ab-testing-shadow-mode.md](adr/ADR-0032-ab-testing-shadow-mode.md) | Decision (Step 5.7): compare two configs on live traffic, delivered in slices (5.7a analyzer+tracker+surface, 5.7b shadow, 5.7c routing, 5.7d console); the analyzer is pure + stdlib-only in `rag_config.eval` (normal-approx Welch via `statistics.NormalDist`, no numpy/scipy — same spirit as drift PSI / cost z-score); the `ABExperimentTracker` is a pure *sample holder* in `rag_observability` (so it doesn't import `rag_config`), and the gateway composes the two for `GET /v1/status/experiments`; opt-in by default (A/B routing can change responses); `ABAnalysisResult` additive in `rag_core.eval` (not in `dist/schemas`); rejected numpy/scipy, a new package, putting the analyzer in observability, defaulting on | +| [ADR-0033-logical-multi-tenancy.md](adr/ADR-0033-logical-multi-tenancy.md) | Decision (Step 6.1): make per-tenant `rag.yaml` config drive requests. `TenantResolver` (`rag_config`) maps a tenant id → frozen `TenantSettings` (`rag_core`), applied once at the gateway boundary; resolver in config / settings type in core keeps the `config → core` direction; unknown tenants → safe defaults (isolated not privileged); `RequestContext.namespace` defaults to `tenant_id` (a backend-partition primitive — Pinecone uses it — not a chunk field, so `filter_pushdown` is unchanged); scope stops at resolution + threading (ACL push-down 6.3, PII egress 6.5, physical tenancy 6.2); additive + inert in `build_app` | | [ADR-0031-cost-anomaly.md](adr/ADR-0031-cost-anomaly.md) | Decision (Step 5.6c): detect per-tenant spend spikes with a rolling `CostTracker` (not the cumulative quota counter); detect scale-free on the token series (cost = tokens × a constant price) so detection is decoupled from quota pricing and works with quotas off; two gates (ratio + z-score, z relaxed on a flat baseline) → tri-state verdict; put it in `rag-observability` as a `dataclass` (gateway wraps it in a Pydantic `CostStatusResponse`) so there's **no `rag-core` type / `dist/schemas` churn**; feed O(1) from `record_request_usage` before the quota block; pull-based `GET /v1/status/cost` (no per-request span/event); rejected folding into the infra-scoped drift registry, a new package, a `cost.anomaly_detected` push event (deferred), per-model pricing, a time-series DB | | [ADR-0030-drift-monitors.md](adr/ADR-0030-drift-monitors.md) | Decision (Step 5.5): detect retrieval degradation with five drift monitors in a new `rag-drift` package (mirroring rag-feedback); two statistics — PSI (pure, binned, dependency-free) for the distribution monitors + mean-drop for the rate/score monitors — over one scalar-window `DriftMonitor`; infra-scoped registry (like breakers) fed via `observe` from the signals the gateway already computes (query length / retrieval score / HyDE-embedding norm / guard grounded-claim fraction / feedback citation clicks); detection on dashboard-poll `evaluate()` with transition-edge `drift.detected` (structured event + the Step 3.9 webhook, targeting `alert_tenant`); observe-only / inert-by-default / rebaseline; rejected per-tenant monitors, per-dimension embedding PSI, a stats library, a background scheduler, hot-path detection | | [ADR-0029-online-feedback.md](adr/ADR-0029-online-feedback.md) | Decision (Step 5.4): capture online feedback + implicit signals in a new `rag-feedback` package mirroring `rag-provenance` (SPI + types in rag-core; recorder + pure aggregator in the package); one polymorphic `POST /v1/feedback` (a `signal` enum spanning explicit thumbs/rating/comment + implicit citation-click/copy/regenerate/dwell, `kind` inferred); normalise every signal to a `[-1,1]` score so the dashboard has one satisfaction number; **redact-don't-hash** free-text comments via an injected `PIIDetector` (default `NoopPIIDetector` seam, `comment_redacted` flag, PII-free event) — opposite of provenance's hashing; body identity like `/v1/query`; degrade-open + inert-by-default; `GET /v1/status/feedback` dashboard (event-only, no per-call span); admin-UI card deferred to 5.6; rejected separate per-signal endpoints, header-auth, hashing/raw comments, an OTel span per submission, folding into provenance | diff --git a/docs/adr/ADR-0033-logical-multi-tenancy.md b/docs/adr/ADR-0033-logical-multi-tenancy.md new file mode 100644 index 0000000..1e5ea10 --- /dev/null +++ b/docs/adr/ADR-0033-logical-multi-tenancy.md @@ -0,0 +1,77 @@ +# ADR-0033 — Logical multi-tenancy + +**Status:** Accepted +**Date:** 2026-06-07 +**Step:** 6.1 — Logical multi-tenancy (Phase 6 — Governance & Tenancy) +**Related:** [ADR-0005](ADR-0005-policy-engine.md) (PolicyEngine PDP), [request-context.md](../architecture/request-context.md), [reference/tenancy.md](../reference/tenancy.md) + +## Context + +A lot of tenancy machinery already existed: `RequestContext.tenant_id` threaded to +every SPI, per-document/chunk `acl_labels`, `PolicyEngine.filter_pushdown` scoping +by tenant id, per-tenant quotas (Step 4.5), and Pinecone namespacing by tenant id. +But `TenantConfig` in `rag.yaml` was a **passive registry** — only its `quota` fed +anything; the gateway used one global `pii_policy` / config for every tenant, and +the request-context docs even *showed* `pii_policy = tenant_config.pii_policy` +threading that was never actually wired. + +Step 6.1 makes per-tenant config drive requests — the *logical* tenancy +foundation the rest of Phase 6 (physical tenancy, ACL push-down, PII egress) +builds on. + +## Decision + +**1. A resolver produces a frozen `TenantSettings`, applied once at the boundary.** +`TenantResolver` (`rag_config.tenancy`) maps a tenant id → `TenantSettings` +(`rag_core.types`: tenant_id, namespace, pii_policy, acl_labels), merging the +`rag.yaml` entry with defaults. The request-context middleware resolves it at the +gateway edge — where `RequestContext` is already "constructed exactly once" — and +applies it: namespace + pii_policy onto the context, acl_labels unioned into the +principal. No hot-path re-resolution. + +**2. Resolver lives in `rag_config`, the settings type in `rag_core`.** The resolved +type is core (threaded everywhere); the resolver maps `RagConfig → TenantSettings`, +so it belongs in `rag_config` (which already depends on `rag_core`). This keeps the +`config → core` dependency direction intact. + +**3. Unknown tenants resolve to safe defaults.** A tenant absent from `tenants:` +gets namespace = its own id, the default PII action (`redact`), and no labels — +isolated, never privileged. A typo can't inherit another tenant's partition. + +**4. `RequestContext` gains a `namespace` that defaults to `tenant_id`.** A +before-validator fills it in, so every pre-6.1 construction and every tenant-id +scoped backend is isolated exactly as before. The namespace is a *backend-partition* +primitive (Pinecone partitions on `ctx.namespace`), **not** a chunk field — so +`filter_pushdown` is unchanged (it scopes `tenant_id`, which is what chunks carry). + +**5. Scope stops at resolution + threading.** ACL push-down *enforcement* is 6.3, +PII *egress* enforcement is 6.5, physical tenancy is 6.2. 6.1 resolves and threads +the primitives those steps consume; it adds the config model + the boundary plumbing ++ observability (`GET /v1/status/tenant`, `ragctl tenant`). + +**6. Additive, inert by default.** `TenantSettings` joins `dist/schemas`; +`RequestContext` regenerates (new `namespace`); `TenantConfig` gains `namespace` + +`acl_labels` (→ `rag.schema`); `/v1/status/tenant` joins `dist/openapi`. `build_app` +wires no resolver (pre-6.1 behaviour); `build_app_from_config` always builds one. + +## Consequences + +**Positive** +- Per-tenant config finally drives requests; the resolution is a pure, testable + function and the boundary application is a few lines. +- Unknown-tenant defaults make misconfiguration safe-by-default. +- The namespace gives native-partitioning backends a real per-tenant key without + weakening the tenant-id filter the others rely on. + +**Negative / deferred** +- Only isolation + governance primitives are resolved; per-tenant *behavioural* + overrides (retrieval weights, guard threshold) are a documented extension, not + built here. +- The config `PIIPolicy` enum only expresses `block`/`redact`/`allow`; the richer + core `PiiAction`s (`mask`/`encrypt`/`tag_only`) aren't reachable from `rag.yaml`. +- The PolicyEngine still scopes `tenant_id` only; consuming the tenant's ACL labels + in `filter_pushdown` waits for ACL push-down (6.3). + +## See also +- [reference/tenancy.md](../reference/tenancy.md) — API + config + endpoint + CLI +- [architecture/multi-tenancy.md](../architecture/multi-tenancy.md) — design + boundaries diff --git a/docs/architecture/multi-tenancy.md b/docs/architecture/multi-tenancy.md new file mode 100644 index 0000000..601c35e --- /dev/null +++ b/docs/architecture/multi-tenancy.md @@ -0,0 +1,84 @@ +# Logical multi-tenancy — architecture + +How per-tenant `rag.yaml` config becomes per-request behaviour. Decision record: +[ADR-0033](../adr/ADR-0033-logical-multi-tenancy.md). Public API + config: +[reference/tenancy.md](../reference/tenancy.md). + +## Overview + +Before Step 6.1, `TenantConfig` was a *passive registry*: the gateway read one +global `retrieval` / `guard` / PII config for every tenant, and the `tenants:` +block only fed the Step 4.5 quota enforcer. Logical multi-tenancy closes that gap +— a tenant's declared config now drives its requests — without giving each tenant +its own infrastructure (that's *physical* tenancy, Step 6.2). + +``` +request (X-Tenant-Id: acme) + → middleware: settings = TenantResolver.resolve("acme") + └─ RequestContext(namespace=settings.namespace, + pii_policy=settings.pii_policy, + principal=principal + settings.acl_labels) + → every SPI sees ctx.namespace + ctx.pii_policy + principal.acl_labels +``` + +## Key decisions + +- **Resolve once, at the boundary.** `RequestContext` is constructed exactly once + at the gateway edge, so that is where the tenant's settings are applied + (`build_gateway_context`). Downstream code treats the resolved namespace / PII + policy / ACL labels as trusted and immutable — no hot-path re-resolution. +- **`TenantSettings` is the resolved unit.** A frozen `rag_core.types` model + carrying only the isolation + governance primitives (namespace, pii_policy, + acl_labels). The `TenantResolver` (`rag_config.tenancy`) maps a tenant id to it, + merging the `rag.yaml` entry with safe defaults. Keeping the resolver in + `rag_config` (which already depends on `rag_core`) respects the dependency graph. +- **Unknown tenants are isolated, never privileged.** A tenant absent from + `tenants:` resolves to defaults — namespace = its own id, the default PII action, + no ACL labels — so a typo'd or unconfigured tenant can never inherit another + tenant's partition or labels. +- **Namespace defaults to the tenant id.** The new `RequestContext.namespace` + falls back to `tenant_id` via a before-validator, so every pre-6.1 call site — + and every backend that scopes by tenant id — is isolated exactly as before. An + operator only sets a distinct namespace to opt into native backend partitioning. +- **Namespace is a backend-partition primitive, not a chunk field.** Backends that + namespace natively (Pinecone) partition on `ctx.namespace`; filter-based backends + keep their `tenant_id` metadata filter (equivalent when namespace = id). The + PolicyEngine `filter_pushdown` is unchanged — it scopes `tenant_id`, because + chunks carry a tenant id, not a namespace. +- **ACL labels union, not replace.** The tenant's `acl_labels` are unioned into the + principal's own labels, so tenant-wide grants compose with per-principal grants. + +## Governance & boundaries + +This is the *foundation* for Phase 6; it resolves and threads the primitives that +later steps enforce: + +| Concern | Step | This step's part | +|---------|------|------------------| +| Per-tenant config / namespace / ACL+PII *model* | **6.1** | resolve + thread | +| Physical tenancy (dedicated index) | 6.2 | — | +| ACL push-down at retrieval | 6.3 | labels are on the principal, ready to inject | +| ACL egress verifier | 6.4 | — | +| PII egress enforcement | 6.5 | `ctx.pii_policy` is resolved, ready to enforce | + +## Observability + +- `GET /v1/status/tenant` returns the resolved view for a tenant (namespace, PII + action, ACL labels, and whether it's `known` in config) — so an operator can see + exactly what config a tenant's requests run under. +- `ragctl tenant resolve ` prints the same resolution against a `rag.yaml`. + +## Inert by default + +`build_app` wires no resolver, so its `RequestContext` keeps the pre-6.1 behaviour +(namespace = tenant id, default PII policy, principal labels unchanged). +`build_app_from_config` always builds a `TenantResolver` from `cfg.tenants`. + +## Extension points + +- **Resolved behavioural overrides** — the resolver is the natural home for + per-tenant retrieval weights / guard thresholds once a component consumes them. +- **Namespace strategy** — extend any backend to honour `ctx.namespace` for native + partitioning (Pinecone is the worked example). +- **A real PolicyEngine** — consume the principal's (now tenant-augmented) ACL + labels in `filter_pushdown` when ACL push-down lands (6.3). diff --git a/docs/architecture/request-context.md b/docs/architecture/request-context.md index 77e4717..b273c17 100644 --- a/docs/architecture/request-context.md +++ b/docs/architecture/request-context.md @@ -28,6 +28,7 @@ class RequestContext(BaseModel): request_id: RequestId # ULID-like ID; round-trips with the trace tenant_id: TenantId # required; matched against principal.tenant_id principal: Principal # user/service identity + ACL labels + namespace: str # logical-isolation key (Step 6.1); defaults to tenant_id pii_policy: PiiPolicy # per-tenant: redact | mask | encrypt | tag_only | block | allow trace: TraceContext # OTel span + correlation IDs budget: Budget # tokens, dollars, wall_ms, max_iter @@ -57,6 +58,12 @@ async def query_endpoint(req: QueryRequest, auth: AuthInfo) -> QueryResponse: return await pipeline.execute(ctx, req.query) ``` +The gateway resolves `namespace`, `pii_policy`, and the principal's `acl_labels` +from the tenant's `rag.yaml` config via the `TenantResolver` (Step 6.1) — see +[multi-tenancy.md](multi-tenancy.md). `namespace` is the logical-isolation key +threaded to every SPI; it defaults to `tenant_id` when unset, so isolation is +never weaker than tenant scoping. + ### Inside an SPI ```python diff --git a/docs/reference/tenancy.md b/docs/reference/tenancy.md new file mode 100644 index 0000000..6dffec6 --- /dev/null +++ b/docs/reference/tenancy.md @@ -0,0 +1,110 @@ +# Logical multi-tenancy (Step 6.1) + +Make per-tenant `rag.yaml` config actually drive each request. A `TenantResolver` +turns a tenant id into a frozen `TenantSettings` — namespace, PII policy, ACL +labels — which the gateway applies **once** at the request boundary. See +[ADR-0033](../adr/ADR-0033-logical-multi-tenancy.md) and the design in +[architecture/multi-tenancy.md](../architecture/multi-tenancy.md). + +## Overview + +- **`TenantConfig`** (in `rag.yaml`, under `tenants:`) gains `namespace` and + `acl_labels` alongside the existing `pii_policy`, `quota`, and `corpus_ids`. +- **`TenantResolver`** (`rag_config.tenancy`) resolves a tenant id → `TenantSettings` + (`rag_core.types`), merging the tenant entry with safe defaults. An **unknown** + tenant (absent from `tenants:`) resolves to defaults — namespace = the tenant + id, the default PII action (`redact`), no labels — so it is *isolated, never + privileged*. +- **The gateway** builds the resolver from config, and the request-context + middleware applies the resolved settings: `namespace` + `pii_policy` land on the + `RequestContext`, and the tenant's `acl_labels` are unioned into the principal's. +- **`GET /v1/status/tenant`** surfaces the resolved settings for a tenant. + +`RequestContext` now carries a `namespace` field (the logical-isolation key, +threaded to every SPI). It defaults to the tenant id, so every pre-6.1 call site +is isolated exactly as before. + +## Usage + +### Configuration (`tenants:`) + +```yaml +tenants: + - id: acme + name: Acme Corp + pii_policy: block # block | redact | allow → the request's PII action + namespace: acme-prod # logical-isolation key (optional; defaults to id) + acl_labels: [region:eu] # unioned into every acme principal's labels + quota: { qps: 50 } # per-tenant caps (Step 4.5) + - id: beta + name: Beta LLC # no namespace → defaults to "beta" +``` + +| Field | Default | Meaning | +|-------|---------|---------| +| `pii_policy` | `block` | The request's PII action (`block` / `redact` / `allow`). | +| `namespace` | `id` | Logical-isolation key; backends that namespace natively (e.g. Pinecone) partition on it. | +| `acl_labels` | `[]` | Tenant-wide ACL labels unioned into the principal's labels. | +| `quota` | uncapped | Per-tenant rate / usage caps (Step 4.5). | +| `corpus_ids` | `[]` | Corpora visible to the tenant. | + +### Resolver + +```python +from rag_config import RagConfig, TenantResolver + +resolver = TenantResolver.from_config(cfg) +s = resolver.resolve("acme") # → TenantSettings +s.namespace # "acme-prod" +s.pii_policy.action # PiiAction.block +s.acl_labels # frozenset({"region:eu"}) +resolver.known("acme") # True; resolver.known("ghost") → False +``` + +### HTTP + +`GET /v1/status/tenant` → `{ tenant_id, known, namespace, pii_action, acl_labels }`. +Tenant resolves from the `tenant_id` query param, else the `X-Tenant-Id` header, +else the gateway default. `known=false` when the tenant is absent from `rag.yaml` +(or no resolver is wired) — the response then reports the safe defaults. + +### CLI + +```bash +ragctl tenant list -f rag.yaml # the declared tenants +ragctl tenant resolve acme -f rag.yaml # the effective settings the gateway applies +``` + +## Internals + +- **Resolved once, at the boundary.** `RequestContext` is "constructed exactly + once at the gateway boundary", so the resolution happens there + (`build_gateway_context`) — downstream code treats the namespace / PII policy / + ACL labels as trusted and immutable. +- **PII enum maps one-for-one.** The config `PIIPolicy` (`block`/`redact`/`allow`) + maps directly onto the core `PiiAction`; the richer core actions (`mask` / + `encrypt` / `tag_only`) aren't expressible from the config enum yet. +- **Inert by default.** `build_app` wires no resolver (pre-6.1 behaviour: + namespace = tenant id, default PII); `build_app_from_config` always builds one + from `cfg.tenants`. + +## Scope & boundaries + +This step is the *logical-tenancy foundation*. It resolves and threads the +per-tenant isolation + governance primitives; the enforcement that builds on them +lands in later Phase-6 steps: + +- **ACL push-down** at retrieval (inject the labels into every backend query) — **6.3**. +- **PII egress** enforcement (block / redact answers per policy) — **6.5**. +- **Physical tenancy** (a dedicated index per tenant) — **6.2**. + +## Extension points + +- **Namespace strategy** — backends that namespace natively partition on + `ctx.namespace` (Pinecone today); a filter-only backend keeps tenant-id scoping + (equivalent, since the namespace defaults to the tenant id). +- **Richer per-tenant overrides** — the resolver is the single place to add + resolved behavioural overrides (retrieval weights, guard threshold) for the + components that consume them. +- **A real PolicyEngine** — consumes the tenant's `acl_labels` in `filter_pushdown` + once ACL push-down lands (6.3). diff --git a/packages/backends/src/rag_backends/vector/pinecone.py b/packages/backends/src/rag_backends/vector/pinecone.py index ec8d7ec..b0f4495 100644 --- a/packages/backends/src/rag_backends/vector/pinecone.py +++ b/packages/backends/src/rag_backends/vector/pinecone.py @@ -8,10 +8,13 @@ refs = await store.retrieve_ids(ctx, query_vector, top_k=10, corpus_ids=[]) await store.close() -Pinecone's native multi-tenancy primitive is the **namespace** — we use one -namespace per ``tenant_id`` so writes / queries / deletes for one tenant -cannot bleed into another even before the explicit ``tenant_id`` filter is -applied. ACL labels and other metadata are stored alongside each vector +Pinecone's native multi-tenancy primitive is the **namespace** — we partition +on ``ctx.namespace``, the per-tenant logical-isolation key resolved from +``rag.yaml`` (Step 6.1), which defaults to ``tenant_id``. So writes / queries / +deletes for one tenant cannot bleed into another even before the explicit +``tenant_id`` metadata filter is applied, and an operator can give a tenant a +dedicated namespace (or fold several tenants into one) without weakening the +tenant-id filter. ACL labels and other metadata are stored alongside each vector and pushed down via the FilterExpr translator. The ``pinecone`` Python SDK is an optional extra (``rag-backends[pinecone]``) @@ -179,6 +182,7 @@ async def bulk_index( return index = self._require_index() tenant_id = str(ctx.tenant_id) + namespace = str(ctx.namespace) vectors = [ { @@ -194,7 +198,7 @@ async def bulk_index( } for emb in embeddings ] - await index.upsert(vectors=vectors, namespace=tenant_id) + await index.upsert(vectors=vectors, namespace=namespace) async def retrieve_ids( self, @@ -206,6 +210,7 @@ async def retrieve_ids( ) -> list[ChunkRef]: index = self._require_index() tenant_id = str(ctx.tenant_id) + namespace = str(ctx.namespace) # Build the metadata filter by composing the explicit ``filters`` # AST with any corpus_id restriction supplied separately. @@ -217,7 +222,7 @@ async def retrieve_ids( kwargs: dict[str, Any] = { "vector": vector, "top_k": top_k, - "namespace": tenant_id, + "namespace": namespace, "include_metadata": True, } if composite: @@ -256,7 +261,7 @@ async def bulk_delete(self, ctx: RequestContext, chunk_ids: list[ChunkId]) -> No index = self._require_index() tenant_id = str(ctx.tenant_id) ids = [_vector_id(tenant_id, str(cid)) for cid in chunk_ids] - await index.delete(ids=ids, namespace=tenant_id) + await index.delete(ids=ids, namespace=str(ctx.namespace)) async def health(self) -> bool: try: diff --git a/packages/config/src/rag_config/__init__.py b/packages/config/src/rag_config/__init__.py index 6d00faf..a4f25af 100644 --- a/packages/config/src/rag_config/__init__.py +++ b/packages/config/src/rag_config/__init__.py @@ -57,6 +57,7 @@ VectorStoreConfig, VectorStoreProvider, ) +from rag_config.tenancy import TenantResolver __version__ = "0.1.0" @@ -105,6 +106,7 @@ # tenants "TenantConfig", "TenantQuota", + "TenantResolver", "PIIPolicy", # corpora (Step 3.5) "CorpusDefn", diff --git a/packages/config/src/rag_config/schema.py b/packages/config/src/rag_config/schema.py index 94536bd..9bc02cb 100644 --- a/packages/config/src/rag_config/schema.py +++ b/packages/config/src/rag_config/schema.py @@ -313,11 +313,26 @@ class TenantQuota(_StrictBase): class TenantConfig(_StrictBase): + """Per-tenant configuration entry in ``rag.yaml`` (Step 6.1 logical tenancy). + + Registered under ``tenants:`` and resolved at the gateway boundary by the + ``TenantResolver`` into a ``rag_core.types.TenantSettings`` that drives the + request: ``namespace`` is the logical-isolation key (defaults to ``id``); + ``pii_policy`` becomes the request's PII action; ``acl_labels`` are unioned + into the principal's labels; ``quota`` is wired into the Step 4.5 enforcer. + """ + id: Annotated[str, Field(min_length=1)] name: Annotated[str, Field(min_length=1)] pii_policy: PIIPolicy = PIIPolicy.BLOCK quota: TenantQuota = Field(default_factory=TenantQuota) corpus_ids: list[str] = Field(default_factory=list) + # Logical-isolation key for this tenant's data. ``None`` → defaults to ``id`` + # (the resolver fills it in), so isolation is never weaker than tenant scoping. + namespace: Annotated[str, Field(min_length=1)] | None = None + # Tenant-wide ACL labels unioned into every principal of this tenant. The + # config side of ACLs (Step 6.1); retrieval-time push-down enforcement is 6.3. + acl_labels: list[str] = Field(default_factory=list) # --------------------------------------------------------------------------- diff --git a/packages/config/src/rag_config/tenancy.py b/packages/config/src/rag_config/tenancy.py new file mode 100644 index 0000000..57f40bd --- /dev/null +++ b/packages/config/src/rag_config/tenancy.py @@ -0,0 +1,83 @@ +"""Per-tenant settings resolution — the logical-tenancy boundary (Step 6.1). + +Resolves a tenant id against the ``tenants`` block of ``rag.yaml`` into a frozen +:class:`rag_core.types.TenantSettings` that the gateway applies **once** at the +request boundary. An unknown tenant resolves to safe defaults (namespace = the +tenant id, the default PII policy, no extra ACL labels) so a tenant absent from +the config is isolated, never privileged. + +This is where ``rag.yaml`` per-tenant config stops being a passive registry and +starts driving requests. ACL *push-down* enforcement is Step 6.3 and PII *egress* +enforcement is Step 6.5; this module only resolves the per-tenant isolation + +governance primitives (namespace / pii_policy / acl_labels). +""" + +from __future__ import annotations + +from rag_core.types import PiiAction, PiiPolicy, TenantId, TenantSettings + +from rag_config.schema import PIIPolicy, RagConfig, TenantConfig + +__all__ = ["TenantResolver", "settings_for_tenant"] + + +def _pii_policy_from(policy: PIIPolicy) -> PiiPolicy: + """Map a config ``PIIPolicy`` enum to a core ``PiiPolicy`` contract. + + The enum string values align (``block`` / ``redact`` / ``allow``), so the + action maps one-for-one; the richer core actions (``mask`` / ``encrypt`` / + ``tag_only``) aren't expressible from the config enum yet. + """ + return PiiPolicy(action=PiiAction(policy.value)) + + +def settings_for_tenant(tenant: TenantConfig) -> TenantSettings: + """Resolve one ``TenantConfig`` into the frozen ``TenantSettings`` view.""" + return TenantSettings( + tenant_id=TenantId(tenant.id), + namespace=tenant.namespace or tenant.id, + pii_policy=_pii_policy_from(tenant.pii_policy), + acl_labels=frozenset(tenant.acl_labels), + ) + + +class TenantResolver: + """Resolves a tenant id → :class:`TenantSettings` from a ``RagConfig``. + + Built once from config and held on ``app.state``; :meth:`resolve` is an O(1) + dict lookup on the hot path. A tenant not present in ``cfg.tenants`` resolves + to :meth:`default_settings` (namespace = the tenant id, the default PII + policy, no labels), so an unconfigured tenant is isolated rather than + privileged. + """ + + def __init__(self, tenants: dict[str, TenantConfig]) -> None: + self._tenants = tenants + + @classmethod + def from_config(cls, cfg: RagConfig) -> TenantResolver: + return cls({t.id: t for t in cfg.tenants}) + + def resolve(self, tenant_id: str) -> TenantSettings: + """Return the resolved settings for ``tenant_id`` (defaults if unknown).""" + tenant = self._tenants.get(str(tenant_id)) + if tenant is None: + return self.default_settings(tenant_id) + return settings_for_tenant(tenant) + + def known(self, tenant_id: str) -> bool: + """Whether ``tenant_id`` is declared in ``cfg.tenants``.""" + return str(tenant_id) in self._tenants + + @staticmethod + def default_settings(tenant_id: str) -> TenantSettings: + """Safe defaults for a tenant absent from the config. + + Namespace falls back to the tenant id (isolation never weaker than tenant + scoping) and the PII policy is the core default (``redact``) — the same + behaviour a pre-6.1 request got, so an unconfigured tenant is unchanged. + """ + return TenantSettings( + tenant_id=TenantId(str(tenant_id)), + namespace=str(tenant_id), + ) diff --git a/packages/core/src/rag_core/__init__.py b/packages/core/src/rag_core/__init__.py index 1e58cb7..9c85c87 100644 --- a/packages/core/src/rag_core/__init__.py +++ b/packages/core/src/rag_core/__init__.py @@ -218,6 +218,7 @@ StopReason, Tenant, TenantId, + TenantSettings, TraceContext, TrustLevel, WriteVolume, @@ -345,6 +346,7 @@ "StopReason", "Tenant", "TenantId", + "TenantSettings", "TraceContext", "TrustLevel", "WriteVolume", diff --git a/packages/core/src/rag_core/gen_schemas.py b/packages/core/src/rag_core/gen_schemas.py index eed1d9c..7d02d06 100644 --- a/packages/core/src/rag_core/gen_schemas.py +++ b/packages/core/src/rag_core/gen_schemas.py @@ -105,6 +105,7 @@ SpanRecord, StageEvent, Tenant, + TenantSettings, TraceContext, ) from rag_core.webhook_types import ( @@ -126,6 +127,7 @@ CacheStats, RequestContext, Tenant, + TenantSettings, BlobRef, Document, Block, diff --git a/packages/core/src/rag_core/types.py b/packages/core/src/rag_core/types.py index 26eeed4..195fb82 100644 --- a/packages/core/src/rag_core/types.py +++ b/packages/core/src/rag_core/types.py @@ -363,6 +363,7 @@ class RequestContext(BaseModel): request_id: RequestId = Field(default_factory=lambda: RequestId(_new_id())) tenant_id: TenantId principal: Principal + namespace: str = "" pii_policy: PiiPolicy = Field(default_factory=PiiPolicy) trace: TraceContext = Field(default_factory=TraceContext) budget: Budget = Field(default_factory=Budget) @@ -370,6 +371,23 @@ class RequestContext(BaseModel): corpus_routing_hint: str | None = None created_at: datetime = Field(default_factory=_utcnow) + @model_validator(mode="before") + @classmethod + def _default_namespace_to_tenant(cls, data: Any) -> Any: + """Default ``namespace`` to ``tenant_id`` (Step 6.1 logical isolation). + + The namespace is the logical-isolation key threaded to every SPI. When + a caller doesn't set one — the common case, and every pre-6.1 call site — + it falls back to the tenant id, so isolation is never weaker than tenant + scoping. The gateway boundary overrides it with the tenant's configured + namespace via the ``TenantResolver``. + """ + if isinstance(data, dict) and not data.get("namespace"): + tenant = data.get("tenant_id") + if tenant is not None: + data["namespace"] = str(tenant) + return data + @model_validator(mode="after") def _check_tenant_match(self) -> RequestContext: if self.principal.tenant_id != self.tenant_id: @@ -395,6 +413,30 @@ class Tenant(BaseModel): created_at: datetime = Field(default_factory=_utcnow) +class TenantSettings(BaseModel): + """Resolved per-tenant view of config — the unit of logical tenancy (Step 6.1). + + Produced by ``rag_config.tenancy.TenantResolver`` from the ``tenants`` block of + ``rag.yaml``, merged with the global defaults, and applied **once** at the + gateway boundary: ``namespace`` + ``pii_policy`` land on the + :class:`RequestContext`, and ``acl_labels`` are unioned into the principal's + labels. An unknown tenant resolves to safe defaults (``namespace`` = the + tenant id, the default PII policy, no extra labels), so a tenant that isn't in + the config is isolated, never privileged. + + Isolation primitives only — the resolved behavioural overrides (retrieval, + guard) are applied by their own components, not carried here. ACL *push-down* + enforcement is Step 6.3; PII *egress* enforcement is Step 6.5. + """ + + model_config = {"frozen": True} + + tenant_id: TenantId + namespace: str + pii_policy: PiiPolicy = Field(default_factory=PiiPolicy) + acl_labels: frozenset[str] = Field(default_factory=frozenset) + + # --------------------------------------------------------------------------- # BlobRef — lazy pointer to chunk text or other large blobs # --------------------------------------------------------------------------- diff --git a/packages/ragctl/src/ragctl/main.py b/packages/ragctl/src/ragctl/main.py index a64f57b..2d93548 100644 --- a/packages/ragctl/src/ragctl/main.py +++ b/packages/ragctl/src/ragctl/main.py @@ -20,10 +20,10 @@ | rerank | 2.7 | working | | pack | 2.8 | working | | corpus | 3.5 | working | +| tenant | 6.1 | working | | plugin | 1.1 | scaffold | | query | 3.1 | scaffold | | logs | 5.6 | scaffold | -| tenant | 6.1 | scaffold | | secret | 6.7 | scaffold | Scaffold commands print a one-line "delivered in step X.Y" notice and exit 0 @@ -4595,6 +4595,88 @@ async def _run() -> None: asyncio.run(_run()) +# --------------------------------------------------------------------------- +# tenant — Step 6.1 logical multi-tenancy: inspect per-tenant rag.yaml config +# --------------------------------------------------------------------------- +tenant_app = typer.Typer( + help="Inspect per-tenant config + resolve the Step 6.1 logical-tenancy settings.", + no_args_is_help=True, +) +app.add_typer(tenant_app, name="tenant") + + +@tenant_app.command("list") +def tenant_list( + config: Path = typer.Option( + ..., + "--config", + "-f", + exists=True, + file_okay=True, + dir_okay=False, + readable=True, + help="Path to the rag.yaml that declares the tenants.", + ), +) -> None: + """List the tenants declared under ``tenants:`` in ``rag.yaml``.""" + try: + cfg = load(config) + except ConfigError as exc: + typer.echo(f"ERROR: {exc.message}", err=True) + raise typer.Exit(1) # noqa: B904 + + typer.echo(f"config: {config}") + typer.echo(f"tenants: {len(cfg.tenants)}") + for t in cfg.tenants: + ns = t.namespace or t.id + labels = f" acl_labels={len(t.acl_labels)}" if t.acl_labels else "" + typer.echo(f" - {t.id} ({t.name}) ns={ns} pii={t.pii_policy.value}{labels}") + + +@tenant_app.command("resolve") +def tenant_resolve( + tenant: str = typer.Argument(..., help="Tenant id to resolve."), + config: Path = typer.Option( + ..., + "--config", + "-f", + exists=True, + file_okay=True, + dir_okay=False, + readable=True, + help="Path to the rag.yaml that declares the tenants.", + ), +) -> None: + """Resolve a tenant id to its effective :class:`TenantSettings` (Step 6.1). + + Shows exactly what the gateway applies at the request boundary — the + namespace, PII action, and ACL labels — including the safe defaults an + *unknown* tenant (absent from ``tenants:``) falls back to. + + Example:: + + ragctl tenant resolve acme -f rag.yaml + """ + from rag_config import TenantResolver + + try: + cfg = load(config) + except ConfigError as exc: + typer.echo(f"ERROR: {exc.message}", err=True) + raise typer.Exit(1) # noqa: B904 + + resolver = TenantResolver.from_config(cfg) + settings = resolver.resolve(tenant) + known = resolver.known(tenant) + + typer.echo(f"tenant: {settings.tenant_id}") + typer.echo(f"known: {known}{'' if known else ' (resolved to safe defaults)'}") + typer.echo(f"namespace: {settings.namespace}") + typer.echo(f"pii_action: {settings.pii_policy.action.value}") + labels = ", ".join(sorted(settings.acl_labels)) or "(none)" + typer.echo(f"acl_labels: {labels}") + + # --------------------------------------------------------------------------- # Scaffold sub-apps — print a "delivered in Step X.Y" notice and exit 0. # @@ -4625,14 +4707,6 @@ def _entry(ctx: typer.Context) -> None: ), name="logs", ) -app.add_typer( - _scaffold_app( - "tenant", - "6.1", - "Manage tenants (lands in Step 6.1).", - ), - name="tenant", -) app.add_typer( _scaffold_app( "plugin", diff --git a/packages/ragctl/tests/test_scaffold_commands.py b/packages/ragctl/tests/test_scaffold_commands.py index 3ae31f9..4e4b2cf 100644 --- a/packages/ragctl/tests/test_scaffold_commands.py +++ b/packages/ragctl/tests/test_scaffold_commands.py @@ -13,8 +13,9 @@ # (see packages/ragctl/tests/test_ingest.py). # ``query`` was promoted from scaffold to a real command in Step 3.1 # (see packages/ragctl/tests/test_query.py). + # ``tenant`` was promoted from scaffold to a real command in Step 6.1 + # (see packages/ragctl/tests/test_tenant.py). ("logs", "5.6"), - ("tenant", "6.1"), ("plugin", "1.1"), ("secret", "6.7"), ) diff --git a/packages/ragctl/tests/test_tenant.py b/packages/ragctl/tests/test_tenant.py new file mode 100644 index 0000000..5036d97 --- /dev/null +++ b/packages/ragctl/tests/test_tenant.py @@ -0,0 +1,66 @@ +"""Tests for `ragctl tenant` — logical multi-tenancy inspection (Step 6.1).""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +from ragctl.main import app +from typer.testing import CliRunner + +runner = CliRunner() + +_CONFIG = """\ +version: "1" +tenants: + - id: acme + name: Acme Corp + pii_policy: block + namespace: acme-prod + acl_labels: [region:eu, pii:cleared] + - id: beta + name: Beta LLC + pii_policy: allow +""" + + +@pytest.fixture() +def cfg(tmp_path: Path) -> Path: + path = tmp_path / "rag.yaml" + path.write_text(_CONFIG, encoding="utf-8") + return path + + +def test_tenant_list(cfg: Path) -> None: + result = runner.invoke(app, ["tenant", "list", "-f", str(cfg)]) + assert result.exit_code == 0, result.output + assert "tenants: 2" in result.output + assert ( + "acme" in result.output and "ns=acme-prod" in result.output and "pii=block" in result.output + ) + assert "beta" in result.output and "ns=beta" in result.output + + +def test_tenant_resolve_known(cfg: Path) -> None: + result = runner.invoke(app, ["tenant", "resolve", "acme", "-f", str(cfg)]) + assert result.exit_code == 0, result.output + assert "known: True" in result.output + assert "namespace: acme-prod" in result.output + assert "pii_action: block" in result.output + assert "pii:cleared" in result.output and "region:eu" in result.output + + +def test_tenant_resolve_unknown_uses_safe_defaults(cfg: Path) -> None: + result = runner.invoke(app, ["tenant", "resolve", "ghost", "-f", str(cfg)]) + assert result.exit_code == 0, result.output + assert "known: False" in result.output + assert "safe defaults" in result.output + assert "namespace: ghost" in result.output + assert "pii_action: redact" in result.output + assert "acl_labels: (none)" in result.output + + +def test_tenant_help_works() -> None: + result = runner.invoke(app, ["tenant", "--help"]) + assert result.exit_code == 0, result.output + assert "resolve" in result.output and "list" in result.output diff --git a/tests/config/test_tenancy.py b/tests/config/test_tenancy.py new file mode 100644 index 0000000..eaf482d --- /dev/null +++ b/tests/config/test_tenancy.py @@ -0,0 +1,86 @@ +"""Tests for per-tenant settings resolution — logical multi-tenancy (Step 6.1).""" + +from __future__ import annotations + +import pytest +from pydantic import ValidationError +from rag_config import RagConfig, TenantConfig, TenantResolver +from rag_config.schema import PIIPolicy +from rag_config.tenancy import settings_for_tenant +from rag_core.types import PiiAction, TenantSettings + + +def _cfg() -> RagConfig: + return RagConfig( + tenants=[ + TenantConfig( + id="acme", + name="Acme", + pii_policy=PIIPolicy.BLOCK, + namespace="acme-prod", + acl_labels=["region:eu", "pii:cleared"], + ), + TenantConfig(id="beta", name="Beta", pii_policy=PIIPolicy.ALLOW), + ] + ) + + +# --------------------------------------------------------------------------- +# TenantConfig schema +# --------------------------------------------------------------------------- +def test_tenant_config_defaults() -> None: + t = TenantConfig(id="x", name="X") + assert t.namespace is None # resolver fills it from id + assert t.acl_labels == [] + assert t.pii_policy is PIIPolicy.BLOCK + + +def test_tenant_config_rejects_empty_namespace() -> None: + with pytest.raises(ValidationError): + TenantConfig(id="x", name="X", namespace="") + + +# --------------------------------------------------------------------------- +# Resolution +# --------------------------------------------------------------------------- +def test_resolve_known_tenant() -> None: + r = TenantResolver.from_config(_cfg()) + s = r.resolve("acme") + assert isinstance(s, TenantSettings) + assert r.known("acme") is True + assert s.namespace == "acme-prod" + assert s.pii_policy.action is PiiAction.block + assert s.acl_labels == frozenset({"region:eu", "pii:cleared"}) + + +def test_namespace_defaults_to_id_when_unset() -> None: + s = TenantResolver.from_config(_cfg()).resolve("beta") + assert s.namespace == "beta" # no explicit namespace → tenant id + assert s.pii_policy.action is PiiAction.allow + assert s.acl_labels == frozenset() + + +def test_unknown_tenant_resolves_to_safe_defaults() -> None: + r = TenantResolver.from_config(_cfg()) + s = r.resolve("ghost") + assert r.known("ghost") is False + # Isolated, never privileged: namespace = the id, PII = the default (redact). + assert s.namespace == "ghost" + assert s.pii_policy.action is PiiAction.redact + assert s.acl_labels == frozenset() + + +def test_empty_config_resolves_defaults() -> None: + r = TenantResolver.from_config(RagConfig()) + assert r.known("anyone") is False + assert r.resolve("anyone").namespace == "anyone" + + +def test_pii_enum_maps_one_for_one() -> None: + for cfg_policy, action in [ + (PIIPolicy.BLOCK, PiiAction.block), + (PIIPolicy.REDACT, PiiAction.redact), + (PIIPolicy.ALLOW, PiiAction.allow), + ]: + s = settings_for_tenant(TenantConfig(id="t", name="T", pii_policy=cfg_policy)) + assert s.pii_policy.action is action diff --git a/tests/redteam/test_cross_tenant_namespace.py b/tests/redteam/test_cross_tenant_namespace.py new file mode 100644 index 0000000..7fffaa3 --- /dev/null +++ b/tests/redteam/test_cross_tenant_namespace.py @@ -0,0 +1,93 @@ +"""Red-team probe: per-tenant settings must never bleed across tenants (Step 6.1). + +Logical multi-tenancy resolves each tenant's namespace / PII policy / ACL labels +from ``rag.yaml`` at the gateway boundary. This suite asserts that the resolution +keeps tenants isolated: two tenants get distinct namespaces, one tenant's ACL +labels never leak into another's principal, and an *unknown* tenant inherits +nothing — it falls back to its own id as the namespace, never a configured +tenant's partition. +""" + +from __future__ import annotations + +from typing import Any + +from rag_config import RagConfig, TenantConfig, TenantResolver +from rag_config.schema import PIIPolicy +from rag_core.types import PiiAction +from rag_gateway.middleware import build_gateway_context +from starlette.requests import Request + + +class _StubAuth: + async def authenticate(self, token: str, tenant_id: Any) -> Any: # pragma: no cover + raise AssertionError("auth.authenticate should not be called on the dev path") + + +def _request(tenant: str) -> Request: + headers = [(b"x-tenant-id", tenant.encode()), (b"x-principal-id", b"p1")] + return Request({"type": "http", "method": "POST", "path": "/v1/query", "headers": headers}) + + +def _resolver() -> TenantResolver: + return TenantResolver.from_config( + RagConfig( + tenants=[ + TenantConfig( + id="acme", + name="Acme", + pii_policy=PIIPolicy.BLOCK, + namespace="acme-prod", + acl_labels=["region:eu", "tier:gold"], + ), + TenantConfig( + id="globex", + name="Globex", + pii_policy=PIIPolicy.ALLOW, + namespace="globex-eu", + acl_labels=["region:us"], + ), + ] + ) + ) + + +async def _ctx(tenant: str) -> Any: + gw = await build_gateway_context( + _request(tenant), + auth=_StubAuth(), # type: ignore[arg-type] + tenant_resolver=_resolver(), + ) + assert gw.ctx is not None + return gw.ctx + + +async def test_two_tenants_get_distinct_namespaces() -> None: + acme = await _ctx("acme") + globex = await _ctx("globex") + assert acme.namespace == "acme-prod" + assert globex.namespace == "globex-eu" + assert acme.namespace != globex.namespace + + +async def test_acl_labels_do_not_bleed_across_tenants() -> None: + acme = await _ctx("acme") + globex = await _ctx("globex") + assert acme.principal.acl_labels == frozenset({"region:eu", "tier:gold"}) + assert globex.principal.acl_labels == frozenset({"region:us"}) + # No label of one tenant appears on the other. + assert acme.principal.acl_labels.isdisjoint(globex.principal.acl_labels) + + +async def test_pii_policy_is_per_tenant() -> None: + assert (await _ctx("acme")).pii_policy.action is PiiAction.block + assert (await _ctx("globex")).pii_policy.action is PiiAction.allow + + +async def test_unknown_tenant_inherits_no_other_tenants_partition() -> None: + ghost = await _ctx("ghost") + # Falls back to its own id — never a configured tenant's namespace. + assert ghost.namespace == "ghost" + assert ghost.namespace not in {"acme-prod", "globex-eu"} + assert ghost.principal.acl_labels == frozenset() + assert ghost.pii_policy.action is PiiAction.redact