diff --git a/TRACKER.md b/TRACKER.md index 97b0c9c..e4b436a 100644 --- a/TRACKER.md +++ b/TRACKER.md @@ -16,11 +16,12 @@ | **Last updated** | 2026-06-08 | | **Current phase** | Phase 6 — Governance & Tenancy (**6 / 10 steps**) | | **Overall** | **70 / 84 steps** — Phases 0–5 complete | -| **Next action** | **Step 6.7b — Cloud KMS providers + storage wiring**: AWS / GCP / Azure / Vault `EnvelopeKeyManager` subclasses behind `[kms-*]` extras (lazy SDK, wrap/unwrap via KMS) + wire `EncryptingStorage` into the ingest/storage path + per-tenant key config (`cfg.kms` + `TenantConfig` key ref). Completes Step 6.7 with 6.7c (key rotation). 6.7a — the encryption library + local KMS — shipped. | +| **Next action** | **Step 6.7c — GCP/Azure/Vault KMS + key rotation**: the remaining cloud providers (GCP KMS / Azure Key Vault / HashiCorp Vault, same `EnvelopeKeyManager` pattern, behind `[kms-*]` extras) + zero-downtime key rotation (old keys retained decrypt-only until expiry). Closes Step 6.7. 6.7a (library + local) + 6.7b (config/factory/wiring + AWS KMS) shipped. | **Recently shipped** -- **6.7a** 🚧 BYOK envelope encryption library — `KeyManager` SPI + `NoopKeyManager` + `EncryptingStorage` (rag-core); `EnvelopeKeyManager` (AES-256-GCM DEK + `tenant_id` AAD) + `LocalKeyManager` (in-process per-tenant KEK) (rag-backends); per-tenant isolation + **sealing** (`KeyUnavailableError`) + tamper detection; `ragctl kms`; first slice of Step 6.7 (cloud KMS + wiring is 6.7b) — [#155](https://github.com/officialCodeWork/AgentContextOS/pull/155) +- **6.7b** 🚧 Cloud KMS (AWS) + config/factory/wiring — `cfg.kms` (provider `noop`/`local`/`aws`) + `tenants[].kms_key_id`; `build_key_manager_from_config` factory + `app.state.key_manager` gateway seam; **`AwsKmsKeyManager`** (aioboto3, per-tenant CMK, KMS-error → sealing, fake-client unit-tested) — [#156](https://github.com/officialCodeWork/AgentContextOS/pull/156) +- **6.7a** 🚧 BYOK envelope encryption library — `KeyManager` SPI + `NoopKeyManager` + `EncryptingStorage` (rag-core); `EnvelopeKeyManager` (AES-256-GCM DEK + `tenant_id` AAD) + `LocalKeyManager` (in-process per-tenant KEK) (rag-backends); per-tenant isolation + **sealing** (`KeyUnavailableError`) + tamper detection; `ragctl kms` — [#155](https://github.com/officialCodeWork/AgentContextOS/pull/155) - **6.6** ✅ Immutable audit log — **6.6a** shared `AuditWriter`/store on `app.state` + `GET /v1/audit` (tenant-scoped, `chain_verified`) + `GET /v1/audit/verify` (whole-log) + `cfg.audit.enabled` ([#153](https://github.com/officialCodeWork/AgentContextOS/pull/153)); **6.6b** `AuditExporter` self-verifying WORM bundle (SHA-256 `content_hash` + HMAC, offline `verify()`), `POST /v1/audit/export` (tenant-scoped) + `ragctl audit`, `cfg.audit.export_secret` ([#154](https://github.com/officialCodeWork/AgentContextOS/pull/154)) - **6.5** ✅ PII egress policies — `PiiPolicyEngine` (`rag-pii`) answers `egress_text` over the context (`list[Chunk]`) + agent answer (`str`) the gateway already passes, applying the per-tenant `pii_policy` (allow / redact / mask / block); reuses the Step 1.7 detector + rewriters; opt-in `cfg.pii.enabled`; emits `pii.egress_blocked` — [#152](https://github.com/officialCodeWork/AgentContextOS/pull/152) - **6.4** ✅ ACL egress verifier — `AclEgressVerifier` re-checks returned `ChunkRef`s against the principal's labels at the gateway router boundary (defense-in-depth, same overlap semantics, independent of the PDP); `acl.egress_violation` event; red-team zero-violation-rate gate — [#151](https://github.com/officialCodeWork/AgentContextOS/pull/151) @@ -656,7 +657,7 @@ | 6.4 | ACL egress verifier | ✅ | [#151](https://github.com/officialCodeWork/AgentContextOS/pull/151) — `AclEgressVerifier` re-checks returned chunks at the gateway router boundary (defense-in-depth above the 6.3 push-down); `acl.egress_violation`; zero-violation-rate red-team gate | | 6.5 | PII policies | ✅ | [#152](https://github.com/officialCodeWork/AgentContextOS/pull/152) — `PiiPolicyEngine` egress_text decorator (allow / redact / mask / block per tenant) over answer + context; reuses Step 1.7 detector; `pii.egress_blocked` | | 6.6 | Immutable audit log | ✅ | **6.6a** [#153](https://github.com/officialCodeWork/AgentContextOS/pull/153) — read API `GET /v1/audit` + `GET /v1/audit/verify` + shared store + `cfg.audit`. **6.6b** [#154](https://github.com/officialCodeWork/AgentContextOS/pull/154) — `AuditExporter` signed WORM bundle + `POST /v1/audit/export` + `ragctl audit` | -| 6.7 | BYOK (Bring Your Own Key) | 🚧 | **6.7a** ✅ [#155](https://github.com/officialCodeWork/AgentContextOS/pull/155) — `KeyManager` SPI + envelope encryption (`LocalKeyManager`) + `EncryptingStorage` + sealing/isolation/tamper + `ragctl kms`. **6.7b** ⏳ — cloud KMS providers + storage wiring + config; **6.7c** ⏳ — key rotation | +| 6.7 | BYOK (Bring Your Own Key) | 🚧 | **6.7a** ✅ [#155](https://github.com/officialCodeWork/AgentContextOS/pull/155) — `KeyManager` SPI + envelope encryption (`LocalKeyManager`) + `EncryptingStorage` + sealing/isolation/tamper. **6.7b** ✅ [#156](https://github.com/officialCodeWork/AgentContextOS/pull/156) — `cfg.kms` + per-tenant key + `build_key_manager_from_config` factory + `AwsKmsKeyManager`. **6.7c** ⏳ — GCP/Azure/Vault + key rotation | | 6.8 | SSO / SCIM | ⏳ | OIDC + SAML IdP federation; SCIM 2.0 user provisioning; per-tenant IdP config | | 6.9 | Air-gapped install bundle | ⏳ | Signed tarball with all images + Helm chart; offline bootstrap; cosign verification | | 6.10 | Compliance posture | ⏳ | SOC 2 Type II control mapping; GDPR data-residency config; data-retention policies | @@ -740,6 +741,12 @@ New ground — the only prior crypto was HMAC signing. The V1 plan calls for en - **Per-tenant isolation** via KEK-per-tenant **and** tenant-bound AAD (even a shared KEK can't read another tenant's blob); **sealing** via `KeyUnavailableError` (a tenant with no KEK is denied, others unaffected); **tamper-evidence** via the GCM auth tag. `cryptography` in `rag-backends`; `rag-core` stays crypto-free (SPI + decorator + noop) - **Scope:** library + local KMS only. **Deferred:** cloud KMS providers behind `[kms-*]` extras + wiring `EncryptingStorage` into the ingest/storage path + per-tenant key config (6.7b); zero-downtime key rotation (6.7c); vector encryption is out of scope by design. ~19 new tests (KeyManager contract over noop + local; envelope round-trip / non-determinism / tamper / malformed / cross-tenant / shared-KEK-still-blocked / sealing on encrypt+decrypt / wrong-size KEK / per-tenant non-impact / EncryptingStorage stores ciphertext + delegates key ops) + `ragctl kms` smoke; all gates green (ruff, mypy --strict 305 files, RAG001, policy-coverage, log-schema; no dist drift). [ADR-0039](docs/adr/ADR-0039-byok-envelope-encryption.md), [architecture/byok.md](docs/architecture/byok.md), [reference/encryption.md](docs/reference/encryption.md) +#### 6.7b — Cloud KMS (AWS) + config / factory / wiring ✅ [#156](https://github.com/officialCodeWork/AgentContextOS/pull/156) + +- Makes BYOK **configurable + provider-selectable**. New `cfg.kms` (`enabled` / `provider` ∈ `noop`/`local`/`aws` / `default_key_id` / `region` / `local_key`) + **`tenants[].kms_key_id`** (per-tenant key reference — a KMS ARN for `aws`, a hex 32-byte KEK for `local`). **`build_key_manager_from_config(cfg)`** factory (in the gateway, lazy `rag-backends` imports so `rag-backends` stays config-free) selects the provider; `build_app(key_manager=…)` exposes **`app.state.key_manager`** + `app.state.kms_enabled` (passthrough `NoopKeyManager` when disabled) +- **`AwsKmsKeyManager`** (`rag-backends/kms/aws.py`) subclasses `EnvelopeKeyManager` and wraps/unwraps the DEK via **AWS KMS** Encrypt/Decrypt for the tenant's customer-managed key (over the already-present `aioboto3` — **no new dep**). Per-tenant CMK from the key-id map (+ `default_key_id`); any KMS failure (revoked / denied / unreachable) or a missing key id → **`KeyUnavailableError`** (sealing). An **injectable `client` seam** makes it fully unit-testable with a fake KMS — no AWS creds / network +- **Scope:** AWS provider + config + factory + the `app.state.key_manager` seam. **Deferred:** GCP KMS / Azure Key Vault / HashiCorp Vault providers + key rotation (6.7c); wiring `EncryptingStorage` into the ingest path (blocked on tiered-storage plumbing — chunk content is inline today). ~14 new tests (AWS provider over a fake KMS: round-trip / KMS-key-bound cross-tenant block / sealing / default-key / KMS-error mapping; config defaults; factory noop/local/aws; gateway `app.state.key_manager` + `kms_enabled` + a local round-trip). `KmsConfig` → `rag.schema` regenerated; all gates green (ruff, mypy --strict 306 files, RAG001, policy-coverage, log-schema). [ADR-0039](docs/adr/ADR-0039-byok-envelope-encryption.md), [reference/encryption.md](docs/reference/encryption.md), [architecture/byok.md](docs/architecture/byok.md) + --- ## Phase 7 — Pilot, Harden, GA (Weeks 34–40) ⏳ @@ -899,6 +906,7 @@ Complete log of every PR. Routine Dependabot bumps are grouped; everything else | [#153](https://github.com/officialCodeWork/AgentContextOS/pull/153) | 2026-06-08 | feat(gateway): audit read API + chain verification (Step 6.6a) | | [#154](https://github.com/officialCodeWork/AgentContextOS/pull/154) | 2026-06-08 | feat(audit): WORM signed export — AuditExporter + POST /v1/audit/export (Step 6.6b) | | [#155](https://github.com/officialCodeWork/AgentContextOS/pull/155) | 2026-06-08 | feat(crypto): BYOK envelope encryption library + local KMS (Step 6.7a) | +| [#156](https://github.com/officialCodeWork/AgentContextOS/pull/156) | 2026-06-08 | feat(crypto): cfg.kms + key-manager factory + AWS KMS provider (Step 6.7b) | | #78–#80, #116–#118 | Open | Dependabot bumps — awaiting merge | | #81 | Closed | Dependabot bump — superseded | diff --git a/apps/gateway/src/rag_gateway/app.py b/apps/gateway/src/rag_gateway/app.py index 9859d67..9d0a419 100644 --- a/apps/gateway/src/rag_gateway/app.py +++ b/apps/gateway/src/rag_gateway/app.py @@ -376,6 +376,7 @@ def build_app( pii_detector: Any | None = None, audit_enabled: bool = True, audit_export_secret: str = "", + key_manager: Any | None = None, enable_cors: bool = True, default_tenant_id: TenantId | None = None, ) -> FastAPI: @@ -482,6 +483,16 @@ def build_app( # (empty → unsigned but still content-hashed). The artifact an operator # archives to immutable storage (S3 Object Lock). app.state.audit_exporter = AuditExporter(audit_export_secret) + # BYOK key manager (Step 6.7) — exposed for envelope-encrypting blobs at rest + # (wrap a Storage backend with EncryptingStorage). Defaults to the + # passthrough NoopKeyManager; build_app_from_config selects a real provider + # (LocalKeyManager / AwsKmsKeyManager) from cfg.kms. + from rag_core.spi.noop import NoopKeyManager + + if key_manager is None: + key_manager = NoopKeyManager() + app.state.key_manager = key_manager + app.state.kms_enabled = not isinstance(key_manager, NoopKeyManager) # Corpus router (Step 3.5) sits above the retrieval router; with the # default empty corpus store it degrades to a single unconstrained # retrieval, preserving the pre-3.5 demo behaviour. It shares the audit diff --git a/apps/gateway/src/rag_gateway/wiring.py b/apps/gateway/src/rag_gateway/wiring.py index fee9595..ea7b090 100644 --- a/apps/gateway/src/rag_gateway/wiring.py +++ b/apps/gateway/src/rag_gateway/wiring.py @@ -152,6 +152,41 @@ def build_corpus_store_from_config(cfg: RagConfig) -> CorpusStore: return NoopCorpusStore(seed=seed) +def build_key_manager_from_config(cfg: RagConfig) -> Any: + """Build a BYOK ``KeyManager`` from ``cfg.kms`` (Step 6.7). + + Returns a passthrough ``NoopKeyManager`` when disabled / ``provider=noop``; a + ``LocalKeyManager`` (in-process per-tenant KEKs decoded from + ``tenants[].kms_key_id`` hex) for ``local``; an ``AwsKmsKeyManager`` (per-tenant + KMS key ARNs) for ``aws``. GCP / Azure / Vault land in Step 6.7c. Each + tenant's key is ``tenants[].kms_key_id`` with ``kms.default_key_id`` as the + fallback; a tenant with neither is *sealed* at encrypt/decrypt time. + """ + from rag_config.schema import KmsProvider + from rag_core.spi.noop import NoopKeyManager + + kms = cfg.kms + if not kms.enabled or kms.provider is KmsProvider.NOOP: + return NoopKeyManager() + + if kms.provider is KmsProvider.LOCAL: + from rag_backends import LocalKeyManager + + keks = {t.id: bytes.fromhex(t.kms_key_id) for t in cfg.tenants if t.kms_key_id} + default = bytes.fromhex(kms.local_key) if kms.local_key else None + return LocalKeyManager(keks=keks, default_kek=default) + + if kms.provider is KmsProvider.AWS: + from rag_backends import AwsKmsKeyManager + + key_ids = {t.id: t.kms_key_id for t in cfg.tenants if t.kms_key_id} + return AwsKmsKeyManager( + key_ids=key_ids, default_key_id=kms.default_key_id or None, region=kms.region + ) + + return NoopKeyManager() # pragma: no cover - providers above are exhaustive + + async def seed_corpus_store(store: CorpusStore, cfg: RagConfig) -> int: """Initialise + seed a corpus store from ``cfg.corpora``. @@ -716,6 +751,10 @@ def build_app_from_config(cfg: RagConfig, **overrides: Any) -> FastAPI: audit_enabled = overrides.pop("audit_enabled", cfg.audit.enabled) audit_export_secret = overrides.pop("audit_export_secret", cfg.audit.export_secret) + # BYOK key manager (Step 6.7) — select the provider from cfg.kms (Noop when + # disabled). Operators inject their own via the ``key_manager`` override. + key_manager = overrides.pop("key_manager", None) or build_key_manager_from_config(cfg) + return build_app( corpus_store=corpus_store, retrieval_router=retrieval_router, @@ -730,6 +769,7 @@ def build_app_from_config(cfg: RagConfig, **overrides: Any) -> FastAPI: pii_enabled=pii_enabled, audit_enabled=audit_enabled, audit_export_secret=audit_export_secret, + key_manager=key_manager, **overrides, ) diff --git a/apps/gateway/tests/test_kms.py b/apps/gateway/tests/test_kms.py new file mode 100644 index 0000000..b9ea51a --- /dev/null +++ b/apps/gateway/tests/test_kms.py @@ -0,0 +1,88 @@ +"""BYOK config + key-manager factory + gateway wiring (Step 6.7b).""" + +from __future__ import annotations + +from rag_backends import AwsKmsKeyManager, LocalKeyManager +from rag_config import RagConfig +from rag_config.schema import KmsConfig, KmsProvider, TenantConfig +from rag_core.spi.noop import NoopKeyManager +from rag_gateway import build_app +from rag_gateway.wiring import build_app_from_config, build_key_manager_from_config + +_HEX_KEK = "ab" * 32 # 32-byte KEK as hex + + +def _local_cfg() -> RagConfig: + return RagConfig( + kms=KmsConfig(enabled=True, provider=KmsProvider.LOCAL), + tenants=[TenantConfig(id="acme", name="Acme", kms_key_id=_HEX_KEK)], + ) + + +# --------------------------------------------------------------------------- +# config + factory +# --------------------------------------------------------------------------- +def test_kms_config_defaults_off() -> None: + cfg = RagConfig() + assert cfg.kms.enabled is False + assert cfg.kms.provider is KmsProvider.LOCAL + + +def test_factory_noop_when_disabled() -> None: + assert isinstance(build_key_manager_from_config(RagConfig()), NoopKeyManager) + + +def test_factory_local() -> None: + km = build_key_manager_from_config(_local_cfg()) + assert isinstance(km, LocalKeyManager) + + +def test_factory_aws() -> None: + cfg = RagConfig( + kms=KmsConfig(enabled=True, provider=KmsProvider.AWS, default_key_id="arn:x"), + tenants=[TenantConfig(id="acme", name="Acme", kms_key_id="arn:aws:kms:::key/acme")], + ) + assert isinstance(build_key_manager_from_config(cfg), AwsKmsKeyManager) + + +# --------------------------------------------------------------------------- +# gateway wiring +# --------------------------------------------------------------------------- +def test_inert_by_default() -> None: + app = build_app() + assert app.state.kms_enabled is False + assert isinstance(app.state.key_manager, NoopKeyManager) + + +def test_build_from_config_wires_local_key_manager() -> None: + app = build_app_from_config(_local_cfg()) + assert app.state.kms_enabled is True + assert isinstance(app.state.key_manager, LocalKeyManager) + + +def test_build_from_config_disabled_is_noop() -> None: + app = build_app_from_config(RagConfig()) + assert app.state.kms_enabled is False + assert isinstance(app.state.key_manager, NoopKeyManager) + + +async def test_wired_local_key_manager_round_trips() -> None: + from rag_core.types import ( + Principal, + PrincipalId, + PrincipalKind, + RequestContext, + TenantId, + ) + + km = build_app_from_config(_local_cfg()).state.key_manager + tid = TenantId("acme") + ctx = RequestContext( + tenant_id=tid, + principal=Principal( + id=PrincipalId("p"), kind=PrincipalKind.user, display_name="p", tenant_id=tid + ), + ) + ct = await km.encrypt(ctx, b"secret") + assert ct != b"secret" + assert await km.decrypt(ctx, ct) == b"secret" diff --git a/dist/rag.schema.json b/dist/rag.schema.json index 353a522..803988e 100644 --- a/dist/rag.schema.json +++ b/dist/rag.schema.json @@ -890,6 +890,48 @@ "title": "KeywordStoreProvider", "type": "string" }, + "KmsConfig": { + "additionalProperties": false, + "description": "BYOK envelope encryption (Step 6.7).\n\nWhen ``enabled`` the gateway builds a ``KeyManager`` (selected by ``provider``)\nand exposes it on ``app.state.key_manager``; wrap a ``Storage`` backend with\n``EncryptingStorage`` to encrypt chunk content / blobs **at rest** with\nper-tenant, customer-controlled keys (embedding vectors stay plaintext for\nsearch). Each tenant's key reference is ``tenants[].kms_key_id`` (a KMS key\nARN for ``aws``, a hex 32-byte KEK for ``local``); a tenant with no key \u2014\nand no ``default_key_id`` \u2014 is **sealed** (``KeyUnavailableError``).\n\n**Disabled by default.** ``provider``: ``local`` (in-process KEK, dev /\nair-gapped), ``aws`` (AWS KMS), or ``noop`` (passthrough). GCP / Azure /\nVault providers + key rotation land in Step 6.7c.\n\n* ``default_key_id`` \u2014 fallback key reference for tenants without their own.\n* ``region`` \u2014 AWS region for the ``aws`` provider.\n* ``local_key`` \u2014 hex 32-byte default KEK for the ``local`` provider\n (``${ENV_VAR}``-interpolated; keep it out of the file).", + "properties": { + "enabled": { + "default": false, + "title": "Enabled", + "type": "boolean" + }, + "provider": { + "$ref": "#/$defs/KmsProvider", + "default": "local" + }, + "default_key_id": { + "default": "", + "title": "Default Key Id", + "type": "string" + }, + "region": { + "default": "us-east-1", + "title": "Region", + "type": "string" + }, + "local_key": { + "default": "", + "title": "Local Key", + "type": "string" + } + }, + "title": "KmsConfig", + "type": "object" + }, + "KmsProvider": { + "description": "BYOK key-manager provider (Step 6.7).", + "enum": [ + "noop", + "local", + "aws" + ], + "title": "KmsProvider", + "type": "string" + }, "LLMConfig": { "additionalProperties": false, "properties": { @@ -1392,6 +1434,18 @@ ], "default": null, "title": "Dedicated Index Name" + }, + "kms_key_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Kms Key Id" } }, "required": [ @@ -1658,6 +1712,9 @@ "audit": { "$ref": "#/$defs/AuditConfig" }, + "kms": { + "$ref": "#/$defs/KmsConfig" + }, "webhooks": { "$ref": "#/$defs/WebhooksConfig" }, diff --git a/dist/rag.schema.yaml b/dist/rag.schema.yaml index b171a27..81e35e1 100644 --- a/dist/rag.schema.yaml +++ b/dist/rag.schema.yaml @@ -921,6 +921,52 @@ $defs: - azure_search title: KeywordStoreProvider type: string + KmsConfig: + additionalProperties: false + description: "BYOK envelope encryption (Step 6.7).\n\nWhen ``enabled`` the gateway\ + \ builds a ``KeyManager`` (selected by ``provider``)\nand exposes it on ``app.state.key_manager``;\ + \ wrap a ``Storage`` backend with\n``EncryptingStorage`` to encrypt chunk content\ + \ / blobs **at rest** with\nper-tenant, customer-controlled keys (embedding\ + \ vectors stay plaintext for\nsearch). Each tenant's key reference is ``tenants[].kms_key_id``\ + \ (a KMS key\nARN for ``aws``, a hex 32-byte KEK for ``local``); a tenant with\ + \ no key —\nand no ``default_key_id`` — is **sealed** (``KeyUnavailableError``).\n\ + \n**Disabled by default.** ``provider``: ``local`` (in-process KEK, dev /\n\ + air-gapped), ``aws`` (AWS KMS), or ``noop`` (passthrough). GCP / Azure /\n\ + Vault providers + key rotation land in Step 6.7c.\n\n* ``default_key_id`` —\ + \ fallback key reference for tenants without their own.\n* ``region`` — AWS\ + \ region for the ``aws`` provider.\n* ``local_key`` — hex 32-byte default KEK\ + \ for the ``local`` provider\n (``${ENV_VAR}``-interpolated; keep it out of\ + \ the file)." + properties: + enabled: + default: false + title: Enabled + type: boolean + provider: + $ref: '#/$defs/KmsProvider' + default: local + default_key_id: + default: '' + title: Default Key Id + type: string + region: + default: us-east-1 + title: Region + type: string + local_key: + default: '' + title: Local Key + type: string + title: KmsConfig + type: object + KmsProvider: + description: BYOK key-manager provider (Step 6.7). + enum: + - noop + - local + - aws + title: KmsProvider + type: string LLMConfig: additionalProperties: false properties: @@ -1379,6 +1425,12 @@ $defs: - type: 'null' default: null title: Dedicated Index Name + kms_key_id: + anyOf: + - type: string + - type: 'null' + default: null + title: Kms Key Id required: - id - name @@ -1587,6 +1639,8 @@ properties: $ref: '#/$defs/PiiConfig' audit: $ref: '#/$defs/AuditConfig' + kms: + $ref: '#/$defs/KmsConfig' webhooks: $ref: '#/$defs/WebhooksConfig' provenance: diff --git a/docs/README.md b/docs/README.md index 3a06a92..54e50ac 100644 --- a/docs/README.md +++ b/docs/README.md @@ -10,7 +10,7 @@ | [multi-tenancy.md](architecture/multi-tenancy.md) | Logical multi-tenancy (Step 6.1): make per-tenant `rag.yaml` config drive requests. `TenantResolver` (`rag_config.tenancy`) maps a tenant id → frozen `TenantSettings` (namespace / pii_policy / acl_labels), applied **once** at the gateway boundary (namespace + pii_policy onto the `RequestContext`, acl_labels unioned into the principal); unknown tenants resolve to safe defaults (namespace = id, default PII, no labels — isolated not privileged); `RequestContext.namespace` defaults to `tenant_id` (Pinecone partitions on it; `filter_pushdown` unchanged); resolves+threads only — ACL push-down is 6.3, PII egress 6.5, physical tenancy 6.2; `GET /v1/status/tenant`; inert in `build_app` | | [policy-engine.md](architecture/policy-engine.md) | `PolicyEngine` (PDP) — single decision point for ACL, PII, quotas, redaction; replaces scattered governance checks | | [audit-log.md](architecture/audit-log.md) | Immutable audit log (Step 6.6): tamper-evidence (SHA-256 hash chain) vs immutability-at-rest (WORM export, 6.6b); one shared `AuditWriter`/store on `app.state`; read-path tenant scoping (tenant-scoped list vs whole-log verify); why the read API defaults on | -| [byok.md](architecture/byok.md) | BYOK / envelope encryption (Step 6.7): what's encrypted (chunk content at rest) vs not (embedding vectors — search needs plaintext); DEK+KEK envelope (client-side AES-GCM DEK, provider wraps the DEK); per-tenant isolation via KEK + `tenant_id` AAD; sealing as a typed error; rag-core/rag-backends split; slicing (6.7a library, 6.7b cloud KMS + wiring, 6.7c rotation) | +| [byok.md](architecture/byok.md) | BYOK / envelope encryption (Step 6.7): what's encrypted (chunk content at rest) vs not (embedding vectors — search needs plaintext); DEK+KEK envelope (client-side AES-GCM DEK, provider wraps the DEK); per-tenant isolation via KEK + `tenant_id` AAD; sealing as a typed error; rag-core/rag-backends split; slicing (6.7a library, 6.7b config + factory + AWS KMS, 6.7c GCP/Azure/Vault + rotation) | | [caching.md](architecture/caching.md) | Three-cache split: `EmbeddingCache`, `RetrievalCache`, `AnswerCache` — distinct invalidation rules | | [performance.md](architecture/performance.md) | Hot-path discipline, per-SPI p99 budgets, async telemetry, reviewer checklist | | [pipeline-batcher.md](architecture/pipeline-batcher.md) | `Pipeline` (async DAG, bounded queues, per-stage workers) + `Batcher` (DataLoader-pattern coalescing) primitives — Step 1.1d | @@ -107,7 +107,7 @@ | [admin-ui.md](reference/admin-ui.md) | Admin console (Step 3.10) — Next.js 14 operator GUI (`apps/admin-ui`); 9 pages (dashboard, corpora, connectors, glossary, webhooks, audit, API keys, tenants, config), live-vs-seed hybrid + `NEXT_PUBLIC_GATEWAY_URL`, header identity, running it, internals (shell/primitives/data layer), extension points | | [tenancy.md](reference/tenancy.md) | Logical multi-tenancy (Step 6.1) — per-tenant `rag.yaml` config (`namespace` / `acl_labels` / `pii_policy` / `quota`); `TenantResolver.resolve(id) → TenantSettings`; `RequestContext.namespace`; `GET /v1/status/tenant`; `ragctl tenant list` / `resolve`; config table + scope/boundaries (6.2/6.3/6.5) + extension points; physical tenancy (6.2), ACL push-down (6.3) + egress verifier (6.4 — `cfg.acl.verify_egress`) sections | | [audit.md](reference/audit.md) | Audit log (Step 6.6) — `AuditEvent` / `AuditStore` (append / events / verify_chain) / `NoopAuditStore` SHA-256 hash chain / `AuditWriter` (+ `.store`); read API `GET /v1/audit` (tenant-scoped, `chain_verified`) + `GET /v1/audit/verify` (whole-log); WORM signed export (6.6b) — `AuditExporter` (content_hash + HMAC), `POST /v1/audit/export`, offline `verify()`, `ragctl audit`; `cfg.audit.enabled` / `export_secret`; durable-store extension points | -| [encryption.md](reference/encryption.md) | BYOK envelope encryption (Step 6.7a) — `KeyManager` SPI (encrypt/decrypt opaque bytes) + `NoopKeyManager`; `EnvelopeKeyManager` (AES-256-GCM DEK + `tenant_id` AAD) + `LocalKeyManager` (in-process per-tenant KEK); `EncryptingStorage` decorator; `EncryptionError` / `KeyUnavailableError` (sealing); guarantees table (confidentiality / tamper / isolation / sealing); `ragctl kms`; cloud-provider extension points (6.7b) | +| [encryption.md](reference/encryption.md) | BYOK envelope encryption (Step 6.7) — `KeyManager` SPI + `NoopKeyManager`; `EnvelopeKeyManager` (AES-256-GCM DEK + `tenant_id` AAD) + `LocalKeyManager` + **`AwsKmsKeyManager`** (6.7b); `EncryptingStorage` decorator; `EncryptionError` / `KeyUnavailableError` (sealing); guarantees table; `cfg.kms` + `tenants[].kms_key_id` + `build_key_manager_from_config` factory; `ragctl kms`; GCP/Azure/Vault extension points (6.7c) | | [webhooks.md](reference/webhooks.md) | Outbound webhooks (Step 3.9) — event catalogue (`ingest.completed` / `audit.policy_violation` / `drift.detected` / `eval.regression`), event envelope, HMAC signing + `verify()`, at-least-once delivery, `/v1/webhooks/subscriptions` CRUD + test, `rag.yaml` block, `ragctl webhooks demo`, internals + extension points | | [integrations.md](reference/integrations.md) | Framework adapters (Step 3.8) — `agentcontextos.integrations.*` for LangChain / LlamaIndex / Haystack / DSPy / LangGraph / CrewAI / AutoGen / Semantic Kernel; per-framework extras, shared config + chunk metadata, usage per framework, internals + extension points | | [status-api.md](reference/status-api.md) | Status & Metrics API (Step 3.11) — `/v1/status/health` / `metrics` / `logs` (+ SSE `logs/stream`), `WS /v1/status/ws`, `/v1/connectors/status`; metric catalogue + request-timing middleware, the `MetricsCollector` / `LogTail` read-side, CORS + query-param identity for browser streams, extension points | diff --git a/docs/adr/ADR-0039-byok-envelope-encryption.md b/docs/adr/ADR-0039-byok-envelope-encryption.md index d518e44..4a2957b 100644 --- a/docs/adr/ADR-0039-byok-envelope-encryption.md +++ b/docs/adr/ADR-0039-byok-envelope-encryption.md @@ -74,9 +74,13 @@ where the real provider impls live. rag-core stays crypto-free (SPI + decorator - Data is decrypted server-side for use, so plaintext exists in memory during a request — BYOK protects data **at rest** + gates access via the KEK, not against a compromised running server. -- **6.7b** (cloud KMS providers behind `[kms-*]` extras + wiring `EncryptingStorage` - into the ingest/storage path + per-tenant key config) and **6.7c** (zero-downtime - rotation, decrypt-only retention of old keys) are deferred. +- **6.7b** added the per-tenant key config (`cfg.kms` + `tenants[].kms_key_id`), a + `build_key_manager_from_config` factory (`noop` / `local` / `aws`), the + `app.state.key_manager` gateway seam, and the **AWS KMS** provider + (`AwsKmsKeyManager` over aioboto3, per-tenant CMK, KMS-error → sealing, + fake-client unit-tested). **GCP KMS / Azure Key Vault / HashiCorp Vault** + providers, wiring `EncryptingStorage` into the ingest/storage path (blocked on + tiered-storage plumbing), and **6.7c** zero-downtime rotation remain deferred. ## See also - [architecture/byok.md](../architecture/byok.md) — envelope design, AAD, sealing, slicing diff --git a/docs/architecture/byok.md b/docs/architecture/byok.md index 5ed1da7..f0b4055 100644 --- a/docs/architecture/byok.md +++ b/docs/architecture/byok.md @@ -74,12 +74,18 @@ asserts its reads fail while another tenant's succeed. ## Slicing -- **6.7a** (this) — the encryption library + `LocalKeyManager` + `EncryptingStorage` - + full crypto tests + `ragctl kms`. A complete, demonstrable BYOK foundation. -- **6.7b** — cloud KMS providers behind extras; wire `EncryptingStorage` into the - ingest/storage path; per-tenant key config (`cfg.kms` + `TenantConfig` key ref). -- **6.7c** — zero-downtime key rotation; old keys retained decrypt-only until - expiry; background re-encryption. +- **6.7a** — the encryption library + `LocalKeyManager` + `EncryptingStorage` + + full crypto tests + `ragctl kms`. A complete, demonstrable BYOK foundation. +- **6.7b** — per-tenant key config (`cfg.kms` + `tenants[].kms_key_id`), the + `build_key_manager_from_config` factory (`noop` / `local` / `aws`), the + `app.state.key_manager` gateway seam, and the **AWS KMS** provider + (`AwsKmsKeyManager`). The factory lives in the gateway (it maps the rag-config + type → a rag-backends provider via lazy imports, keeping rag-backends + config-free). Wiring `EncryptingStorage` into the ingest path is deferred — + chunk content is inline today; it lands with tiered storage (ADR-0007). +- **6.7c** — GCP KMS / Azure Key Vault / HashiCorp Vault providers (behind + `[kms-*]` extras, same `EnvelopeKeyManager` pattern) + zero-downtime key + rotation (old keys retained decrypt-only until expiry; background re-encryption). ## Boundary note diff --git a/docs/reference/encryption.md b/docs/reference/encryption.md index bc5f646..f92ec66 100644 --- a/docs/reference/encryption.md +++ b/docs/reference/encryption.md @@ -55,6 +55,33 @@ await store.get(ctx, "doc/1") # → b"secret" ragctl kms --tenant acme --text "secret" # round-trip + isolation + sealing demo ``` +### Configuration + providers (Step 6.7b) + +Select a provider from `rag.yaml`; the gateway builds it +(`build_key_manager_from_config`) and exposes it on `app.state.key_manager`. + +```yaml +kms: + enabled: true + provider: aws # noop | local | aws (gcp / azure / vault → 6.7c) + default_key_id: "" # fallback key reference for tenants without their own + region: us-east-1 # AWS + local_key: ${LOCAL_KEK} # hex 32-byte default KEK for provider=local +tenants: + - id: acme + name: Acme + kms_key_id: arn:aws:kms:us-east-1:111122223333:key/abcd… # aws CMK + - id: beta + name: Beta + kms_key_id: ${BETA_KEK_HEX} # provider=local → hex 32-byte KEK +``` + +- **`AwsKmsKeyManager`** (`rag-backends`, over the already-present `aioboto3`) + wraps/unwraps the DEK via AWS KMS Encrypt/Decrypt for the tenant's CMK; any KMS + failure (revoked / denied / unreachable) → `KeyUnavailableError` (sealing). +- `tenants[].kms_key_id` is the per-tenant key (a KMS ARN for `aws`, a hex KEK for + `local`); a tenant with neither its own key nor `default_key_id` is sealed. + ## Internals - **Envelope wire format:** `b"RAGK" | version | u32(len(wrapped_dek)) | diff --git a/packages/backends/src/rag_backends/__init__.py b/packages/backends/src/rag_backends/__init__.py index 7b0a579..b815e14 100644 --- a/packages/backends/src/rag_backends/__init__.py +++ b/packages/backends/src/rag_backends/__init__.py @@ -11,6 +11,7 @@ from rag_backends.cache.redis import RedisCache from rag_backends.cache.redis_typed import RedisAnswerCache, RedisRetrievalCache from rag_backends.corpus.pg_corpus_store import PgCorpusStore +from rag_backends.kms.aws import AwsKmsKeyManager from rag_backends.kms.local import LocalKeyManager from rag_backends.quota.redis import RedisQuotaStore from rag_backends.storage.local import LocalFileStorage @@ -19,6 +20,7 @@ from rag_backends.vector.qdrant import QdrantVectorStore __all__ = [ + "AwsKmsKeyManager", "LocalKeyManager", "PgCorpusStore", "PgVectorStore", diff --git a/packages/backends/src/rag_backends/kms/__init__.py b/packages/backends/src/rag_backends/kms/__init__.py index 42a0505..37817c0 100644 --- a/packages/backends/src/rag_backends/kms/__init__.py +++ b/packages/backends/src/rag_backends/kms/__init__.py @@ -2,11 +2,13 @@ ``EnvelopeKeyManager`` is the client-side envelope-encryption base; ``LocalKeyManager`` wraps the data key with an in-process per-tenant KEK (dev / -tests / air-gapped). Cloud providers (AWS KMS / GCP KMS / Azure Key Vault / -HashiCorp Vault) land in Step 6.7b behind ``[kms-*]`` extras. +tests / air-gapped); ``AwsKmsKeyManager`` (Step 6.7b) wraps it via AWS KMS. +GCP KMS / Azure Key Vault / HashiCorp Vault providers land in Step 6.7c behind +``[kms-*]`` extras. """ +from rag_backends.kms.aws import AwsKmsKeyManager from rag_backends.kms.envelope import EnvelopeKeyManager from rag_backends.kms.local import LocalKeyManager -__all__ = ["EnvelopeKeyManager", "LocalKeyManager"] +__all__ = ["AwsKmsKeyManager", "EnvelopeKeyManager", "LocalKeyManager"] diff --git a/packages/backends/src/rag_backends/kms/aws.py b/packages/backends/src/rag_backends/kms/aws.py new file mode 100644 index 0000000..33e54e4 --- /dev/null +++ b/packages/backends/src/rag_backends/kms/aws.py @@ -0,0 +1,106 @@ +"""AwsKmsKeyManager — AWS KMS as the BYOK key-encryption key (Step 6.7b). + +Subclasses :class:`~rag_backends.kms.envelope.EnvelopeKeyManager`: the data-key +half (per-payload AES-256-GCM DEK + tenant AAD + envelope framing) is inherited; +this class only **wraps / unwraps the DEK** by calling **AWS KMS** Encrypt / +Decrypt for the tenant's customer-managed key (CMK). The KEK never leaves KMS — +only the small DEK is sent to be wrapped — so a customer controls (and can +revoke) access to their key independently of the platform. + +Per-tenant: the CMK is resolved from a ``tenant → key-id`` map (the ARNs / +key-ids from ``tenants[].kms_key_id``), with an optional shared default. A +tenant with no key is **sealed** (:class:`~rag_core.errors.KeyUnavailableError`); +any KMS failure (revoked key, denied access, unreachable) maps to the same — by +design that tenant's data becomes unreadable, with no cross-tenant impact. + +Uses the async ``aioboto3`` KMS client (already a ``rag-backends`` dependency). +The ``client`` seam lets tests inject a fake KMS without AWS credentials. +""" + +from __future__ import annotations + +from collections.abc import AsyncIterator, Mapping +from contextlib import asynccontextmanager +from typing import Any + +from rag_core.errors import KeyUnavailableError +from rag_core.types import RequestContext + +from rag_backends.kms.envelope import EnvelopeKeyManager + +__all__ = ["AwsKmsKeyManager"] + + +class AwsKmsKeyManager(EnvelopeKeyManager): + """Envelope encryption with per-tenant AWS KMS customer-managed keys.""" + + def __init__( + self, + *, + key_ids: Mapping[str, str] | None = None, + default_key_id: str | None = None, + region: str = "us-east-1", + endpoint_url: str | None = None, + client: Any | None = None, + ) -> None: + self._key_ids = dict(key_ids or {}) + self._default = default_key_id + self._endpoint = endpoint_url + self._client = client # injected (tests) bypasses the aioboto3 session + self._session: Any | None = None + if client is None: + import aioboto3 # lazy — only when a real client is needed + + self._session = aioboto3.Session(region_name=region) + + def _key_id_for(self, ctx: RequestContext) -> str: + key_id = self._key_ids.get(str(ctx.tenant_id), self._default) + if not key_id: + raise KeyUnavailableError( + f"no KMS key id configured for tenant {ctx.tenant_id!r} (sealed)", + tenant_id=str(ctx.tenant_id), + ) + return key_id + + @asynccontextmanager + async def _kms(self) -> AsyncIterator[Any]: + if self._client is not None: + yield self._client + return + kwargs: dict[str, Any] = {} + if self._endpoint: + kwargs["endpoint_url"] = self._endpoint + assert self._session is not None # noqa: S101 — set whenever client is None + async with self._session.client("kms", **kwargs) as client: + yield client + + async def _wrap_dek(self, ctx: RequestContext, dek: bytes) -> bytes: + key_id = self._key_id_for(ctx) + try: + async with self._kms() as kms: + resp = await kms.encrypt(KeyId=key_id, Plaintext=dek) + return bytes(resp["CiphertextBlob"]) + except KeyUnavailableError: + raise + except Exception as exc: # noqa: BLE001 — any KMS failure seals this tenant + raise KeyUnavailableError( + f"AWS KMS wrap failed for tenant {ctx.tenant_id!r}: {exc}", + tenant_id=str(ctx.tenant_id), + ) from exc + + async def _unwrap_dek(self, ctx: RequestContext, wrapped: bytes) -> bytes: + key_id = self._key_id_for(ctx) + try: + async with self._kms() as kms: + resp = await kms.decrypt(CiphertextBlob=wrapped, KeyId=key_id) + return bytes(resp["Plaintext"]) + except KeyUnavailableError: + raise + except Exception as exc: # noqa: BLE001 — revoked / denied / unreachable → sealed + raise KeyUnavailableError( + f"AWS KMS unwrap failed for tenant {ctx.tenant_id!r}: {exc}", + tenant_id=str(ctx.tenant_id), + ) from exc + + async def health(self) -> bool: + return True diff --git a/packages/config/src/rag_config/schema.py b/packages/config/src/rag_config/schema.py index 539a04b..9332316 100644 --- a/packages/config/src/rag_config/schema.py +++ b/packages/config/src/rag_config/schema.py @@ -339,6 +339,12 @@ class TenantConfig(_StrictBase): # ``dedicated_index_name`` when set, else the tenant ``id``. dedicated_index: bool = False dedicated_index_name: Annotated[str, Field(min_length=1)] | None = None + # BYOK (Step 6.7): this tenant's key reference, interpreted by the configured + # KMS provider — a KMS key ARN/resource name for ``aws``/cloud, or a hex-encoded + # 32-byte KEK for the ``local`` provider. ``None`` → the tenant is *sealed* + # (encrypt/decrypt raise ``KeyUnavailableError``) unless ``kms.default_key_id`` + # is set. Supports ``${ENV_VAR}`` interpolation. + kms_key_id: str | None = None # --------------------------------------------------------------------------- @@ -609,6 +615,42 @@ class AuditConfig(_StrictBase): export_secret: str = "" +class KmsProvider(StrEnum): + """BYOK key-manager provider (Step 6.7).""" + + NOOP = "noop" # passthrough, no encryption (dev) + LOCAL = "local" # in-process per-tenant KEK (LocalKeyManager) + AWS = "aws" # AWS KMS (Step 6.7b) + + +class KmsConfig(_StrictBase): + """BYOK envelope encryption (Step 6.7). + + When ``enabled`` the gateway builds a ``KeyManager`` (selected by ``provider``) + and exposes it on ``app.state.key_manager``; wrap a ``Storage`` backend with + ``EncryptingStorage`` to encrypt chunk content / blobs **at rest** with + per-tenant, customer-controlled keys (embedding vectors stay plaintext for + search). Each tenant's key reference is ``tenants[].kms_key_id`` (a KMS key + ARN for ``aws``, a hex 32-byte KEK for ``local``); a tenant with no key — + and no ``default_key_id`` — is **sealed** (``KeyUnavailableError``). + + **Disabled by default.** ``provider``: ``local`` (in-process KEK, dev / + air-gapped), ``aws`` (AWS KMS), or ``noop`` (passthrough). GCP / Azure / + Vault providers + key rotation land in Step 6.7c. + + * ``default_key_id`` — fallback key reference for tenants without their own. + * ``region`` — AWS region for the ``aws`` provider. + * ``local_key`` — hex 32-byte default KEK for the ``local`` provider + (``${ENV_VAR}``-interpolated; keep it out of the file). + """ + + enabled: bool = False + provider: KmsProvider = KmsProvider.LOCAL + default_key_id: str = "" + region: str = "us-east-1" + local_key: str = "" + + class QuotaConfig(_StrictBase): """Per-tenant quota & rate-limit enforcement knobs (Step 4.5). @@ -922,6 +964,7 @@ class RagConfig(_StrictBase): acl: AclConfig = Field(default_factory=AclConfig) pii: PiiConfig = Field(default_factory=PiiConfig) audit: AuditConfig = Field(default_factory=AuditConfig) + kms: KmsConfig = Field(default_factory=KmsConfig) webhooks: WebhooksConfig = Field(default_factory=WebhooksConfig) provenance: ProvenanceConfig = Field(default_factory=ProvenanceConfig) feedback: FeedbackConfig = Field(default_factory=FeedbackConfig) diff --git a/tests/kms/test_aws.py b/tests/kms/test_aws.py new file mode 100644 index 0000000..de2c67d --- /dev/null +++ b/tests/kms/test_aws.py @@ -0,0 +1,105 @@ +"""AwsKmsKeyManager tests with an injected fake KMS client (Step 6.7b). + +The envelope crypto is covered by ``test_envelope.py`` (LocalKeyManager); here we +verify the AWS provider's wrap/unwrap glue: per-tenant key-id resolution, the +KMS-bound DEK, sealing on a missing key, and KMS-error → KeyUnavailableError. +No AWS credentials / network — a fake KMS client stands in for aioboto3. +""" + +from __future__ import annotations + +import os +from typing import Any + +import pytest +from rag_backends import AwsKmsKeyManager +from rag_core.errors import KeyUnavailableError +from rag_core.types import ( + Principal, + PrincipalId, + PrincipalKind, + RequestContext, + TenantId, +) + +_PLAINTEXT = b"secret chunk text" + + +def _ctx(tenant: str) -> RequestContext: + tid = TenantId(tenant) + return RequestContext( + tenant_id=tid, + principal=Principal( + id=PrincipalId("p"), kind=PrincipalKind.user, display_name="p", tenant_id=tid + ), + ) + + +class _FakeKms: + """In-memory stand-in for the aioboto3 KMS client. + + Mirrors real KMS: the ciphertext blob is opaque and **bound to the key** used + to produce it, so decrypting under a different KeyId fails. + """ + + def __init__(self) -> None: + self._store: dict[bytes, tuple[str, bytes]] = {} + self.fail = False + + async def encrypt(self, *, KeyId: str, Plaintext: bytes) -> dict[str, Any]: # noqa: N803 + if self.fail: + raise RuntimeError("kms unavailable") + token = os.urandom(16) + self._store[token] = (KeyId, Plaintext) + return {"CiphertextBlob": token} + + async def decrypt(self, *, CiphertextBlob: bytes, KeyId: str) -> dict[str, Any]: # noqa: N803 + if self.fail: + raise RuntimeError("kms unavailable") + key_id, plaintext = self._store[CiphertextBlob] # KeyError if unknown blob + if key_id != KeyId: + raise RuntimeError("key mismatch") # KMS rejects a wrong-key decrypt + return {"Plaintext": plaintext} + + +def _km(client: _FakeKms | None = None) -> AwsKmsKeyManager: + return AwsKmsKeyManager( + key_ids={"acme": "arn:aws:kms:::key/acme", "globex": "arn:aws:kms:::key/globex"}, + client=client or _FakeKms(), + ) + + +async def test_round_trip() -> None: + km = _km() + ct = await km.encrypt(_ctx("acme"), _PLAINTEXT) + assert ct != _PLAINTEXT + assert await km.decrypt(_ctx("acme"), ct) == _PLAINTEXT + + +async def test_dek_is_kms_key_bound_cross_tenant_blocked() -> None: + km = _km() # acme + globex share one fake KMS but distinct key ids + ct = await km.encrypt(_ctx("acme"), _PLAINTEXT) + with pytest.raises(KeyUnavailableError): + await km.decrypt(_ctx("globex"), ct) + + +async def test_unknown_tenant_is_sealed() -> None: + with pytest.raises(KeyUnavailableError): + await _km().encrypt(_ctx("ghost"), _PLAINTEXT) + + +async def test_default_key_id_used_when_tenant_has_none() -> None: + km = AwsKmsKeyManager(default_key_id="arn:aws:kms:::key/shared", client=_FakeKms()) + ct = await km.encrypt(_ctx("whoever"), _PLAINTEXT) + assert await km.decrypt(_ctx("whoever"), ct) == _PLAINTEXT + + +async def test_kms_failure_maps_to_key_unavailable() -> None: + fake = _FakeKms() + fake.fail = True + with pytest.raises(KeyUnavailableError): + await _km(fake).encrypt(_ctx("acme"), _PLAINTEXT) + + +async def test_health() -> None: + assert await _km().health() is True