feat(crypto): GCP / Azure / Vault KMS providers (Step 6.7c)

TRACKER.md

@@ -16,10 +16,11 @@
 | **Last updated** | 2026-06-08 |
 | **Current phase** | Phase 6 — Governance & Tenancy (**6 / 10 steps**) |
 | **Overall** | **70 / 84 steps** — Phases 0–5 complete |
-| **Next action** | **Step 6.7c — GCP/Azure/Vault KMS + key rotation**: the remaining cloud providers (GCP KMS / Azure Key Vault / HashiCorp Vault, same `EnvelopeKeyManager` pattern, behind `[kms-*]` extras) + zero-downtime key rotation (old keys retained decrypt-only until expiry). Closes Step 6.7. 6.7a (library + local) + 6.7b (config/factory/wiring + AWS KMS) shipped. |
+| **Next action** | **Step 6.7d — Key rotation**: zero-downtime KEK rotation — old keys retained decrypt-only until expiry, background re-encryption. Closes Step 6.7. 6.7a (library + local) + 6.7b (config/factory + AWS KMS) + 6.7c (GCP/Azure/Vault providers) shipped. |
 
 **Recently shipped**
 
+- **6.7c** 🚧 GCP/Azure/Vault KMS providers — `GcpKmsKeyManager` (google-cloud-kms), `AzureKeyVaultKeyManager` (`wrap_key`/`unwrap_key`), `VaultKeyManager` (Vault Transit) — each subclasses `EnvelopeKeyManager` behind a `[kms-*]` extra (lazy SDK, injectable client, fake-client unit-tested); factory + enum extended; completes all four KMS providers — [#157](https://github.com/officialCodeWork/AgentContextOS/pull/157)
 - **6.7b** 🚧 Cloud KMS (AWS) + config/factory/wiring — `cfg.kms` (provider `noop`/`local`/`aws`) + `tenants[].kms_key_id`; `build_key_manager_from_config` factory + `app.state.key_manager` gateway seam; **`AwsKmsKeyManager`** (aioboto3, per-tenant CMK, KMS-error → sealing, fake-client unit-tested) — [#156](https://github.com/officialCodeWork/AgentContextOS/pull/156)
 - **6.7a** 🚧 BYOK envelope encryption library — `KeyManager` SPI + `NoopKeyManager` + `EncryptingStorage` (rag-core); `EnvelopeKeyManager` (AES-256-GCM DEK + `tenant_id` AAD) + `LocalKeyManager` (in-process per-tenant KEK) (rag-backends); per-tenant isolation + **sealing** (`KeyUnavailableError`) + tamper detection; `ragctl kms` — [#155](https://github.com/officialCodeWork/AgentContextOS/pull/155)
 - **6.6** ✅ Immutable audit log — **6.6a** shared `AuditWriter`/store on `app.state` + `GET /v1/audit` (tenant-scoped, `chain_verified`) + `GET /v1/audit/verify` (whole-log) + `cfg.audit.enabled` ([#153](https://github.com/officialCodeWork/AgentContextOS/pull/153)); **6.6b** `AuditExporter` self-verifying WORM bundle (SHA-256 `content_hash` + HMAC, offline `verify()`), `POST /v1/audit/export` (tenant-scoped) + `ragctl audit`, `cfg.audit.export_secret` ([#154](https://github.com/officialCodeWork/AgentContextOS/pull/154))
@@ -657,7 +658,7 @@
 | 6.4 | ACL egress verifier | ✅ | [#151](https://github.com/officialCodeWork/AgentContextOS/pull/151) — `AclEgressVerifier` re-checks returned chunks at the gateway router boundary (defense-in-depth above the 6.3 push-down); `acl.egress_violation`; zero-violation-rate red-team gate |
 | 6.5 | PII policies | ✅ | [#152](https://github.com/officialCodeWork/AgentContextOS/pull/152) — `PiiPolicyEngine` egress_text decorator (allow / redact / mask / block per tenant) over answer + context; reuses Step 1.7 detector; `pii.egress_blocked` |
 | 6.6 | Immutable audit log | ✅ | **6.6a** [#153](https://github.com/officialCodeWork/AgentContextOS/pull/153) — read API `GET /v1/audit` + `GET /v1/audit/verify` + shared store + `cfg.audit`.  **6.6b** [#154](https://github.com/officialCodeWork/AgentContextOS/pull/154) — `AuditExporter` signed WORM bundle + `POST /v1/audit/export` + `ragctl audit` |
-| 6.7 | BYOK (Bring Your Own Key) | 🚧 | **6.7a** ✅ [#155](https://github.com/officialCodeWork/AgentContextOS/pull/155) — `KeyManager` SPI + envelope encryption (`LocalKeyManager`) + `EncryptingStorage` + sealing/isolation/tamper.  **6.7b** ✅ [#156](https://github.com/officialCodeWork/AgentContextOS/pull/156) — `cfg.kms` + per-tenant key + `build_key_manager_from_config` factory + `AwsKmsKeyManager`.  **6.7c** ⏳ — GCP/Azure/Vault + key rotation |
+| 6.7 | BYOK (Bring Your Own Key) | 🚧 | **6.7a** ✅ [#155](https://github.com/officialCodeWork/AgentContextOS/pull/155) — `KeyManager` SPI + envelope encryption (`LocalKeyManager`) + `EncryptingStorage` + sealing/isolation/tamper.  **6.7b** ✅ [#156](https://github.com/officialCodeWork/AgentContextOS/pull/156) — `cfg.kms` + per-tenant key + `build_key_manager_from_config` factory + `AwsKmsKeyManager`.  **6.7c** ✅ [#157](https://github.com/officialCodeWork/AgentContextOS/pull/157) — `GcpKmsKeyManager` / `AzureKeyVaultKeyManager` / `VaultKeyManager` behind `[kms-*]` extras.  **6.7d** ⏳ — key rotation |
 | 6.8 | SSO / SCIM | ⏳ | OIDC + SAML IdP federation; SCIM 2.0 user provisioning; per-tenant IdP config |
 | 6.9 | Air-gapped install bundle | ⏳ | Signed tarball with all images + Helm chart; offline bootstrap; cosign verification |
 | 6.10 | Compliance posture | ⏳ | SOC 2 Type II control mapping; GDPR data-residency config; data-retention policies |
@@ -747,6 +748,12 @@ New ground — the only prior crypto was HMAC signing.  The V1 plan calls for en
 - **`AwsKmsKeyManager`** (`rag-backends/kms/aws.py`) subclasses `EnvelopeKeyManager` and wraps/unwraps the DEK via **AWS KMS** Encrypt/Decrypt for the tenant's customer-managed key (over the already-present `aioboto3` — **no new dep**).  Per-tenant CMK from the key-id map (+ `default_key_id`); any KMS failure (revoked / denied / unreachable) or a missing key id → **`KeyUnavailableError`** (sealing).  An **injectable `client` seam** makes it fully unit-testable with a fake KMS — no AWS creds / network
 - **Scope:** AWS provider + config + factory + the `app.state.key_manager` seam.  **Deferred:** GCP KMS / Azure Key Vault / HashiCorp Vault providers + key rotation (6.7c); wiring `EncryptingStorage` into the ingest path (blocked on tiered-storage plumbing — chunk content is inline today).  ~14 new tests (AWS provider over a fake KMS: round-trip / KMS-key-bound cross-tenant block / sealing / default-key / KMS-error mapping; config defaults; factory noop/local/aws; gateway `app.state.key_manager` + `kms_enabled` + a local round-trip).  `KmsConfig` → `rag.schema` regenerated; all gates green (ruff, mypy --strict 306 files, RAG001, policy-coverage, log-schema).  [ADR-0039](docs/adr/ADR-0039-byok-envelope-encryption.md), [reference/encryption.md](docs/reference/encryption.md), [architecture/byok.md](docs/architecture/byok.md)
 
+#### 6.7c — GCP / Azure / Vault KMS providers ✅ [#157](https://github.com/officialCodeWork/AgentContextOS/pull/157)
+
+- Completes the **four-provider KMS matrix** — each subclasses `EnvelopeKeyManager` (inheriting the DEK + AES-GCM + tenant-AAD + envelope framing) and only wraps/unwraps the DEK via its KMS API: **`GcpKmsKeyManager`** (google-cloud-kms `encrypt`/`decrypt`), **`AzureKeyVaultKeyManager`** (azure-keyvault-keys `wrap_key`/`unwrap_key`, RSA-OAEP-256; a `CryptographyClient` per key via a cached factory since Azure binds a client to one key), **`VaultKeyManager`** (HashiCorp Vault Transit `encrypt_data`/`decrypt_data`, sync `hvac` run in a thread, `mount_point`)
+- Each is **behind a `[kms-gcp]` / `[kms-azure]` / `[kms-vault]` extra** (the SDKs are lazy-imported, so the modules import without the SDK and selecting a provider without its extra raises a clear `ImportError`); an **injectable `client` / `client_factory` seam** makes every provider fully unit-testable with a fake KMS — no cloud creds / network.  Connection uses each SDK's standard credential discovery (GCP ADC, Azure `DefaultAzureCredential`, Vault `VAULT_ADDR`/`VAULT_TOKEN`); per-tenant key resolution + sealing (`KeyUnavailableError`) are uniform with AWS
+- `KmsProvider` enum + `build_key_manager_from_config` factory + `KmsConfig.vault_mount` extended.  mypy overrides added for the three SDK module trees (absent at lint time).  **Scope:** providers only — key rotation is **6.7d**; wiring `EncryptingStorage` into the ingest path stays deferred (tiered storage).  ~18 new tests (each provider over a fake client: round-trip / cross-tenant block / sealing / SDK-error mapping; factory cloud-provider-requires-extra contract).  `KmsConfig` → `rag.schema` regenerated; all gates green (ruff, mypy --strict 309 files, RAG001, policy-coverage, log-schema).  [ADR-0039](docs/adr/ADR-0039-byok-envelope-encryption.md), [reference/encryption.md](docs/reference/encryption.md), [architecture/byok.md](docs/architecture/byok.md)
+
 ---
 
 ## Phase 7 — Pilot, Harden, GA (Weeks 34–40) ⏳
@@ -907,6 +914,7 @@ Complete log of every PR. Routine Dependabot bumps are grouped; everything else
 | [#154](https://github.com/officialCodeWork/AgentContextOS/pull/154) | 2026-06-08 | feat(audit): WORM signed export — AuditExporter + POST /v1/audit/export (Step 6.6b) |
 | [#155](https://github.com/officialCodeWork/AgentContextOS/pull/155) | 2026-06-08 | feat(crypto): BYOK envelope encryption library + local KMS (Step 6.7a) |
 | [#156](https://github.com/officialCodeWork/AgentContextOS/pull/156) | 2026-06-08 | feat(crypto): cfg.kms + key-manager factory + AWS KMS provider (Step 6.7b) |
+| [#157](https://github.com/officialCodeWork/AgentContextOS/pull/157) | 2026-06-08 | feat(crypto): GCP / Azure / Vault KMS providers (Step 6.7c) |
 | #78–#80, #116–#118 | Open | Dependabot bumps — awaiting merge |
 | #81 | Closed | Dependabot bump — superseded |
 

apps/gateway/src/rag_gateway/wiring.py

@@ -157,10 +157,11 @@ def build_key_manager_from_config(cfg: RagConfig) -> Any:
 
     Returns a passthrough ``NoopKeyManager`` when disabled / ``provider=noop``; a
     ``LocalKeyManager`` (in-process per-tenant KEKs decoded from
-    ``tenants[].kms_key_id`` hex) for ``local``; an ``AwsKmsKeyManager`` (per-tenant
-    KMS key ARNs) for ``aws``.  GCP / Azure / Vault land in Step 6.7c.  Each
-    tenant's key is ``tenants[].kms_key_id`` with ``kms.default_key_id`` as the
-    fallback; a tenant with neither is *sealed* at encrypt/decrypt time.
+    ``tenants[].kms_key_id`` hex) for ``local``; or a cloud KMS provider —
+    ``AwsKmsKeyManager`` / ``GcpKmsKeyManager`` / ``AzureKeyVaultKeyManager`` /
+    ``VaultKeyManager`` — keyed by ``tenants[].kms_key_id`` (an ARN / resource name
+    / key URL / transit key name).  Each tenant's key has ``kms.default_key_id`` as
+    the fallback; a tenant with neither is *sealed* at encrypt/decrypt time.
     """
     from rag_config.schema import KmsProvider
     from rag_core.spi.noop import NoopKeyManager
@@ -176,12 +177,32 @@ def build_key_manager_from_config(cfg: RagConfig) -> Any:
         default = bytes.fromhex(kms.local_key) if kms.local_key else None
         return LocalKeyManager(keks=keks, default_kek=default)
 
+    # Cloud providers share the per-tenant key-id map (ARN / resource name / key
+    # URL / transit key) + default.  Connection comes from each SDK's standard
+    # credential discovery; the provider modules import their SDK lazily.
+    key_ids = {t.id: t.kms_key_id for t in cfg.tenants if t.kms_key_id}
+    default_key_id = kms.default_key_id or None
+
     if kms.provider is KmsProvider.AWS:
         from rag_backends import AwsKmsKeyManager
 
-        key_ids = {t.id: t.kms_key_id for t in cfg.tenants if t.kms_key_id}
-        return AwsKmsKeyManager(
-            key_ids=key_ids, default_key_id=kms.default_key_id or None, region=kms.region
+        return AwsKmsKeyManager(key_ids=key_ids, default_key_id=default_key_id, region=kms.region)
+
+    if kms.provider is KmsProvider.GCP:
+        from rag_backends import GcpKmsKeyManager
+
+        return GcpKmsKeyManager(key_ids=key_ids, default_key_id=default_key_id)
+
+    if kms.provider is KmsProvider.AZURE:
+        from rag_backends import AzureKeyVaultKeyManager
+
+        return AzureKeyVaultKeyManager(key_ids=key_ids, default_key_id=default_key_id)
+
+    if kms.provider is KmsProvider.VAULT:
+        from rag_backends import VaultKeyManager
+
+        return VaultKeyManager(
+            key_ids=key_ids, default_key_id=default_key_id, mount_point=kms.vault_mount
         )
 
     return NoopKeyManager()  # pragma: no cover - providers above are exhaustive

apps/gateway/tests/test_kms.py

@@ -45,6 +45,23 @@ def test_factory_aws() -> None:
     assert isinstance(build_key_manager_from_config(cfg), AwsKmsKeyManager)
 
 
+def test_factory_gcp_requires_extra() -> None:
+    # Selecting a cloud provider whose SDK extra isn't installed raises a clear
+    # ImportError (in CI the [kms-gcp] extra is absent).
+    try:
+        import pytest
+        from google.cloud import kms_v1  # noqa: F401
+
+        pytest.skip("kms-gcp extra installed")
+    except ImportError:
+        pass
+    cfg = RagConfig(kms=KmsConfig(enabled=True, provider=KmsProvider.GCP, default_key_id="k"))
+    import pytest
+
+    with pytest.raises(ImportError):
+        build_key_manager_from_config(cfg)
+
+
 # ---------------------------------------------------------------------------
 # gateway wiring
 # ---------------------------------------------------------------------------

dist/rag.schema.json

@@ -892,7 +892,7 @@
     },
     "KmsConfig": {
       "additionalProperties": false,
-      "description": "BYOK envelope encryption (Step 6.7).\n\nWhen ``enabled`` the gateway builds a ``KeyManager`` (selected by ``provider``)\nand exposes it on ``app.state.key_manager``; wrap a ``Storage`` backend with\n``EncryptingStorage`` to encrypt chunk content / blobs **at rest** with\nper-tenant, customer-controlled keys (embedding vectors stay plaintext for\nsearch).  Each tenant's key reference is ``tenants[].kms_key_id`` (a KMS key\nARN for ``aws``, a hex 32-byte KEK for ``local``); a tenant with no key \u2014\nand no ``default_key_id`` \u2014 is **sealed** (``KeyUnavailableError``).\n\n**Disabled by default.**  ``provider``: ``local`` (in-process KEK, dev /\nair-gapped), ``aws`` (AWS KMS), or ``noop`` (passthrough).  GCP / Azure /\nVault providers + key rotation land in Step 6.7c.\n\n* ``default_key_id`` \u2014 fallback key reference for tenants without their own.\n* ``region`` \u2014 AWS region for the ``aws`` provider.\n* ``local_key`` \u2014 hex 32-byte default KEK for the ``local`` provider\n  (``${ENV_VAR}``-interpolated; keep it out of the file).",
+      "description": "BYOK envelope encryption (Step 6.7).\n\nWhen ``enabled`` the gateway builds a ``KeyManager`` (selected by ``provider``)\nand exposes it on ``app.state.key_manager``; wrap a ``Storage`` backend with\n``EncryptingStorage`` to encrypt chunk content / blobs **at rest** with\nper-tenant, customer-controlled keys (embedding vectors stay plaintext for\nsearch).  Each tenant's key reference is ``tenants[].kms_key_id`` (a KMS key\nARN for ``aws``, a hex 32-byte KEK for ``local``); a tenant with no key \u2014\nand no ``default_key_id`` \u2014 is **sealed** (``KeyUnavailableError``).\n\n**Disabled by default.**  ``provider``: ``local`` (in-process KEK, dev /\nair-gapped), ``aws`` / ``gcp`` / ``azure`` / ``vault`` (customer-controlled\nKMS), or ``noop`` (passthrough).  Cloud-provider connection uses standard\ncredential discovery (AWS chain, GCP ADC, Azure ``DefaultAzureCredential``,\nVault ``VAULT_ADDR`` / ``VAULT_TOKEN``); the per-tenant key reference is\n``tenants[].kms_key_id``.  Key rotation lands in Step 6.7d.\n\n* ``default_key_id`` \u2014 fallback key reference for tenants without their own.\n* ``region`` \u2014 AWS region for the ``aws`` provider.\n* ``local_key`` \u2014 hex 32-byte default KEK for the ``local`` provider\n  (``${ENV_VAR}``-interpolated; keep it out of the file).\n* ``vault_mount`` \u2014 Transit secrets-engine mount path for the ``vault`` provider.",
       "properties": {
         "enabled": {
           "default": false,
@@ -917,6 +917,11 @@
           "default": "",
           "title": "Local Key",
           "type": "string"
+        },
+        "vault_mount": {
+          "default": "transit",
+          "title": "Vault Mount",
+          "type": "string"
         }
       },
       "title": "KmsConfig",
@@ -927,7 +932,10 @@
       "enum": [
         "noop",
         "local",
-        "aws"
+        "aws",
+        "gcp",
+        "azure",
+        "vault"
       ],
       "title": "KmsProvider",
       "type": "string"

dist/rag.schema.yaml

@@ -931,12 +931,15 @@ $defs:
       \ (a KMS key\nARN for ``aws``, a hex 32-byte KEK for ``local``); a tenant with\
       \ no key —\nand no ``default_key_id`` — is **sealed** (``KeyUnavailableError``).\n\
       \n**Disabled by default.**  ``provider``: ``local`` (in-process KEK, dev /\n\
-      air-gapped), ``aws`` (AWS KMS), or ``noop`` (passthrough).  GCP / Azure /\n\
-      Vault providers + key rotation land in Step 6.7c.\n\n* ``default_key_id`` —\
-      \ fallback key reference for tenants without their own.\n* ``region`` — AWS\
-      \ region for the ``aws`` provider.\n* ``local_key`` — hex 32-byte default KEK\
-      \ for the ``local`` provider\n  (``${ENV_VAR}``-interpolated; keep it out of\
-      \ the file)."
+      air-gapped), ``aws`` / ``gcp`` / ``azure`` / ``vault`` (customer-controlled\n\
+      KMS), or ``noop`` (passthrough).  Cloud-provider connection uses standard\n\
+      credential discovery (AWS chain, GCP ADC, Azure ``DefaultAzureCredential``,\n\
+      Vault ``VAULT_ADDR`` / ``VAULT_TOKEN``); the per-tenant key reference is\n``tenants[].kms_key_id``.\
+      \  Key rotation lands in Step 6.7d.\n\n* ``default_key_id`` — fallback key reference\
+      \ for tenants without their own.\n* ``region`` — AWS region for the ``aws``\
+      \ provider.\n* ``local_key`` — hex 32-byte default KEK for the ``local`` provider\n\
+      \  (``${ENV_VAR}``-interpolated; keep it out of the file).\n* ``vault_mount``\
+      \ — Transit secrets-engine mount path for the ``vault`` provider."
     properties:
       enabled:
         default: false
@@ -957,6 +960,10 @@ $defs:
         default: ''
         title: Local Key
         type: string
+      vault_mount:
+        default: transit
+        title: Vault Mount
+        type: string
     title: KmsConfig
     type: object
   KmsProvider:
@@ -965,6 +972,9 @@ $defs:
     - noop
     - local
     - aws
+    - gcp
+    - azure
+    - vault
     title: KmsProvider
     type: string
   LLMConfig: