diff --git a/.github/workflows/release-airgap.yml b/.github/workflows/release-airgap.yml new file mode 100644 index 0000000..de2ca78 --- /dev/null +++ b/.github/workflows/release-airgap.yml @@ -0,0 +1,66 @@ +name: Release — air-gapped bundle + +# Builds the offline install bundle (all runtime images + Helm chart + installer), +# cosign-signs its SHA256SUMS (keyless via GitHub OIDC), and attaches the tarball to +# the GitHub Release. Runs only on version tags (the build pulls every image, so it +# is deliberately release-scoped, not per-PR). + +on: + push: + tags: ["v*.*.*"] + workflow_dispatch: + inputs: + version: + description: "Bundle version (e.g. v0.1.0)" + required: true + +permissions: + contents: write # upload the bundle as a release asset + id-token: write # cosign keyless OIDC signing + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: false + +jobs: + bundle: + name: Build → Sign → Attach + runs-on: ubuntu-22.04 + steps: + - uses: actions/checkout@v4 + + - name: Set up uv + uses: astral-sh/setup-uv@v7 + with: + enable-cache: true + cache-dependency-glob: "uv.lock" + + - run: uv sync --all-packages + + - name: Set up Helm + uses: azure/setup-helm@v4 + + - name: Install cosign + uses: sigstore/cosign-installer@v3 + + - name: Resolve version + id: ver + run: | + version="${{ github.event.inputs.version }}" + [ -n "$version" ] || version="${GITHUB_REF_NAME}" + echo "version=${version}" >> "$GITHUB_OUTPUT" + + - name: Build + sign bundle + run: uv run ragctl airgap build --version "${{ steps.ver.outputs.version }}" --sign + + - name: Verify bundle (content hash) + run: uv run ragctl airgap verify dist/airgap.tar.gz + + - name: Inspect bundle + run: uv run ragctl airgap inspect dist/airgap.tar.gz + + - name: Attach bundle to the release + if: startsWith(github.ref, 'refs/tags/') + uses: softprops/action-gh-release@v2 + with: + files: dist/airgap.tar.gz diff --git a/TRACKER.md b/TRACKER.md index 6014829..46262e9 100644 --- a/TRACKER.md +++ b/TRACKER.md @@ -14,12 +14,13 @@ | | | |---|---| | **Last updated** | 2026-06-08 | -| **Current phase** | Phase 6 — Governance & Tenancy (**8 / 10 steps**) | -| **Overall** | **72 / 84 steps** — Phases 0–5 complete | -| **Next action** | **Step 6.9 — Air-gapped install bundle**: signed tarball with all images + Helm chart; offline bootstrap; cosign verification. | +| **Current phase** | Phase 6 — Governance & Tenancy (**9 / 10 steps**) | +| **Overall** | **73 / 84 steps** — Phases 0–5 complete | +| **Next action** | **Step 6.10 — Compliance posture**: SOC 2 Type II control mapping; GDPR data-residency config; data-retention policies. | **Recently shipped** +- **6.9** ✅ Air-gapped install bundle — `ragctl airgap build`/`inspect`/`verify`/`install`: one signed `.tar.gz` of all runtime images (`docker save`) + the packaged Helm chart + `rag.yaml` + a standalone installer, for networks with no registry / internet. Integrity = a standard `SHA256SUMS` (verifiable with `sha256sum -c`, no cosign/network) pinned as `manifest.content_hash` + an optional cosign signature over it (the 6.6b content-hash + optional-signature pattern); the same `SHA256SUMS` drives the Python verifier and the standalone `install.{sh,ps1}` (need only docker+helm). Typed/tested logic in `ragctl.airgap` (pure core + stubbable docker/helm/cosign seam; `--dry-run` = verifiable bundle minus blobs); digest-pinned manifest-driven image set (`infra/airgap/images.txt`); `task airgap:*`; `release-airgap.yml` on tags — [#161](https://github.com/officialCodeWork/AgentContextOS/pull/161) - **6.8** ✅ SSO / SCIM — `rag-sso`: `FederatedAuth` (an `Auth` SPI backend over per-tenant `OidcProvider` / `SamlProvider`) federates a bearer token / SAML assertion → `Principal` at the existing `authenticate` seam (group claims → `acl_labels`); dependency-free defaults (stdlib HS256 JWT + `defusedxml` SAML), asymmetric OIDC / XML-DSig behind `[oidc]` / `[saml]` extras; algorithm-allowlist downgrade defense; SCIM 2.0 `ScimService` over the new tenant-scoped `ScimStore` SPI driving `/scim/v2/{Users,Groups}` (per-tenant bearer token); per-tenant IdP on `tenants[].sso`; `cfg.sso` / `cfg.scim`; `GET /v1/status/sso`; PII-free `sso.*` / `scim.*` events; `ragctl sso` / `scim` — [#159](https://github.com/officialCodeWork/AgentContextOS/pull/159) - **6.7** ✅ BYOK envelope encryption — `KeyManager` SPI + `EncryptingStorage` + `EnvelopeKeyManager` (AES-256-GCM DEK + tenant AAD); `LocalKeyManager` + four cloud KMS providers (`Aws`/`Gcp`/`AzureKeyVault`/`Vault`, behind `[kms-*]` extras); `cfg.kms` + per-tenant key + factory; per-tenant isolation + sealing + tamper-evidence; zero-downtime rotation (`RotatingKeyManager` + `rewrap`); `ragctl kms` — [#155](https://github.com/officialCodeWork/AgentContextOS/pull/155)–[#158](https://github.com/officialCodeWork/AgentContextOS/pull/158) - **6.6** ✅ Immutable audit log — **6.6a** shared `AuditWriter`/store on `app.state` + `GET /v1/audit` (tenant-scoped, `chain_verified`) + `GET /v1/audit/verify` (whole-log) + `cfg.audit.enabled` ([#153](https://github.com/officialCodeWork/AgentContextOS/pull/153)); **6.6b** `AuditExporter` self-verifying WORM bundle (SHA-256 `content_hash` + HMAC, offline `verify()`), `POST /v1/audit/export` (tenant-scoped) + `ragctl audit`, `cfg.audit.export_secret` ([#154](https://github.com/officialCodeWork/AgentContextOS/pull/154)) @@ -62,9 +63,9 @@ | 3 | Gateway & Agent Runtime | 11 | **11** | 0 | | 4 | Reliability | 6 | **6** | 0 | | 5 | Eval & Observability | 7 | **7** | 0 | -| 6 | Governance & Tenancy | 10 | **8** | 2 | +| 6 | Governance & Tenancy | 10 | **9** | 1 | | 7 | Pilot, Harden, GA | 10 | 0 | 10 | -| **Total** | | **84** | **72** | **12** | +| **Total** | | **84** | **73** | **11** | --- @@ -647,7 +648,7 @@ - **Phase-5 close-out:** Step 5.7 ✅ → **Phase 5 complete (7 / 7)**; deferred items remain documented (per-tenant drift / per-dimension embedding PSI; feedback/breaker/quota Grafana export + Loki-events dashboard; gRPC proto mirror of `corpus_decision` + `experiment`; sequential / multi-metric experiments) - [reference/experiments.md](docs/reference/experiments.md), [reference/admin-ui.md](docs/reference/admin-ui.md) -## Phase 6 — Governance & Tenancy (Weeks 28–34) 🚧 (2 / 10) +## Phase 6 — Governance & Tenancy (Weeks 28–34) 🚧 (1 / 10) | Step | Title | Status | Planned deliverables | |------|-------|:------:|----------------------| @@ -659,7 +660,7 @@ | 6.6 | Immutable audit log | ✅ | **6.6a** [#153](https://github.com/officialCodeWork/AgentContextOS/pull/153) — read API `GET /v1/audit` + `GET /v1/audit/verify` + shared store + `cfg.audit`. **6.6b** [#154](https://github.com/officialCodeWork/AgentContextOS/pull/154) — `AuditExporter` signed WORM bundle + `POST /v1/audit/export` + `ragctl audit` | | 6.7 | BYOK (Bring Your Own Key) | ✅ | **6.7a** ✅ [#155](https://github.com/officialCodeWork/AgentContextOS/pull/155) — `KeyManager` SPI + envelope encryption (`LocalKeyManager`) + `EncryptingStorage` + sealing/isolation/tamper. **6.7b** ✅ [#156](https://github.com/officialCodeWork/AgentContextOS/pull/156) — `cfg.kms` + per-tenant key + `build_key_manager_from_config` factory + `AwsKmsKeyManager`. **6.7c** ✅ [#157](https://github.com/officialCodeWork/AgentContextOS/pull/157) — `GcpKmsKeyManager` / `AzureKeyVaultKeyManager` / `VaultKeyManager` behind `[kms-*]` extras. **6.7d** ✅ [#158](https://github.com/officialCodeWork/AgentContextOS/pull/158) — `RotatingKeyManager` zero-downtime rotation + `rewrap` | | 6.8 | SSO / SCIM | ✅ | [#159](https://github.com/officialCodeWork/AgentContextOS/pull/159) — `rag-sso`: `FederatedAuth` (`Auth` SPI backend over per-tenant `OidcProvider` / `SamlProvider`) federates a token / assertion → `Principal` at the existing `authenticate` seam (groups → `acl_labels`); stdlib HS256 + `defusedxml` defaults, asymmetric OIDC / XML-DSig behind `[oidc]` / `[saml]` extras; SCIM 2.0 `ScimService` + `ScimStore` SPI → `/scim/v2/*` (per-tenant bearer); `tenants[].sso` + `cfg.sso` / `cfg.scim`; `GET /v1/status/sso`; `ragctl sso` / `scim` | -| 6.9 | Air-gapped install bundle | ⏳ | Signed tarball with all images + Helm chart; offline bootstrap; cosign verification | +| 6.9 | Air-gapped install bundle | ✅ | [#161](https://github.com/officialCodeWork/AgentContextOS/pull/161) — `ragctl airgap build`/`verify`/`install`: signed `.tar.gz` of all images (`docker save`) + Helm chart + config + standalone installer; `SHA256SUMS` content-hash (offline-verifiable) + optional cosign; `infra/airgap/images.txt`; `install.{sh,ps1}`; `task airgap:*`; `release-airgap.yml` | | 6.10 | Compliance posture | ⏳ | SOC 2 Type II control mapping; GDPR data-residency config; data-retention policies | ### 6.1 — Logical multi-tenancy ✅ [#148](https://github.com/officialCodeWork/AgentContextOS/pull/148) @@ -770,6 +771,15 @@ New ground — the only prior crypto was HMAC signing. The V1 plan calls for en - New core types **`FederatedIdentity` / `SsoProtocol` / `ScimUser` / `ScimGroup`** (+ nested) + **`SsoError`** (401) / **`ScimError`** (400) / **`ScimNotFoundError`** (404) / **`ScimConflictError`** (409); wire types `ScimListResponse` / `ScimPatchOp` / `ScimErrorBody` / `SsoStatusResponse` (`dist/schemas` + `dist/openapi` regenerated); `cfg.sso` / `cfg.scim` / `tenants[].sso` (`dist/rag.schema`); PII-free `sso.*` / `scim.*` events (subject **hashed**, never email / userName); **`ragctl sso`** (list + in-process OIDC demo) + **`ragctl scim`** (in-process provisioning demo) - **Scope:** verification + provisioning at the boundary. **Deferred:** remote JWKS discovery + rotation (configured static keys only), SAML SP-initiated redirect + metadata, SCIM bulk / `/Me` / ETag, directory-backed deprovisioning at authenticate-time, the admin-console SSO/SCIM card. ~80 new tests (jwt / oidc / saml / federated-auth / scim-service unit; `ScimStore` contract suite; gateway SCIM CRUD + token guard + isolation + disabled-404; SSO status + JWT-through-middleware; `ragctl`). All gates green (ruff, mypy --strict, RAG001, schema/openapi/config-drift, policy-coverage, log-schema/event-registry/PII). [ADR-0040](docs/adr/ADR-0040-sso-scim.md), [reference/sso.md](docs/reference/sso.md), [architecture/sso-scim.md](docs/architecture/sso-scim.md) +### 6.9 — Air-gapped install bundle ✅ [#161](https://github.com/officialCodeWork/AgentContextOS/pull/161) + +- Ships the platform as **one signed, self-contained artifact** for networks with no registry / internet: all runtime container images (`docker save`), the packaged Helm chart (`helm package`), an example `rag.yaml`, and a standalone offline installer. Packages what already exists — the gateway image (built from `./Dockerfile`, signed keyless in `docker.yml`) + the chart (0.9) + the runtime image set (`docker-compose`) +- **Integrity = a standard `SHA256SUMS` + optional cosign** (mirrors the 6.6b WORM export). A `content_hash` (SHA-256 over the canonical `SHA256SUMS`) is **always** present for tamper-evidence and is verifiable with nothing but `sha256sum -c` — no network, no cosign — and is pinned in `manifest.json`. A cosign signature **over `SHA256SUMS`** adds authenticity when present. The *same* `SHA256SUMS` drives both the Python verifier and the standalone shell installer, so they can never disagree; `manifest`/`SHA256SUMS`/`*.sig`/`*.pem` are excluded from the hashed payload +- **Standalone `install.{sh,ps1}` need no Python workspace** — an air-gap host has bash/pwsh + docker + helm (+ maybe cosign / sha256sum), not uv or the repo. Shipped *inside* every bundle, they verify → `docker load images.tar` → `helm upgrade --install`; `ragctl airgap install` is the same flow for operators who have the toolchain. `--verify-only` checks integrity without installing +- **Build/verify logic in `ragctl.airgap`** (under `packages/`, so `mypy --strict` + unit-tested): a **pure** core (`BundleManifest` model, `render_sha256sums` / `content_hash` / `build_manifest` / `verify_bundle`) cleanly separated from the **subprocess seam** (docker / helm / cosign through one `_run` chokepoint). A **`--dry-run`** produces a complete, *verifiable* bundle minus the image blobs, so the whole packaging + verification path is testable with no Docker +- **Digest-pinned, manifest-driven image set:** `infra/airgap/images.txt` lists the third-party images (tags); the build resolves each to its digest into `manifest.json`. The gateway image is added from the chart's `image.repository` + `--version`, so the app image is always release-versioned. **Key-based cosign** is the air-gap recommendation (keyless needs Rekor + a deployment-specific identity policy); the release path (`release-airgap.yml` on `v*` tags) signs keyless and attaches the tarball to the GitHub Release +- **Scope:** docker-based image load + the gateway chart. **Deferred:** `ctr`/podman load, registry re-tag/push, multi-arch selection, bundling the *backend* charts (chart deploys the gateway; backends stay external), TUF-rooted offline keyless verify. `task airgap:build` / `build-dry` / `verify`; ~17 tests (manifest / hashing / tamper detection / tarball round-trip / signature paths via stubbed cosign / CLI on a dry-run bundle), shell installer syntax-checked + exercised via `--verify-only`. All gates green (ruff, mypy --strict, RAG001, schema/openapi/config-drift, policy-coverage, log-schema). [ADR-0041](docs/adr/ADR-0041-airgap-bundle.md), [reference/airgap.md](docs/reference/airgap.md), [architecture/airgap-bundle.md](docs/architecture/airgap-bundle.md), [guides/airgap-install.md](docs/guides/airgap-install.md) + --- ## Phase 7 — Pilot, Harden, GA (Weeks 34–40) ⏳ diff --git a/Taskfile.yml b/Taskfile.yml index 689467e..67a1775 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -270,6 +270,26 @@ tasks: cmds: - helm template rag-platform infra/helm/rag-platform/ + # --------------------------------------------------------------------------- + # Air-gapped install bundle (Step 6.9) + # --------------------------------------------------------------------------- + airgap:build: + desc: "Build the signed offline install bundle → dist/airgap.tar.gz (needs docker + helm + cosign)" + cmds: + - uv run ragctl airgap build --sign + + airgap:build-dry: + desc: "Build a dry-run bundle (no docker/helm/cosign) — preview packaging + verify logic" + cmds: + - uv run ragctl airgap build --dry-run + + airgap:verify: + desc: "Verify an air-gapped bundle's content hash + signature (BUNDLE=dist/airgap.tar.gz)" + vars: + BUNDLE: '{{.BUNDLE | default "dist/airgap.tar.gz"}}' + cmds: + - uv run ragctl airgap verify {{.BUNDLE}} + # --------------------------------------------------------------------------- # Cleanup — platform-specific commands via platforms: key # --------------------------------------------------------------------------- diff --git a/docs/README.md b/docs/README.md index 36e469e..7848aee 100644 --- a/docs/README.md +++ b/docs/README.md @@ -12,6 +12,7 @@ | [audit-log.md](architecture/audit-log.md) | Immutable audit log (Step 6.6): tamper-evidence (SHA-256 hash chain) vs immutability-at-rest (WORM export, 6.6b); one shared `AuditWriter`/store on `app.state`; read-path tenant scoping (tenant-scoped list vs whole-log verify); why the read API defaults on | | [byok.md](architecture/byok.md) | BYOK / envelope encryption (Step 6.7): what's encrypted (chunk content at rest) vs not (embedding vectors — search needs plaintext); DEK+KEK envelope (client-side AES-GCM DEK, provider wraps the DEK); per-tenant isolation via KEK + `tenant_id` AAD; sealing as a typed error; rag-core/rag-backends split; slicing (6.7a library, 6.7b config + factory + AWS KMS, 6.7c GCP/Azure/Vault, 6.7d rotation) | | [sso-scim.md](architecture/sso-scim.md) | SSO / SCIM (Step 6.8): OIDC + SAML federation + SCIM 2.0 provisioning + per-tenant IdP config. `FederatedAuth` *is* an `Auth` SPI backend (the `authenticate(token, tenant_id) → Principal` seam — no middleware change); group claims → `acl_labels` so Step 6.3/6.5 govern federated users; dependency-free defaults (stdlib HS256 JWT, `defusedxml` SAML) with asymmetric OIDC / XML-DSig behind `[oidc]` / `[saml]` extras; algorithm-allowlist (`alg:none`/downgrade defense); per-tenant IdP on `tenants[].sso`; SCIM is a separate surface with its own per-tenant bearer token + tenant-scoped `ScimStore`; PII-free `sso.*`/`scim.*` events (hashed subject); deferred (JWKS rotation, SP-initiated SAML, directory-backed deprovisioning) | +| [airgap-bundle.md](architecture/airgap-bundle.md) | Air-gapped install bundle (Step 6.9): one signed `.tar.gz` of all runtime images (`docker save`) + the packaged Helm chart + `rag.yaml` + a standalone installer, for networks with no registry / internet. Integrity = a standard `SHA256SUMS` (verifiable with `sha256sum -c`, no cosign/network) whose hash is pinned as `manifest.content_hash`, plus an optional cosign signature over it (the 6.6b content-hash + optional-signature pattern); the same `SHA256SUMS` drives the Python verifier and the standalone shell installer. Logic in `ragctl.airgap` (typed/tested, subprocess seam stubbable; `--dry-run` = verifiable bundle minus blobs); standalone `install.{sh,ps1}` need only docker+helm; digest-pinned manifest-driven image set; key-based cosign for air-gap, keyless for connected releases; deferred (ctr/podman load, registry re-tag/push, backend charts) | | [caching.md](architecture/caching.md) | Three-cache split: `EmbeddingCache`, `RetrievalCache`, `AnswerCache` — distinct invalidation rules | | [performance.md](architecture/performance.md) | Hot-path discipline, per-SPI p99 budgets, async telemetry, reviewer checklist | | [pipeline-batcher.md](architecture/pipeline-batcher.md) | `Pipeline` (async DAG, bounded queues, per-stage workers) + `Batcher` (DataLoader-pattern coalescing) primitives — Step 1.1d | @@ -110,6 +111,7 @@ | [audit.md](reference/audit.md) | Audit log (Step 6.6) — `AuditEvent` / `AuditStore` (append / events / verify_chain) / `NoopAuditStore` SHA-256 hash chain / `AuditWriter` (+ `.store`); read API `GET /v1/audit` (tenant-scoped, `chain_verified`) + `GET /v1/audit/verify` (whole-log); WORM signed export (6.6b) — `AuditExporter` (content_hash + HMAC), `POST /v1/audit/export`, offline `verify()`, `ragctl audit`; `cfg.audit.enabled` / `export_secret`; durable-store extension points | | [encryption.md](reference/encryption.md) | BYOK envelope encryption (Step 6.7) — `KeyManager` SPI + `NoopKeyManager`; `EnvelopeKeyManager` (AES-256-GCM DEK + `tenant_id` AAD) + `LocalKeyManager` + cloud providers **`AwsKmsKeyManager`** / **`GcpKmsKeyManager`** / **`AzureKeyVaultKeyManager`** / **`VaultKeyManager`** (behind `[kms-*]` extras); `EncryptingStorage` decorator; `EncryptionError` / `KeyUnavailableError` (sealing); provider table; `cfg.kms` + `tenants[].kms_key_id` + `build_key_manager_from_config` factory; **`RotatingKeyManager`** + `RetiredKey` + `rewrap` (6.7d zero-downtime rotation); `ragctl kms [--rotate]` | | [sso.md](reference/sso.md) | SSO / SCIM (Step 6.8, `rag_sso`) — `FederatedAuth` (`Auth` backend) + `OidcProvider`/`OidcSettings` + `SamlProvider`/`SamlSettings`/`signxml_verifier` + `identity_to_principal`; `ScimService`/`parse_eq_filter`; low-level `verify_jwt`/`encode_jwt_hs256` (stdlib HS256, `[oidc]` RS256); `ScimStore` SPI + `NoopScimStore`; `FederatedIdentity`/`SsoProtocol`/`ScimUser`/`ScimGroup` core types + `ScimListResponse`/`ScimPatchOp`/`ScimErrorBody`/`SsoStatusResponse` wire types; `SsoError`/`ScimError`/`ScimNotFoundError`/`ScimConflictError`; `/scim/v2/*` + `GET /v1/status/sso`; `cfg.sso`/`cfg.scim`/`tenants[].sso`; `sso.*`/`scim.*` events; `ragctl sso`/`ragctl scim`; extension points | +| [airgap.md](reference/airgap.md) | Air-gapped install bundle (Step 6.9) — `ragctl airgap build`/`inspect`/`verify`/`install`; `ragctl.airgap` models (`BundleManifest`/`BundleImage`/`BundleFile`/`BundleVerification`) + pure helpers (`parse_image_list`/`render_sha256sums`/`content_hash`/`build_manifest`/`verify_bundle`/`gateway_image_ref`) + the docker/helm/cosign seam (`assemble_bundle`/`save_images`/`package_chart`/`load_images`/`install_chart`/`cosign_*`); bundle layout + `SHA256SUMS`/`content_hash` integrity (+ `reason` codes); `infra/airgap/images.txt`; standalone `install.{sh,ps1}` (`--verify-only`); `--dry-run`; `task airgap:build`/`build-dry`/`verify`; `release-airgap.yml` | | [webhooks.md](reference/webhooks.md) | Outbound webhooks (Step 3.9) — event catalogue (`ingest.completed` / `audit.policy_violation` / `drift.detected` / `eval.regression`), event envelope, HMAC signing + `verify()`, at-least-once delivery, `/v1/webhooks/subscriptions` CRUD + test, `rag.yaml` block, `ragctl webhooks demo`, internals + extension points | | [integrations.md](reference/integrations.md) | Framework adapters (Step 3.8) — `agentcontextos.integrations.*` for LangChain / LlamaIndex / Haystack / DSPy / LangGraph / CrewAI / AutoGen / Semantic Kernel; per-framework extras, shared config + chunk metadata, usage per framework, internals + extension points | | [status-api.md](reference/status-api.md) | Status & Metrics API (Step 3.11) — `/v1/status/health` / `metrics` / `logs` (+ SSE `logs/stream`), `WS /v1/status/ws`, `/v1/connectors/status`; metric catalogue + request-timing middleware, the `MetricsCollector` / `LogTail` read-side, CORS + query-param identity for browser streams, extension points | @@ -120,6 +122,7 @@ |------|-------------| | [logging-policy.md](guides/logging-policy.md) | RAG001 policy: structured-logging requirement, allowlist, and how to extend it | | [ragctl-quickstart.md](guides/ragctl-quickstart.md) | Five-minute tour of the `ragctl` CLI | +| [airgap-install.md](guides/airgap-install.md) | Operator runbook (Step 6.9): build + sign an offline bundle on a connected host (`ragctl airgap build --sign`), transfer it, then verify + install on the air-gapped target with the standalone `install.sh`/`install.ps1` (`sha256sum -c SHA256SUMS` → cosign → `docker load` → `helm upgrade --install`); trimming the image set; pointing the chart at in-network backends; offline-verifying a bundle with stdlib tools | | [curl-quickstart.md](guides/curl-quickstart.md) | 🥈 Curl-able RAG (Step 3.1): 5-minute walkthrough from `curl` to gateway response, including ingest, query, generate, OpenAPI | | [grpcurl-quickstart.md](guides/grpcurl-quickstart.md) | gRPC quickstart (Step 3.2): 5-minute walkthrough using `grpcurl` against the in-process server — health check, list corpora, server-streaming query, structured errors | | [mcp-quickstart.md](guides/mcp-quickstart.md) | MCP quickstart (Step 3.3): 5-minute walkthrough — `ragctl mcp-query`, running the stdio server, mounting `@ragplatform/mcp` in Claude Desktop, the three tools, error shape | @@ -181,6 +184,7 @@ broken, and what to fix before committing to the next phase. | [ADR-0034-physical-multi-tenancy.md](adr/ADR-0034-physical-multi-tenancy.md) | Decision (Step 6.2): a *dedicated* vector index/collection per tenant. `TenantConfig.dedicated_index` resolves to a `physical_index` key on `TenantSettings`, threaded onto `RequestContext.physical_index`; backends read only `ctx` (graph is backends→core, never rag-config) and namespace their base under it (`-`), lazily creating it; one instance + per-tenant derivation (no per-tenant instances, no SPI change); Noop is the CI conformance oracle (keyed by `physical_index`) for a cross-tenant probe gate that proves isolation independent of the tenant filter; Noop + Pinecone + Qdrant this step, others later | | [ADR-0035-acl-pushdown.md](adr/ADR-0035-acl-pushdown.md) | Decision (Step 6.3): label-based ACL push-down at retrieval. `AclPolicyEngine` (a decorator like `QuotaPolicyEngine`) And-merges `any_in("acl_labels", principal.acl_labels)` into every `read_chunk` push-down at the canonical `HybridRetriever` PDP site — overlap semantics via the existing `AnyIn` predicate (zero backend/translator changes), **fail-closed** (label-less principal matches nothing; "public" = a shared label), **opt-in** via `cfg.acl.enabled`; emits `acl.egress_denied` on a request-level denial; graph edge ACLs + post-retrieval re-verification (6.4) deferred | | [ADR-0036-acl-egress-verifier.md](adr/ADR-0036-acl-egress-verifier.md) | Decision (Step 6.4): a post-retrieval ACL re-check as a **defense-in-depth second layer** behind the 6.3 push-down. `AclEgressVerifier.verify(ctx, refs)` drops any returned `ChunkRef` whose labels don't overlap the principal's — same overlap semantics (no-op on correct results), reading `ChunkRef.acl_labels` (no re-hydration), **independent of the PDP** (consults only `ctx.principal.acl_labels`) so a push-down bug/bypass can't disable both; wired at the gateway as a `SupportsRoute` wrapper around `app.state.retrieval_router` (covers query/retrieve/corpus/OpenAI/agent); `cfg.acl.verify_egress` default on but gated by `enabled`; emits `acl.egress_violation` on a caught leak; a red-team gate proves a zero escaped-violation rate when the push-down is bypassed; backend-mislabel re-hydration + per-tenant violation metrics deferred | +| [ADR-0041-airgap-bundle.md](adr/ADR-0041-airgap-bundle.md) | Decision (Step 6.9): ship the platform as one signed, self-contained offline bundle (all runtime images + Helm chart + config + installer). Integrity reuses the WORM-export pattern (6.6b): a standard `SHA256SUMS` whose hash is pinned as `manifest.content_hash` is the **hard gate** — verifiable with nothing but `sha256sum`, no network/cosign — and a cosign signature **over `SHA256SUMS`** adds authenticity; the *same* `SHA256SUMS` drives the Python verifier and the standalone shell installer so they can't diverge. The shell/pwsh `install.{sh,ps1}` (shipped inside the bundle) need only docker+helm (air-gap hosts lack uv/the workspace); `ragctl airgap` holds the typed/tested build+verify logic (pure core separated from a stubbable docker/helm/cosign subprocess seam; `--dry-run` = a verifiable bundle minus image blobs, so the path is testable with no Docker). Digest-pinned manifest-driven image set (`infra/airgap/images.txt` + the chart-derived gateway image); key-based cosign is the air-gap recommendation (keyless needs Rekor + an identity policy), keyless is the connected-release path (`release-airgap.yml` on tags). Deferred: ctr/podman load, registry re-tag/push, multi-arch selection, bundling backend charts, TUF-rooted offline keyless verify; rejected `oras`/OCI (no registry to pull from in an air-gap), a second HMAC scheme (cosign already the signer), a pure-shell build (would escape mypy/tests) | | [ADR-0040-sso-scim.md](adr/ADR-0040-sso-scim.md) | Decision (Step 6.8): enterprise identity in two surfaces. **Federation** — `FederatedAuth` *is* an `Auth` SPI backend (the `authenticate(token, tenant_id) → Principal` seam already runs at the boundary, so wiring it is the whole integration — no middleware change); group claims → `acl_labels` so Step 6.3 push-down + 6.5 PII egress govern federated users unchanged (`authorize` stays a coarse allow — federation establishes *who*, the PDP decides *what*). Dependency-free defaults (stdlib HS256 JWT with full `exp`/`nbf`/`iss`/`aud` + constant-time compare; `defusedxml` SAML validating Issuer/Conditions/Audience) with asymmetric OIDC (PyJWT, `[oidc]`) + SAML XML-DSig (signxml, `[saml]`, injected verifier → fail-closed) behind extras; **algorithm-allowlist** designs out `alg:none`/RS↔HS confusion. Per-tenant IdP on `tenants[].sso` (reuses Step 6.1 config; no provider → bearer rejected, header-identity still works). **Provisioning** — SCIM 2.0 is a separate surface with its own per-tenant bearer token (`cfg.scim.tokens`, not a user JWT), a tenant-scoped `ScimStore` SPI (`NoopScimStore`) + `ScimService`, SCIM-shaped errors, disabled→404; no new governed SPI call (linter passes). PII-free `sso.*`/`scim.*` events (hashed subject, never email/userName). Deferred: JWKS rotation, SP-initiated SAML + metadata, SCIM bulk/`/Me`/ETag, directory-backed deprovisioning, admin-console card; rejected Authlib/python3-saml (heavy lxml/xmlsec on the default install), a dedicated SSO middleware, SCIM token on `TenantConfig` | | [ADR-0037-pii-egress-policies.md](adr/ADR-0037-pii-egress-policies.md) | Decision (Step 6.5): per-tenant PII enforcement at egress via a `PiiPolicyEngine` `egress_text` decorator (mirrors `QuotaPolicyEngine` / `AclPolicyEngine`), living in `rag-pii` (gains a `rag-policy` dep, like `rag-quota`). Handles both subject shapes the gateway already passes — `list[Chunk]` context + `str` answer — so it plugs into the existing `egress_text` call sites with no route change; maps `ctx.pii_policy.action` allow→delegate / redact·mask→`transform` / block→`deny`, reusing the Step 1.7 detector + rewriters and the same `min_score`+`entities` filter (no-op on clean text); opt-in `cfg.pii.enabled` (injects `RegexPIIDetector`); PII-free `pii.egress_blocked` (block) / `pii.detected` (redact·mask); post-gen answer re-check for query/OpenAI/gRPC + citation egress deferred (stored chunks are ingest-sanitised) | | [ADR-0038-immutable-audit-log.md](adr/ADR-0038-immutable-audit-log.md) | Decision (Step 6.6): make the 0.7c hash-chain audit log usable + provably intact, in two slices. 6.6a — the SHA-256 chain is the tamper-*evidence* mechanism (no second scheme); one shared `AuditWriter`/store on `app.state` (corpus router + read API write/read the same chain); `GET /v1/audit` tenant-scoped (a tenant sees only its own events, newest-first, `chain_verified` inline) + `GET /v1/audit/verify` whole-log `{ok,event_count}` (content-free, so global verification leaks nothing cross-tenant); read API on by default (`cfg.audit.enabled=true` — passive compliance record, unlike behaviour-changing ACL/PII). 6.6b — WORM signed export: `AuditExporter` builds a self-verifying `AuditExport` (SHA-256 `content_hash` over the events + HMAC signature, mirroring the ProvenanceSigner scheme), `POST /v1/audit/export` (tenant-scoped) + `ragctl audit` (whole-log), verifiable offline (`{content_ok, verified, reason}`); the artifact for immutable storage (S3 Object Lock) → immutability at rest; unsigned when no `export_secret`. Audit-coverage expansion + a durable live-store backend deferred | diff --git a/docs/adr/ADR-0041-airgap-bundle.md b/docs/adr/ADR-0041-airgap-bundle.md new file mode 100644 index 0000000..c936c50 --- /dev/null +++ b/docs/adr/ADR-0041-airgap-bundle.md @@ -0,0 +1,90 @@ +# ADR-0041 — Air-gapped install bundle + +**Status:** Accepted +**Date:** 2026-06-08 +**Step:** 6.9 — Air-gapped install bundle (Phase 6 — Governance & Tenancy) +**Related:** [ADR-0038](ADR-0038-immutable-audit-log.md) (content-hash + optional-signature pattern), [ADR-0003](ADR-0003-iac-foundation.md) (Helm chart), [architecture/airgap-bundle.md](../architecture/airgap-bundle.md), [reference/airgap.md](../reference/airgap.md), [guides/airgap-install.md](../guides/airgap-install.md) + +## Context + +The V1 plan (Step 6.9) calls for an **air-gapped install bundle**: a signed tarball +with all container images + the Helm chart, an **offline bootstrap**, and **cosign +verification**. Regulated / disconnected buyers install into networks with no +registry access and no internet — so the platform must ship as one self-contained, +verifiable artifact that installs with only the tools already on the host. + +The pieces already exist: the gateway image (built from `./Dockerfile`, signed +keyless via cosign in `docker.yml`), the `rag-platform` Helm chart (Step 0.9), and +the runtime image set (`docker-compose.yml`). 6.9 packages them. + +## Decision + +**1. Integrity = a standard `SHA256SUMS` + optional cosign signature.** Mirrors the +WORM audit export (6.6b): a `content_hash` (SHA-256 over the canonical +`SHA256SUMS`) is **always** present for tamper-evidence and is verifiable with +nothing but `sha256sum` — no network, no cosign. A cosign signature **over +`SHA256SUMS`** adds authenticity when present. The *same* `SHA256SUMS` drives both +the Python verifier and the standalone shell installer, so they can never disagree. +`sha256sum -c SHA256SUMS` is the universal primitive; cosign is layered on top, not +load-bearing for integrity. + +**2. The standalone shell/pwsh installer needs no Python workspace.** An air-gapped +host has bash/pwsh + docker + helm (+ maybe cosign + sha256sum) — *not* uv or the +repo. So `install.sh` / `install.ps1` (shipped *inside* the bundle) are +self-contained: verify `SHA256SUMS` → optional cosign → `docker load images.tar` → +`helm upgrade --install`. `ragctl airgap install` is the same flow for operators who +already have the toolchain, but the bundle never depends on it. + +**3. Build/verify logic lives in `ragctl airgap` (typed, tested).** The pure logic +(manifest model, hashing, `verify_bundle`) sits in `packages/ragctl/airgap.py` — +under `packages/`, so it gets `mypy --strict` + unit tests — cleanly separated from +the subprocess orchestration (docker / helm / cosign), which is a stubbable seam. +A `--dry-run` produces a complete, **verifiable** bundle *minus* the image blobs, so +the whole packaging + verification path is testable with no Docker. + +**4. Digest-pinned, manifest-driven image set.** `infra/airgap/images.txt` is the +human-editable list of third-party runtime images (tags, as in `docker-compose`); +`ragctl airgap build` resolves each tag to its immutable digest and records it in +`manifest.json`, so a built bundle is digest-pinned. The platform's own gateway +image is added by the build from the chart's `image.repository` + `--version`, so +the application image is always release-versioned (never a stale pin in a static +file). + +**5. Key-based cosign is the air-gap recommendation; keyless is for releases.** +Keyless `verify-blob` needs Rekor (online) and a deployment-specific +`--certificate-identity` / `--certificate-oidc-issuer` policy — neither fits an +air-gap. So the installer's automated signature check is **key-based** (distribute +the cosign public key out-of-band); a keyless cert in the bundle is treated as +*advisory* (the content hash still gates integrity). The CI release workflow signs +keyless (connected, public), and the content hash remains verifiable offline +regardless. + +## Consequences + +- `ragctl airgap build / inspect / verify / install`; `infra/airgap/images.txt`; + `scripts/airgap_install.{sh,ps1}` (shipped in every bundle as `install.{sh,ps1}`); + `task airgap:build` / `airgap:build-dry` / `airgap:verify`; a release-only + `release-airgap.yml` workflow that builds + keyless-signs + attaches the tarball to + the GitHub Release. +- A bundle is one `*.tar.gz` containing `images.tar`, `chart/*.tgz`, `rag.yaml`, + `install.{sh,ps1}`, `README.md`, `SHA256SUMS` (+ `.sig` / `.pem` when signed), and + `manifest.json`. It verifies + installs with no network. +- ~17 tests (manifest / hashing / tamper detection / tarball round-trip / signature + paths / CLI). The pure core is `mypy --strict` + fully unit-tested; the shell + installer is syntax-checked and exercised via its `--verify-only` path. +- **Deferred:** containerd/`ctr`/podman load paths (docker only today), re-tag + + push into a private registry, multi-arch bundle selection, bundling the backend + *charts* (the chart deploys the gateway; backends stay external), TUF-rooted + offline keyless verification. + +## Alternatives considered + +- **`oras` / OCI artifact distribution.** Elegant for connected registries, but an + air-gapped host has no registry to `oras pull` from — a plain tarball + `docker + load` is the lowest-common-denominator that always works. +- **A second signing scheme (HMAC) like the audit export.** Unnecessary — cosign is + already the platform's image-signing tool (Step 0.5/`docker.yml`); reusing it for + the bundle keeps one signing story, with `SHA256SUMS` as the content anchor. +- **Pure shell build script (no ragctl).** Would duplicate the manifest/verify logic + in bash + pwsh and escape `mypy`/unit tests. Keeping the logic in `ragctl` (typed, + tested) with thin standalone *install* scripts is the better split. diff --git a/docs/architecture/airgap-bundle.md b/docs/architecture/airgap-bundle.md new file mode 100644 index 0000000..ccf60ef --- /dev/null +++ b/docs/architecture/airgap-bundle.md @@ -0,0 +1,107 @@ +# Air-gapped install bundle (Step 6.9) + +## Overview + +A single, self-contained, verifiable artifact that installs AgentContextOS into a +network with **no registry access and no internet**: all runtime container images, +the packaged Helm chart, an example `rag.yaml`, and a standalone offline installer +— pinned by a `SHA256SUMS` and (optionally) cosign-signed. + +``` +agentcontextos-airgap-.tar.gz +├── images.tar # docker save of every runtime image (digest-pinned) +├── chart/rag-platform-.tgz # helm package output +├── rag.yaml # example production config +├── install.sh / install.ps1 # standalone offline installer (no Python needed) +├── README.md # operator quick-reference +├── SHA256SUMS # for every payload file +├── SHA256SUMS.sig / .pem # cosign signature + cert (when --sign) +└── manifest.json # images (+ digests), versions, content_hash +``` + +## Usage + +```bash +# Build (online build host): all images + chart, signed. +task airgap:build # → dist/airgap.tar.gz (ragctl airgap build --sign) +ragctl airgap inspect dist/airgap.tar.gz +ragctl airgap verify dist/airgap.tar.gz + +# Install (offline target host — needs only docker + helm): +tar xzf agentcontextos-airgap-v0.1.0.tar.gz -C bundle && cd bundle +./install.sh --namespace agentcontextos # verify → docker load → helm install +``` + +The image set is `infra/airgap/images.txt` (the third-party runtime images) plus +the gateway image, which the build derives from the chart's `image.repository` + +`--version`. + +## Internals + +### Integrity model (content hash + optional signature) + +Mirrors the WORM audit export (Step 6.6b). Every payload file is hashed into a +standard **`SHA256SUMS`** (` `, sorted, two-spaced — `sha256sum -c` +format). The **`content_hash`** is the SHA-256 *of* that `SHA256SUMS` text and is +recorded in `manifest.json`. Verification: + +1. Re-hash every payload file and re-render `SHA256SUMS`; compare to the bundled one. +2. Re-hash that and compare to `manifest.content_hash` (ties the manifest to the + payload). +3. If `SHA256SUMS.sig` is present and a cosign **key** is supplied, verify the + signature over `SHA256SUMS`. + +Step 1–2 are the **hard integrity gate** — verifiable with nothing but +`sha256sum`, so an air-gapped host with no cosign and no network still detects any +added / removed / edited file. The signature is layered authenticity on top. +`manifest.json` / `SHA256SUMS` / `*.sig` / `*.pem` are excluded from the hashed set +(they describe / sign the payload). + +### Code layout + +- `ragctl.airgap` (`packages/ragctl/src/ragctl/airgap.py`) — **pure** logic + (`BundleManifest`, `parse_image_list`, `render_sha256sums`, `content_hash`, + `build_manifest`, `verify_bundle`) separated from the **subprocess seam** + (`save_images` / `package_chart` / `load_images` / `install_chart` / + `cosign_sign_blob` / `cosign_verify_blob`, all through one `_run` chokepoint). The + separation is what makes the core `mypy --strict` + unit-testable with no Docker. +- `ragctl airgap build / inspect / verify / install` — the CLI. +- `scripts/airgap_install.{sh,ps1}` — the standalone installer shipped *inside* each + bundle as `install.{sh,ps1}`; depends only on docker + helm (+ optional cosign / + sha256sum), never on the Python workspace. + +### `--dry-run` + +`ragctl airgap build --dry-run` skips `docker save` / `helm package` (no Docker +required) but still writes a complete, **verifiable** bundle — the chart is copied +raw, `images.tar` is omitted (`manifest.images_saved=false`). Used for previewing +packaging, for CI verification of the build path, and by the test-suite. + +### Build vs install flow + +``` +BUILD (online): images.txt + chart ──ragctl airgap build──▶ docker save + helm package + ─▶ SHA256SUMS + manifest.json ─▶ cosign sign-blob ─▶ tar.gz ─▶ release asset + +INSTALL (offline): tar xzf ─▶ install.sh ─▶ sha256sum -c SHA256SUMS [+ cosign verify-blob] + ─▶ docker load images.tar ─▶ helm upgrade --install +``` + +## Extension points + +- **Container runtimes** — `load_images` shells out to `docker load`; swap in + `ctr images import` / `podman load` for containerd / podman hosts. +- **Private registry** — extend `install` to re-tag + `docker push` into an + in-network registry before `helm install` (today images load straight into the + node's docker). +- **Image set** — edit `infra/airgap/images.txt` (drop observability for a minimal + core bundle, add backends you depend on); digests are resolved at build time. +- **Signing** — `--sign --cosign-key ` for key-based (the air-gap + recommendation); the CI release uses keyless OIDC for connected, public artifacts. + +## Boundaries / deferred + +Docker-only image load (no `ctr`/podman yet), registry re-tag/push, multi-arch +bundle selection, bundling the *backend* charts (the chart deploys the gateway; +backends stay external), and TUF-rooted offline keyless verification are deferred. +See [ADR-0041](../adr/ADR-0041-airgap-bundle.md). diff --git a/docs/guides/airgap-install.md b/docs/guides/airgap-install.md new file mode 100644 index 0000000..08c5981 --- /dev/null +++ b/docs/guides/airgap-install.md @@ -0,0 +1,85 @@ +# Guide: installing AgentContextOS into an air-gapped network (Step 6.9) + +This runbook takes a signed offline bundle from a connected build host to a running +deployment inside a disconnected network — with no registry access and no internet +on the target side. + +## Prerequisites + +| Side | Needs | +|------|-------| +| **Build host** (connected) | `docker`, `helm`, `cosign`, `uv` + this repo | +| **Target host** (air-gapped) | `docker`, `helm`, a Kubernetes context (`kubectl`/`helm` configured), and `sha256sum` (+ optional `cosign`) | + +## 1. Build + sign the bundle (connected host) + +```bash +# Resolves the gateway image from the chart + version and the third-party images +# from infra/airgap/images.txt, saves them, packages the chart, signs SHA256SUMS. +ragctl airgap build --version v0.1.0 --sign --cosign-key cosign.key +# → dist/airgap.tar.gz + +ragctl airgap inspect dist/airgap.tar.gz # sanity-check images + versions +ragctl airgap verify dist/airgap.tar.gz # content hash OK +``` + +To trim the bundle (e.g. drop observability images), edit `infra/airgap/images.txt` +before building. For a **key-based** signature (recommended for air-gap) pass +`--cosign-key`; distribute the matching **public** key (`cosign.pub`) to the target +operator out-of-band. Without `--cosign-key`, signing is keyless (Fulcio/Rekor) and +is best for connected, public releases. + +## 2. Transfer + +Copy `dist/airgap.tar.gz` (and `cosign.pub`, if key-signed) to the target host via +your approved medium (physical media, one-way diode, sanctioned transfer host). + +## 3. Verify + install (air-gapped host) + +```bash +tar xzf airgap.tar.gz -C bundle && cd bundle + +# Standalone installer — needs only docker + helm (+ sha256sum / cosign): +./install.sh --namespace agentcontextos --key cosign.pub +``` + +`install.sh`: + +1. `sha256sum -c SHA256SUMS` — verifies every payload file (the hard integrity + gate; works with no network and no cosign). +2. cosign verifies `SHA256SUMS.sig` when `--key` is given (key-based). A *keyless* + signature is advisory — verify it manually with your + `--certificate-identity` / `--certificate-oidc-issuer` policy. +3. `docker load -i images.tar` — loads every image onto the node. +4. `helm upgrade --install rag-platform chart/*.tgz -n --create-namespace -f rag.yaml`. + +Useful flags: `--verify-only` (check integrity, don't install), `--skip-verify` +(not recommended), `--release `. + +On Windows hosts use `./install.ps1 -Namespace agentcontextos -Key cosign.pub`. + +## 4. Post-install + +- Point the chart at your in-network backends (Postgres/pgvector, Redis, Qdrant, + Elasticsearch, object storage) via `rag.yaml` / `--set backends.*`. The bundle + ships those backend *images* (load them the same way) but the chart expects them + as external services — bring up your own StatefulSets or managed instances and set + their host/port. +- Configure SSO / SCIM, BYOK keys, quotas, and PII/ACL policy in `rag.yaml` as for a + connected install (see the per-step references). + +## Verifying without installing + +Anyone can confirm a bundle is untampered offline: + +```bash +cd bundle && sha256sum -c SHA256SUMS # every file OK? +ragctl airgap verify . # or, with the toolchain +``` + +The `content_hash` in `manifest.json` is the SHA-256 of `SHA256SUMS`, so the chain +`manifest → SHA256SUMS → every file` is fully checkable with stdlib tools. + +See [reference/airgap.md](../reference/airgap.md) and +[architecture/airgap-bundle.md](../architecture/airgap-bundle.md) for the full +design and the integrity model. diff --git a/docs/reference/airgap.md b/docs/reference/airgap.md new file mode 100644 index 0000000..e8bddad --- /dev/null +++ b/docs/reference/airgap.md @@ -0,0 +1,93 @@ +# Air-gapped bundle reference (`ragctl airgap`, Step 6.9) + +Build, inspect, verify, and install the offline air-gapped install bundle. Logic: +`ragctl.airgap` (`packages/ragctl/src/ragctl/airgap.py`). + +## CLI + +| Command | Purpose | +|---------|---------| +| `ragctl airgap build` | Assemble the bundle: all images + Helm chart + config + installer → `dist/airgap[.tar.gz]` | +| `ragctl airgap inspect ` | Print the manifest (images + digests, versions, content hash) | +| `ragctl airgap verify ` | Verify the content hash (offline) + cosign signature (with `--key`); non-zero exit on tamper | +| `ragctl airgap install ` | Offline install: verify → `docker load` → `helm upgrade --install` | + +### `build` options + +| Flag | Default | Meaning | +|------|---------|---------| +| `--out, -o` | `dist/airgap` | Bundle output directory (a `.tar.gz` is also written unless `--no-tar`) | +| `--images` | `infra/airgap/images.txt` | Pinned third-party image list | +| `--chart` | `infra/helm/rag-platform` | Helm chart directory | +| `--config, -f` | `rag.yaml` | Example config copied into the bundle | +| `--version` | chart `appVersion` | Bundle version + gateway image tag | +| `--gateway-image` | from chart values + version | Override the gateway image ref | +| `--sign` / `--cosign-key` | off | cosign `sign-blob` over `SHA256SUMS` (keyless OIDC, or key-based) | +| `--dry-run` | off | Skip `docker save` / `helm package` — a verifiable bundle minus image blobs | +| `--tar / --no-tar` | `--tar` | Also pack a `.tar.gz` | + +```bash +ragctl airgap build --version v0.1.0 --sign # full signed bundle (needs docker+helm+cosign) +ragctl airgap build --dry-run # preview / CI / test (no Docker) +ragctl airgap verify dist/airgap.tar.gz # offline integrity check +ragctl airgap verify dist/airgap.tar.gz --key cosign.pub +``` + +## Public API + +```python +from ragctl import airgap + +images = airgap.parse_image_list(open("infra/airgap/images.txt").read()) +manifest = airgap.assemble_bundle( + out_dir=Path("dist/airgap"), bundle_version="v0.1.0", + images=images, chart_dir=Path("infra/helm/rag-platform"), + config=Path("rag.yaml"), sign=False, dry_run=True, +) +airgap.pack_tarball(Path("dist/airgap"), Path("dist/airgap.tar.gz")) + +result = airgap.verify_bundle(Path("dist/airgap")) # BundleVerification +assert result.content_ok # tamper-evident, offline +``` + +- **Models** (frozen Pydantic): `BundleManifest`, `BundleImage`, `BundleFile`, + `BundleVerification`. +- **Pure**: `parse_image_list`, `sha256_file`, `scan_files`, `render_sha256sums`, + `content_hash`, `build_manifest`, `load_manifest`, `verify_bundle`, + `chart_app_version`, `gateway_image_ref`. +- **Orchestration** (stubbable seam): `save_images`, `package_chart`, `load_images`, + `install_chart`, `cosign_sign_blob`, `cosign_verify_blob`, `tool_available`, + `resolve_digest`, `assemble_bundle`, `pack_tarball`, `extract_bundle`. + +## Bundle layout + integrity + +A bundle is a `.tar.gz` of: `images.tar`, `chart/rag-platform-.tgz`, `rag.yaml`, +`install.sh` / `install.ps1`, `README.md`, `SHA256SUMS` (+ `.sig` / `.pem` when +signed), and `manifest.json`. Integrity is a standard `SHA256SUMS` (verifiable with +`sha256sum -c`) whose hash is pinned as `manifest.content_hash`; a cosign signature +over `SHA256SUMS` adds authenticity. `BundleVerification.reason` ∈ `ok` / `unsigned` +/ `keyless` / `cosign_unavailable` / `content_mismatch` / `signature_mismatch` / +`no_checksums` (only `content_mismatch` / `signature_mismatch` / `no_checksums` fail +the gate). See [architecture/airgap-bundle.md](../architecture/airgap-bundle.md). + +## Offline installer (standalone) + +The bundle ships `install.sh` / `install.ps1` that need only docker + helm on the +target host (cosign + sha256sum when present) — no Python workspace: + +```sh +./install.sh --namespace agentcontextos [--release rag-platform] [--key cosign.pub] +./install.sh --verify-only # integrity check, no load/install +``` + +## Task targets + +```bash +task airgap:build # ragctl airgap build --sign → dist/airgap.tar.gz +task airgap:build-dry # dry-run (no docker/helm/cosign) +task airgap:verify # verify dist/airgap.tar.gz (BUNDLE=… to override) +``` + +CI: `.github/workflows/release-airgap.yml` builds + keyless-signs + attaches the +tarball to the GitHub Release on `v*.*.*` tags. Operator runbook: +[guides/airgap-install.md](../guides/airgap-install.md). diff --git a/infra/airgap/images.txt b/infra/airgap/images.txt new file mode 100644 index 0000000..d77a92b --- /dev/null +++ b/infra/airgap/images.txt @@ -0,0 +1,30 @@ +# AgentContextOS — air-gapped bundle image manifest (Step 6.9) +# +# The declarative list of third-party runtime images included in the offline +# install bundle. One "repository:tag" per line; blank lines and "#" comments are +# ignored. Tags are resolved to immutable digests at build time and the resolved +# digest is recorded in the bundle's manifest.json (so a built bundle is pinned +# even though this file is human-editable). +# +# The platform's own gateway image is NOT listed here — `ragctl airgap build` +# adds it from the Helm chart's image.repository + the --version flag (or the +# chart appVersion), so the application image is always release-versioned. +# +# Edit this list to match what your deployment actually runs (drop the +# observability images for a minimal core bundle, add backends you depend on). + +# ── Core data backends ────────────────────────────────────────────────────── +pgvector/pgvector:pg16 +redis:7-alpine +qdrant/qdrant:v1.10.1 +elasticsearch:8.14.3 + +# ── Object storage (chunk blobs / tiered storage) ─────────────────────────── +minio/minio:RELEASE.2024-07-16T23-46-41Z + +# ── Observability (optional — comment out for a minimal bundle) ────────────── +otel/opentelemetry-collector-contrib:0.107.0 +prom/prometheus:v2.54.1 +grafana/grafana:11.1.4 +grafana/loki:3.1.0 +jaegertracing/all-in-one:1.59 diff --git a/packages/ragctl/src/ragctl/airgap.py b/packages/ragctl/src/ragctl/airgap.py new file mode 100644 index 0000000..03d08b4 --- /dev/null +++ b/packages/ragctl/src/ragctl/airgap.py @@ -0,0 +1,509 @@ +"""Air-gapped install bundle — build + verify logic (Step 6.9). + +Assembles a self-verifying offline install bundle: all runtime container images +(``docker save``), the packaged Helm chart (``helm package``), an example +``rag.yaml``, and a standalone offline installer — pinned by a standard +``SHA256SUMS`` file and (optionally) a **cosign** signature over it. + +The integrity design mirrors the WORM audit export (Step 6.6b): a ``content_hash`` +(SHA-256 over the canonical ``SHA256SUMS``) is *always* present for +tamper-evidence and is verifiable with nothing but ``sha256sum`` — even on an +air-gapped host with no network and no cosign. A cosign signature over +``SHA256SUMS`` adds authenticity when present. The same ``SHA256SUMS`` drives both +this module's :func:`verify_bundle` and the standalone shell installer, so the two +can never disagree. + +Pure functions (manifest building, hashing, verification) are separated from the +subprocess orchestration (docker / helm / cosign) so the core is unit-testable +with no external tooling; ``build`` additionally supports a ``dry_run`` that +produces a complete, verifiable bundle *minus* the image blobs. +""" + +from __future__ import annotations + +import hashlib +import shutil +import subprocess +import tarfile +from collections.abc import Iterable, Sequence +from datetime import UTC, datetime +from pathlib import Path + +from pydantic import BaseModel, Field + +__all__ = [ + "SCHEMA_VERSION", + "MANIFEST_NAME", + "SHA256SUMS_NAME", + "BundleImage", + "BundleFile", + "BundleManifest", + "BundleVerification", + "parse_image_list", + "sha256_file", + "scan_files", + "render_sha256sums", + "content_hash", + "build_manifest", + "load_manifest", + "verify_bundle", + "assemble_bundle", + "pack_tarball", + "extract_bundle", + "chart_app_version", + "gateway_image_ref", + "tool_available", + "resolve_digest", + "save_images", + "package_chart", + "load_images", + "install_chart", + "cosign_sign_blob", + "cosign_verify_blob", +] + +SCHEMA_VERSION = "airgap-bundle-v1" +MANIFEST_NAME = "manifest.json" +SHA256SUMS_NAME = "SHA256SUMS" +SIG_NAME = "SHA256SUMS.sig" +CERT_NAME = "SHA256SUMS.pem" +IMAGES_TAR = "images.tar" + +# Files that describe / sign the bundle are excluded from the content hash (the +# hash covers the *payload* — images, chart, config, installer — so the manifest +# and signature can wrap it without a chicken-and-egg). +_EXCLUDED_FROM_HASH = frozenset({MANIFEST_NAME, SHA256SUMS_NAME, SIG_NAME, CERT_NAME}) + + +# --------------------------------------------------------------------------- +# Models +# --------------------------------------------------------------------------- +class BundleImage(BaseModel): + """One container image included in the bundle (digest-pinned when resolvable).""" + + model_config = {"frozen": True} + + ref: str # repository:tag as listed / requested + digest: str | None = None # repo@sha256:… resolved at build time + + +class BundleFile(BaseModel): + """A payload file in the bundle, with its SHA-256 + size (posix-relative path).""" + + model_config = {"frozen": True} + + path: str + sha256: str + size_bytes: int + + +class BundleManifest(BaseModel): + """Self-describing manifest of an air-gapped bundle (Step 6.9).""" + + model_config = {"frozen": True} + + schema_version: str = SCHEMA_VERSION + bundle_version: str + app_version: str = "" + chart_version: str = "" + created_at: datetime = Field(default_factory=lambda: datetime.now(tz=UTC)) + images: tuple[BundleImage, ...] = () + files: tuple[BundleFile, ...] = () + content_hash: str = "" # sha256 over the canonical SHA256SUMS + images_saved: bool = True # False for a --dry-run bundle (no image blobs) + signed: bool = False + + +class BundleVerification(BaseModel): + """Result of verifying a bundle (mirrors AuditExportVerification, Step 6.6b).""" + + model_config = {"frozen": True} + + content_ok: bool = False # recomputed SHA256SUMS matches + ties to manifest + signed: bool = False # a cosign signature was present + verified: bool = False # content_ok AND the signature verified + files_checked: int = 0 + # "ok" | "unsigned" | "no_checksums" | "content_mismatch" + # | "signature_mismatch" | "cosign_unavailable" + reason: str = "" + + +# --------------------------------------------------------------------------- +# Pure helpers +# --------------------------------------------------------------------------- +def parse_image_list(text: str) -> list[str]: + """Parse an ``images.txt`` — one ``repo:tag`` per line, ``#`` comments ignored.""" + out: list[str] = [] + for raw in text.splitlines(): + line = raw.split("#", 1)[0].strip() + if line: + out.append(line) + return out + + +def sha256_file(path: Path) -> str: + """Return the SHA-256 hex digest of a file, read in chunks.""" + h = hashlib.sha256() + with path.open("rb") as fh: + for chunk in iter(lambda: fh.read(1024 * 1024), b""): + h.update(chunk) + return h.hexdigest() + + +def _posix_rel(path: Path, root: Path) -> str: + return path.relative_to(root).as_posix() + + +def scan_files(bundle_dir: Path) -> list[BundleFile]: + """Hash every payload file under *bundle_dir* (excluding manifest/checksums/sig). + + Returns entries sorted by posix path so the rendered ``SHA256SUMS`` is + deterministic for a given set of bytes. + """ + files: list[BundleFile] = [] + for path in sorted(bundle_dir.rglob("*")): + if not path.is_file(): + continue + rel = _posix_rel(path, bundle_dir) + if rel in _EXCLUDED_FROM_HASH: + continue + files.append(BundleFile(path=rel, sha256=sha256_file(path), size_bytes=path.stat().st_size)) + files.sort(key=lambda f: f.path) + return files + + +def render_sha256sums(files: Iterable[BundleFile]) -> str: + """Render the standard `` `` text (two spaces, ``sha256sum -c``).""" + rows = sorted(files, key=lambda f: f.path) + return "".join(f"{f.sha256} {f.path}\n" for f in rows) + + +def content_hash(sha256sums_text: str) -> str: + """The bundle content hash — SHA-256 over the canonical ``SHA256SUMS`` text.""" + return hashlib.sha256(sha256sums_text.encode("utf-8")).hexdigest() + + +def build_manifest( + bundle_dir: Path, + *, + bundle_version: str, + images: Sequence[BundleImage], + app_version: str = "", + chart_version: str = "", + images_saved: bool = True, + signed: bool = False, +) -> tuple[BundleManifest, str]: + """Scan *bundle_dir* and build its :class:`BundleManifest` + ``SHA256SUMS`` text.""" + files = scan_files(bundle_dir) + sha_text = render_sha256sums(files) + manifest = BundleManifest( + bundle_version=bundle_version, + app_version=app_version, + chart_version=chart_version, + images=tuple(images), + files=tuple(files), + content_hash=content_hash(sha_text), + images_saved=images_saved, + signed=signed, + ) + return manifest, sha_text + + +def load_manifest(bundle_dir: Path) -> BundleManifest: + """Load + validate ``manifest.json`` from a bundle directory.""" + return BundleManifest.model_validate_json((bundle_dir / MANIFEST_NAME).read_text("utf-8")) + + +def verify_bundle(bundle_dir: Path, *, cosign_pubkey: Path | None = None) -> BundleVerification: + """Verify a bundle's integrity (content hash) and, if present, its signature. + + ``content_ok`` is True iff the recomputed ``SHA256SUMS`` matches the bundled one + **and** its hash matches ``manifest.content_hash`` — so any added / removed / + edited payload file is caught with nothing but stdlib hashing. A cosign + signature over ``SHA256SUMS`` is checked when present + a key/cosign is + available; ``verified`` is ``content_ok AND signature-valid``. + """ + sums_path = bundle_dir / SHA256SUMS_NAME + if not sums_path.exists(): + return BundleVerification(reason="no_checksums") + + stored = sums_path.read_text("utf-8") + recomputed = render_sha256sums(scan_files(bundle_dir)) + files_checked = recomputed.count("\n") + manifest_hash = "" + if (bundle_dir / MANIFEST_NAME).exists(): + manifest_hash = load_manifest(bundle_dir).content_hash + content_ok = recomputed == stored and content_hash(stored) == manifest_hash + + if not content_ok: + return BundleVerification( + content_ok=False, files_checked=files_checked, reason="content_mismatch" + ) + + sig_path = bundle_dir / SIG_NAME + if not sig_path.exists(): + return BundleVerification(content_ok=True, files_checked=files_checked, reason="unsigned") + + # Signed. We auto-verify only with a supplied public key: keyless (Fulcio + # cert) verification also needs a deployment-specific identity policy + # (--certificate-identity / --certificate-oidc-issuer), which we won't guess — + # the content hash remains the hard integrity gate either way. + if cosign_pubkey is None: + return BundleVerification( + content_ok=True, signed=True, files_checked=files_checked, reason="keyless" + ) + if not tool_available("cosign"): + return BundleVerification( + content_ok=True, signed=True, files_checked=files_checked, reason="cosign_unavailable" + ) + ok = cosign_verify_blob(sums_path, sig_path, key=cosign_pubkey) + return BundleVerification( + content_ok=True, + signed=True, + verified=ok, + files_checked=files_checked, + reason="ok" if ok else "signature_mismatch", + ) + + +# --------------------------------------------------------------------------- +# Subprocess orchestration (docker / helm / cosign) — stubbable seam +# --------------------------------------------------------------------------- +def _run(cmd: Sequence[str], *, check: bool = True) -> subprocess.CompletedProcess[str]: + """Run a fixed-argv external tool (docker / helm / cosign) with no shell. + + The single subprocess chokepoint: argv is a fixed list of known executables on + PATH (never a shell string, never user-interpolated), so there is no injection + surface — the same rationale the repo applies to ``scripts/``. + """ + return subprocess.run( # noqa: S603 - fixed argv, known tools, shell=False + list(cmd), check=check, capture_output=True, text=True + ) + + +def tool_available(name: str) -> bool: + """Whether *name* (docker / helm / cosign) is on PATH.""" + return shutil.which(name) is not None + + +def resolve_digest(ref: str) -> str | None: + """Resolve an image ``repo:tag`` to its ``repo@sha256:…`` digest via docker.""" + result = _run( + ["docker", "image", "inspect", "--format", "{{index .RepoDigests 0}}", ref], + check=False, + ) + digest = result.stdout.strip() + return digest or None + + +def save_images(refs: Sequence[str], out_tar: Path) -> None: + """``docker pull`` each ref then ``docker save`` them all into one tar.""" + for ref in refs: + _run(["docker", "pull", ref]) + _run(["docker", "save", "-o", str(out_tar), *refs]) + + +def package_chart(chart_dir: Path, out_dir: Path) -> Path: + """``helm package`` *chart_dir* into *out_dir*; return the produced ``.tgz``.""" + before = set(out_dir.glob("*.tgz")) + _run(["helm", "package", str(chart_dir), "-d", str(out_dir)]) + produced = sorted(set(out_dir.glob("*.tgz")) - before) + if not produced: + raise RuntimeError(f"helm package produced no chart in {out_dir}") + return produced[0] + + +def load_images(tar: Path) -> None: + """``docker load`` images from a saved tar (offline host).""" + _run(["docker", "load", "-i", str(tar)]) + + +def install_chart( + chart_tgz: Path, + *, + release: str, + namespace: str, + values: Path | None = None, + extra_args: Sequence[str] = (), +) -> None: + """``helm upgrade --install`` the packaged chart (offline host).""" + cmd = [ + "helm", + "upgrade", + "--install", + release, + str(chart_tgz), + "--namespace", + namespace, + "--create-namespace", + ] + if values is not None: + cmd += ["-f", str(values)] + cmd += list(extra_args) + _run(cmd) + + +def cosign_sign_blob( + path: Path, *, key: Path | None, out_sig: Path, out_cert: Path | None = None +) -> None: + """Sign *path* with cosign (key-based when *key* is set, else keyless OIDC).""" + cmd = ["cosign", "sign-blob", "--yes", "--output-signature", str(out_sig)] + if key is not None: + cmd += ["--key", str(key)] + if out_cert is not None: + cmd += ["--output-certificate", str(out_cert)] + cmd.append(str(path)) + _run(cmd) + + +def cosign_verify_blob( + path: Path, sig: Path, *, key: Path | None = None, cert: Path | None = None +) -> bool: + """Verify a cosign blob signature; return True on success (never raises).""" + cmd = ["cosign", "verify-blob", "--signature", str(sig)] + if key is not None: + cmd += ["--key", str(key)] + if cert is not None: + cmd += ["--certificate", str(cert)] + cmd.append(str(path)) + return _run(cmd, check=False).returncode == 0 + + +# --------------------------------------------------------------------------- +# High-level assembly (used by the CLI) +# --------------------------------------------------------------------------- +def assemble_bundle( + *, + out_dir: Path, + bundle_version: str, + images: Sequence[str], + chart_dir: Path, + config: Path, + installer_files: Sequence[Path] = (), + app_version: str = "", + sign: bool = False, + cosign_key: Path | None = None, + dry_run: bool = False, +) -> BundleManifest: + """Assemble a bundle directory: images.tar + chart + config + installers + manifest. + + With ``dry_run`` the image blobs are skipped (``docker save`` is not invoked), + producing a complete, verifiable bundle *minus* ``images.tar`` — useful for + previewing packaging and for tests that have no Docker. Returns the written + :class:`BundleManifest`. + """ + out_dir.mkdir(parents=True, exist_ok=True) + + bundle_images = [BundleImage(ref=ref) for ref in images] + if not dry_run: + if not tool_available("docker"): + raise RuntimeError("docker is required to build an image bundle (use --dry-run)") + bundle_images = [BundleImage(ref=ref, digest=resolve_digest(ref)) for ref in images] + save_images(images, out_dir / IMAGES_TAR) + + chart_version = "" + chart_out = out_dir / "chart" + chart_out.mkdir(exist_ok=True) + if dry_run or not tool_available("helm"): + # Copy the raw chart so the bundle is still complete + verifiable. + shutil.copytree(chart_dir, chart_out / chart_dir.name, dirs_exist_ok=True) + chart_version = _chart_version(chart_dir) + else: + tgz = package_chart(chart_dir, chart_out) + chart_version = tgz.stem.rsplit("-", 1)[-1] + + if config.exists(): + shutil.copy2(config, out_dir / "rag.yaml") + for installer in installer_files: + if installer.exists(): + # Land as install.sh / install.ps1 inside the bundle (what operators run). + shutil.copy2(installer, out_dir / f"install{installer.suffix}") + + (out_dir / "README.md").write_text( + _render_readme(bundle_version, len(images), images_saved=not dry_run), "utf-8" + ) + + manifest, sha_text = build_manifest( + out_dir, + bundle_version=bundle_version, + images=bundle_images, + app_version=app_version, + chart_version=chart_version, + images_saved=not dry_run, + signed=sign, + ) + (out_dir / SHA256SUMS_NAME).write_text(sha_text, "utf-8") + (out_dir / MANIFEST_NAME).write_text(manifest.model_dump_json(indent=2) + "\n", "utf-8") + + if sign: + cosign_sign_blob( + out_dir / SHA256SUMS_NAME, + key=cosign_key, + out_sig=out_dir / SIG_NAME, + out_cert=None if cosign_key else out_dir / CERT_NAME, + ) + return manifest + + +def pack_tarball(bundle_dir: Path, out_tar: Path) -> Path: + """Pack a bundle directory into a ``.tar.gz``; return the tarball path.""" + out_tar.parent.mkdir(parents=True, exist_ok=True) + with tarfile.open(out_tar, "w:gz") as tar: + for path in sorted(bundle_dir.rglob("*")): + if path.is_file(): + tar.add(path, arcname=_posix_rel(path, bundle_dir)) + return out_tar + + +def extract_bundle(source: Path, dest: Path) -> Path: + """Resolve *source* to a bundle directory — extract a tarball, or pass a dir through.""" + if source.is_dir(): + return source + dest.mkdir(parents=True, exist_ok=True) + with tarfile.open(source, "r:*") as tar: + tar.extractall(dest, filter="data") + return dest + + +def _chart_version(chart_dir: Path) -> str: + for line in (chart_dir / "Chart.yaml").read_text("utf-8").splitlines(): + if line.startswith("version:"): + return line.split(":", 1)[1].strip().strip('"') + return "" + + +def chart_app_version(chart_dir: Path) -> str: + """Read ``appVersion`` from a chart's ``Chart.yaml`` (the gateway image tag default).""" + for line in (chart_dir / "Chart.yaml").read_text("utf-8").splitlines(): + if line.startswith("appVersion:"): + return line.split(":", 1)[1].strip().strip('"') + return "" + + +def gateway_image_ref(chart_dir: Path, version: str) -> str: + """Derive the gateway image ``repository:tag`` from the chart's ``values.yaml``. + + The application image is always release-versioned: the repository comes from + ``image.repository`` in the chart values, the tag from *version* (falling back + to the chart ``appVersion``). Returns ``""`` when no repository is configured. + """ + import yaml + + values = yaml.safe_load((chart_dir / "values.yaml").read_text("utf-8")) or {} + repo = str((values.get("image") or {}).get("repository", "")).strip() + tag = version or chart_app_version(chart_dir) + return f"{repo}:{tag}" if repo else "" + + +def _render_readme(bundle_version: str, image_count: int, *, images_saved: bool) -> str: + note = "" if images_saved else "\n> **Dry-run bundle** — image blobs are not included.\n" + return ( + f"# AgentContextOS — air-gapped install bundle `{bundle_version}`\n" + f"\n{image_count} container image(s) + the `rag-platform` Helm chart.\n{note}\n" + "## Verify (no network required)\n\n" + "```sh\nsha256sum -c SHA256SUMS\n" + "cosign verify-blob --signature SHA256SUMS.sig SHA256SUMS # if signed\n```\n\n" + "## Install (offline host with docker + helm)\n\n" + "```sh\n./install.sh --namespace agentcontextos\n```\n\n" + "See `docs/guides/airgap-install.md` for the full runbook.\n" + ) diff --git a/packages/ragctl/src/ragctl/main.py b/packages/ragctl/src/ragctl/main.py index 50c7e3a..b64ff2c 100644 --- a/packages/ragctl/src/ragctl/main.py +++ b/packages/ragctl/src/ragctl/main.py @@ -4863,6 +4863,210 @@ async def _run() -> None: asyncio.run(_run()) +# --------------------------------------------------------------------------- +# Air-gapped install bundle (Step 6.9) +# --------------------------------------------------------------------------- +airgap_app = typer.Typer( + help="Build / verify / install the offline air-gapped install bundle (Step 6.9).", + no_args_is_help=True, +) +app.add_typer(airgap_app, name="airgap") + + +@airgap_app.command("build") +def airgap_build( + out: Path = typer.Option(Path("dist/airgap"), "--out", "-o", help="Bundle output directory."), + images: Path = typer.Option( + Path("infra/airgap/images.txt"), "--images", help="Pinned third-party image list." + ), + chart: Path = typer.Option( + Path("infra/helm/rag-platform"), "--chart", help="Helm chart directory." + ), + config: Path = typer.Option(Path("rag.yaml"), "--config", "-f", help="Example rag.yaml."), + version: str = typer.Option("", "--version", help="Bundle version (default: chart appVersion)"), + gateway_image: str = typer.Option( + "", "--gateway-image", help="Gateway image ref (default: chart values + version)." + ), + sign: bool = typer.Option(False, "--sign", help="cosign sign-blob over SHA256SUMS."), + cosign_key: Path | None = typer.Option( + None, "--cosign-key", help="cosign private key (omit for keyless OIDC)." + ), + dry_run: bool = typer.Option( + False, "--dry-run", help="Skip docker save / helm package (no image blobs)." + ), + tar: bool = typer.Option(True, "--tar/--no-tar", help="Also pack a .tar.gz."), +) -> None: + """Assemble the offline install bundle: all images + Helm chart + config + installer. + + Resolves the gateway image from the chart values (release-versioned) and the + third-party images from ``infra/airgap/images.txt``, saves them, packages the + chart, writes a ``SHA256SUMS`` + ``manifest.json``, and (with ``--sign``) + cosign-signs the checksums. ``--dry-run`` produces a complete, verifiable + bundle minus the image blobs — runnable with no Docker. + + Example:: + + ragctl airgap build --version v0.1.0 --sign + """ + from ragctl import airgap + + version = version or airgap.chart_app_version(chart) + bundle_version = version or "0.0.0" + gw = gateway_image or airgap.gateway_image_ref(chart, version) + image_list = ([gw] if gw else []) + airgap.parse_image_list(images.read_text("utf-8")) + if not image_list: + typer.echo("ERROR: no images to bundle (empty images.txt + no gateway image)", err=True) + raise typer.Exit(1) + + installers = [Path("scripts/airgap_install.sh"), Path("scripts/airgap_install.ps1")] + try: + manifest = airgap.assemble_bundle( + out_dir=out, + bundle_version=bundle_version, + images=image_list, + chart_dir=chart, + config=config, + installer_files=installers, + app_version=version, + sign=sign, + cosign_key=cosign_key, + dry_run=dry_run, + ) + except (RuntimeError, OSError) as exc: + typer.echo(f"ERROR: {exc}", err=True) + raise typer.Exit(1) # noqa: B904 + + saved = "saved" if manifest.images_saved else "dry-run, not saved" + typer.echo(f"\nair-gapped bundle — {bundle_version}") + typer.echo("─" * 64) + typer.echo(f" out: {out}") + typer.echo(f" images: {len(manifest.images)} ({saved})") + typer.echo(f" chart version: {manifest.chart_version}") + typer.echo(f" files: {len(manifest.files)}") + typer.echo(f" content_hash: {manifest.content_hash}") + typer.echo(f" signed: {manifest.signed}") + if tar: + out_tar = out.with_suffix(".tar.gz") + airgap.pack_tarball(out, out_tar) + typer.echo(f" tarball: {out_tar}") + + +@airgap_app.command("inspect") +def airgap_inspect( + bundle: Path = typer.Argument(..., help="Bundle directory or .tar.gz."), +) -> None: + """Print the manifest of a bundle (images, chart, files, content hash).""" + import tempfile + + from ragctl import airgap + + with tempfile.TemporaryDirectory() as tmp: + bundle_dir = airgap.extract_bundle(bundle, Path(tmp) / "bundle") + manifest = airgap.load_manifest(bundle_dir) + typer.echo(f"bundle_version: {manifest.bundle_version}") + typer.echo(f"app_version: {manifest.app_version}") + typer.echo(f"chart_version: {manifest.chart_version}") + typer.echo(f"created_at: {manifest.created_at.isoformat()}") + typer.echo(f"content_hash: {manifest.content_hash}") + typer.echo(f"images_saved: {manifest.images_saved}") + typer.echo(f"signed: {manifest.signed}") + typer.echo(f"images ({len(manifest.images)}):") + for img in manifest.images: + suffix = f" ({img.digest.split('@')[-1][:19]}…)" if img.digest else "" + typer.echo(f" - {img.ref}{suffix}") + typer.echo(f"files: {len(manifest.files)}") + + +@airgap_app.command("verify") +def airgap_verify( + bundle: Path = typer.Argument(..., help="Bundle directory or .tar.gz."), + key: Path | None = typer.Option(None, "--key", help="cosign public key for verification."), +) -> None: + """Verify a bundle's content hash (offline) + cosign signature (when present). + + Exits non-zero if the payload was tampered, or if it is signed but the + signature does not verify. Content verification needs no network / cosign. + """ + import tempfile + + from ragctl import airgap + + with tempfile.TemporaryDirectory() as tmp: + bundle_dir = airgap.extract_bundle(bundle, Path(tmp) / "bundle") + result = airgap.verify_bundle(bundle_dir, cosign_pubkey=key) + typer.echo(f"content_ok: {result.content_ok}") + typer.echo(f"files_checked: {result.files_checked}") + typer.echo(f"signed: {result.signed}") + typer.echo(f"verified: {result.verified}") + typer.echo(f"reason: {result.reason}") + # The content hash is the hard gate; a key-based signature failure also + # fails. Unsigned / keyless / cosign-unavailable pass (content verified). + if not result.content_ok or result.reason in { + "content_mismatch", + "signature_mismatch", + "no_checksums", + }: + raise typer.Exit(1) + + +@airgap_app.command("install") +def airgap_install( + bundle: Path = typer.Argument(..., help="Bundle directory or .tar.gz."), + namespace: str = typer.Option("agentcontextos", "--namespace", "-n", help="Target namespace."), + release: str = typer.Option("rag-platform", "--release", help="Helm release name."), + key: Path | None = typer.Option(None, "--key", help="cosign public key for verification."), + skip_verify: bool = typer.Option(False, "--skip-verify", help="Skip integrity verification."), + dry_run: bool = typer.Option(False, "--dry-run", help="Verify + report; don't load/install."), +) -> None: + """Offline-install a bundle on an air-gapped host: verify → docker load → helm install. + + Requires ``docker`` + ``helm`` on the host. Prefer the standalone + ``install.sh`` shipped *inside* the bundle for hosts without ``ragctl``; this + command is the same flow for operators who have the toolchain. + """ + import tempfile + + from ragctl import airgap + + with tempfile.TemporaryDirectory() as tmp: + bundle_dir = airgap.extract_bundle(bundle, Path(tmp) / "bundle") + if not skip_verify: + result = airgap.verify_bundle(bundle_dir, cosign_pubkey=key) + if not result.content_ok or result.reason in { + "content_mismatch", + "signature_mismatch", + "no_checksums", + }: + typer.echo(f"ERROR: bundle verification failed (reason={result.reason})", err=True) + raise typer.Exit(1) + typer.echo(f"verified: content_ok={result.content_ok} reason={result.reason}") + + manifest = airgap.load_manifest(bundle_dir) + if not manifest.images_saved: + typer.echo("ERROR: dry-run bundle has no image blobs to load", err=True) + raise typer.Exit(1) + if dry_run: + typer.echo(f"dry-run: would load {len(manifest.images)} images + install '{release}'") + return + if not (airgap.tool_available("docker") and airgap.tool_available("helm")): + typer.echo("ERROR: docker + helm are required to install", err=True) + raise typer.Exit(1) + + airgap.load_images(bundle_dir / "images.tar") + charts = sorted((bundle_dir / "chart").glob("*.tgz")) + if not charts: + typer.echo("ERROR: no packaged chart found in the bundle", err=True) + raise typer.Exit(1) + cfg = bundle_dir / "rag.yaml" + airgap.install_chart( + charts[0], + release=release, + namespace=namespace, + values=cfg if cfg.exists() else None, + ) + typer.echo(f"installed release '{release}' in namespace '{namespace}'") + + # --------------------------------------------------------------------------- # Scaffold sub-apps — print a "delivered in Step X.Y" notice and exit 0. # diff --git a/packages/ragctl/tests/test_airgap.py b/packages/ragctl/tests/test_airgap.py new file mode 100644 index 0000000..a41609c --- /dev/null +++ b/packages/ragctl/tests/test_airgap.py @@ -0,0 +1,261 @@ +"""Tests for the air-gapped install bundle — ragctl airgap (Step 6.9). + +Exercises the pure bundle logic (manifest, SHA256SUMS, content hash, tamper +detection) + the CLI on a ``--dry-run`` bundle (no Docker / Helm / cosign), and +the cosign signature paths via a stubbed subprocess seam. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +from ragctl import airgap +from ragctl.main import app +from typer.testing import CliRunner + +runner = CliRunner() + +REPO_ROOT = Path(__file__).resolve().parents[3] +CHART = REPO_ROOT / "infra/helm/rag-platform" +IMAGES = REPO_ROOT / "infra/airgap/images.txt" +CONFIG = REPO_ROOT / "rag.yaml" + + +def _fake_chart(tmp: Path) -> Path: + chart = tmp / "mychart" + chart.mkdir() + (chart / "Chart.yaml").write_text('version: 1.2.3\nappVersion: "9.9.9"\n', encoding="utf-8") + (chart / "values.yaml").write_text("image:\n repository: ghcr.io/acme/gw\n", encoding="utf-8") + return chart + + +def _dry_bundle(tmp: Path, *, images=("a:1", "b:2"), sign: bool = False) -> Path: + out = tmp / "bundle" + airgap.assemble_bundle( + out_dir=out, + bundle_version="v1.0.0", + images=list(images), + chart_dir=_fake_chart(tmp), + config=tmp / "missing-rag.yaml", # absent → not copied + sign=sign, + dry_run=True, + ) + return out + + +# --------------------------------------------------------------------------- +# Pure helpers +# --------------------------------------------------------------------------- +def test_parse_image_list_strips_comments_and_blanks() -> None: + text = "# header\n\nrepo:tag # inline\n \nother/img:v2\n" + assert airgap.parse_image_list(text) == ["repo:tag", "other/img:v2"] + + +def test_sha256_file(tmp_path: Path) -> None: + f = tmp_path / "x" + f.write_bytes(b"hello") + # sha256("hello") + assert airgap.sha256_file(f) == ( + "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824" + ) + + +def test_render_sha256sums_is_sorted_and_two_spaced() -> None: + files = [ + airgap.BundleFile(path="z", sha256="ff", size_bytes=1), + airgap.BundleFile(path="a", sha256="aa", size_bytes=1), + ] + text = airgap.render_sha256sums(files) + assert text == "aa a\nff z\n" + assert airgap.content_hash(text) == airgap.content_hash(text) # deterministic + + +def test_chart_helpers_read_real_chart() -> None: + assert airgap.chart_app_version(CHART) == "0.1.0" + ref = airgap.gateway_image_ref(CHART, "v0.1.0") + assert ref == "ghcr.io/officialcodework/agentcontextos/rag-gateway:v0.1.0" + + +def test_gateway_image_ref_falls_back_to_app_version(tmp_path: Path) -> None: + assert airgap.gateway_image_ref(_fake_chart(tmp_path), "") == "ghcr.io/acme/gw:9.9.9" + + +# --------------------------------------------------------------------------- +# Assemble + verify +# --------------------------------------------------------------------------- +def test_dry_run_bundle_verifies(tmp_path: Path) -> None: + bundle = _dry_bundle(tmp_path) + manifest = airgap.load_manifest(bundle) + assert manifest.images_saved is False + assert manifest.chart_version == "1.2.3" + assert [i.ref for i in manifest.images] == ["a:1", "b:2"] + assert not (bundle / airgap.IMAGES_TAR).exists() # dry-run: no image blobs + + result = airgap.verify_bundle(bundle) + assert result.content_ok is True + assert result.reason == "unsigned" + assert result.files_checked == len(manifest.files) + + +def test_verify_detects_payload_tamper(tmp_path: Path) -> None: + bundle = _dry_bundle(tmp_path) + (bundle / "README.md").write_text("tampered", encoding="utf-8") + result = airgap.verify_bundle(bundle) + assert result.content_ok is False + assert result.reason == "content_mismatch" + + +def test_verify_detects_added_file(tmp_path: Path) -> None: + bundle = _dry_bundle(tmp_path) + (bundle / "sneaky.txt").write_text("extra payload", encoding="utf-8") + assert airgap.verify_bundle(bundle).reason == "content_mismatch" + + +def test_verify_no_checksums(tmp_path: Path) -> None: + bundle = _dry_bundle(tmp_path) + (bundle / airgap.SHA256SUMS_NAME).unlink() + assert airgap.verify_bundle(bundle).reason == "no_checksums" + + +def test_tarball_round_trip(tmp_path: Path) -> None: + bundle = _dry_bundle(tmp_path) + tar = airgap.pack_tarball(bundle, tmp_path / "out.tar.gz") + extracted = airgap.extract_bundle(tar, tmp_path / "ex") + assert airgap.verify_bundle(extracted).content_ok is True + + +# --------------------------------------------------------------------------- +# Signature paths (cosign stubbed) +# --------------------------------------------------------------------------- +def test_signed_keyless_reports_keyless(tmp_path: Path) -> None: + bundle = _dry_bundle(tmp_path) + (bundle / airgap.SIG_NAME).write_text("sig", encoding="utf-8") + # No key supplied → keyless: content verified, signature not auto-checked. + result = airgap.verify_bundle(bundle) + assert result.signed is True and result.verified is False + assert result.reason == "keyless" + + +def test_signed_key_verified(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + bundle = _dry_bundle(tmp_path) + (bundle / airgap.SIG_NAME).write_text("sig", encoding="utf-8") + monkeypatch.setattr(airgap, "tool_available", lambda name: True) + monkeypatch.setattr(airgap, "cosign_verify_blob", lambda *a, **k: True) + result = airgap.verify_bundle(bundle, cosign_pubkey=tmp_path / "key.pub") + assert result.verified is True and result.reason == "ok" + + +def test_signed_key_mismatch(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + bundle = _dry_bundle(tmp_path) + (bundle / airgap.SIG_NAME).write_text("sig", encoding="utf-8") + monkeypatch.setattr(airgap, "tool_available", lambda name: True) + monkeypatch.setattr(airgap, "cosign_verify_blob", lambda *a, **k: False) + result = airgap.verify_bundle(bundle, cosign_pubkey=tmp_path / "key.pub") + assert result.verified is False and result.reason == "signature_mismatch" + + +def test_sign_invokes_cosign(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + calls: list[Path] = [] + monkeypatch.setattr(airgap, "cosign_sign_blob", lambda path, **k: calls.append(path) or None) + manifest = airgap.assemble_bundle( + out_dir=tmp_path / "b", + bundle_version="v1", + images=["a:1"], + chart_dir=_fake_chart(tmp_path), + config=tmp_path / "none.yaml", + sign=True, + dry_run=True, + ) + assert manifest.signed is True + assert calls and calls[0].name == airgap.SHA256SUMS_NAME + + +# --------------------------------------------------------------------------- +# CLI +# --------------------------------------------------------------------------- +def test_cli_build_inspect_verify(tmp_path: Path) -> None: + out = tmp_path / "bundle" + res = runner.invoke( + app, + [ + "airgap", + "build", + "--dry-run", + "--out", + str(out), + "--images", + str(IMAGES), + "--chart", + str(CHART), + "--config", + str(CONFIG), + "--version", + "v0.1.0", + "--no-tar", + ], + ) + assert res.exit_code == 0, res.output + assert "air-gapped bundle — v0.1.0" in res.output + assert "dry-run, not saved" in res.output + + ins = runner.invoke(app, ["airgap", "inspect", str(out)]) + assert ins.exit_code == 0, ins.output + assert "rag-gateway:v0.1.0" in ins.output + + ver = runner.invoke(app, ["airgap", "verify", str(out)]) + assert ver.exit_code == 0, ver.output + assert "content_ok: True" in ver.output + + +def test_cli_verify_tamper_exits_nonzero(tmp_path: Path) -> None: + out = tmp_path / "bundle" + runner.invoke( + app, + [ + "airgap", + "build", + "--dry-run", + "--out", + str(out), + "--images", + str(IMAGES), + "--chart", + str(CHART), + "--config", + str(CONFIG), + "--version", + "v0.1.0", + "--no-tar", + ], + ) + (out / "rag.yaml").write_text("# tampered\n", encoding="utf-8") + ver = runner.invoke(app, ["airgap", "verify", str(out)]) + assert ver.exit_code == 1 + assert "content_mismatch" in ver.output + + +def test_cli_install_rejects_dry_run_bundle(tmp_path: Path) -> None: + out = tmp_path / "bundle" + runner.invoke( + app, + [ + "airgap", + "build", + "--dry-run", + "--out", + str(out), + "--images", + str(IMAGES), + "--chart", + str(CHART), + "--config", + str(CONFIG), + "--version", + "v0.1.0", + "--no-tar", + ], + ) + res = runner.invoke(app, ["airgap", "install", str(out)]) + assert res.exit_code == 1 + assert "no image blobs" in res.output diff --git a/scripts/airgap_install.ps1 b/scripts/airgap_install.ps1 new file mode 100644 index 0000000..0fae866 --- /dev/null +++ b/scripts/airgap_install.ps1 @@ -0,0 +1,75 @@ +# AgentContextOS — air-gapped offline installer (Step 6.9), Windows / PowerShell. +# +# Runs from inside an extracted bundle directory. Needs only docker + helm on the +# host (cosign is used when available). No network, no Python workspace. +# +# Usage: ./install.ps1 [-Namespace NS] [-Release NAME] [-Key cosign.pub] +# [-SkipVerify] [-VerifyOnly] +[CmdletBinding()] +param( + [string]$Namespace = "agentcontextos", + [string]$Release = "rag-platform", + [string]$Key = "", + [switch]$SkipVerify, + [switch]$VerifyOnly +) + +$ErrorActionPreference = "Stop" +Set-Location -Path $PSScriptRoot + +function Test-Sha256Sums { + # Verify every payload file listed in SHA256SUMS against its recorded digest. + if (-not (Test-Path "SHA256SUMS")) { throw "SHA256SUMS missing — cannot verify integrity" } + foreach ($line in Get-Content "SHA256SUMS") { + if ([string]::IsNullOrWhiteSpace($line)) { continue } + $expected, $relpath = $line -split '\s+', 2 + $relpath = $relpath.Trim() + if (-not (Test-Path $relpath)) { throw "missing bundle file: $relpath" } + $actual = (Get-FileHash -Algorithm SHA256 -Path $relpath).Hash.ToLower() + if ($actual -ne $expected.ToLower()) { throw "checksum mismatch: $relpath" } + } +} + +function Test-Signature { + if (-not (Test-Path "SHA256SUMS.sig")) { Write-Host " (unsigned bundle — content hash only)"; return } + if (-not (Get-Command cosign -ErrorAction SilentlyContinue)) { + Write-Warning "signature present but cosign not found — skipping signature check"; return + } + if ($Key) { + cosign verify-blob --key $Key --signature SHA256SUMS.sig SHA256SUMS + } elseif (Test-Path "SHA256SUMS.pem") { + Write-Host " keyless signature present; verify with your identity policy:" + Write-Host " cosign verify-blob --certificate SHA256SUMS.pem ``" + Write-Host " --certificate-identity --certificate-oidc-issuer ``" + Write-Host " --signature SHA256SUMS.sig SHA256SUMS" + } else { + Write-Warning "signature present but no -Key / SHA256SUMS.pem to verify with" + } +} + +if (-not $SkipVerify) { + Write-Host "==> verifying bundle integrity" + Test-Sha256Sums + Test-Signature + Write-Host " integrity OK" +} + +if ($VerifyOnly) { Write-Host "verify-only: not loading images or installing."; exit 0 } + +if (-not (Get-Command docker -ErrorAction SilentlyContinue)) { throw "docker is required" } +if (-not (Get-Command helm -ErrorAction SilentlyContinue)) { throw "helm is required" } +if (-not (Test-Path "images.tar")) { throw "images.tar missing (a --dry-run bundle cannot be installed)" } + +Write-Host "==> loading container images" +docker load -i images.tar + +$chart = Get-ChildItem -Path "chart" -Filter "*.tgz" | Select-Object -First 1 +if (-not $chart) { throw "no packaged chart (chart/*.tgz) found in the bundle" } + +$helmArgs = @("upgrade", "--install", $Release, $chart.FullName, "--namespace", $Namespace, "--create-namespace") +if (Test-Path "rag.yaml") { $helmArgs += @("-f", "rag.yaml") } + +Write-Host "==> helm $($helmArgs -join ' ')" +helm @helmArgs + +Write-Host "==> installed release '$Release' in namespace '$Namespace'" diff --git a/scripts/airgap_install.sh b/scripts/airgap_install.sh new file mode 100755 index 0000000..197800e --- /dev/null +++ b/scripts/airgap_install.sh @@ -0,0 +1,104 @@ +#!/usr/bin/env bash +# AgentContextOS — air-gapped offline installer (Step 6.9). +# +# Runs from inside an extracted bundle directory. Needs only docker + helm on the +# host (cosign + sha256sum/shasum are used when available). No network, no Python +# workspace — this is the genuinely air-gap-appropriate entry point; `ragctl +# airgap install` is the same flow for operators who already have the toolchain. +# +# Usage: ./install.sh [--namespace NS] [--release NAME] [--key cosign.pub] +# [--skip-verify] [--verify-only] +set -euo pipefail + +NAMESPACE="agentcontextos" +RELEASE="rag-platform" +PUBKEY="" +SKIP_VERIFY=0 +VERIFY_ONLY=0 + +while [ $# -gt 0 ]; do + case "$1" in + --namespace|-n) NAMESPACE="$2"; shift 2 ;; + --release) RELEASE="$2"; shift 2 ;; + --key) PUBKEY="$2"; shift 2 ;; + --skip-verify) SKIP_VERIFY=1; shift ;; + --verify-only) VERIFY_ONLY=1; shift ;; + -h|--help) + sed -n '2,12p' "$0"; exit 0 ;; + *) echo "unknown argument: $1" >&2; exit 2 ;; + esac +done + +DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +cd "$DIR" + +sha_check() { + # Verify every payload file against SHA256SUMS (Linux sha256sum / macOS shasum). + if command -v sha256sum >/dev/null 2>&1; then + sha256sum -c SHA256SUMS + elif command -v shasum >/dev/null 2>&1; then + shasum -a 256 -c SHA256SUMS + else + echo "ERROR: neither sha256sum nor shasum found — cannot verify integrity" >&2 + return 1 + fi +} + +verify_signature() { + [ -f SHA256SUMS.sig ] || { echo " (unsigned bundle — content hash only)"; return 0; } + if ! command -v cosign >/dev/null 2>&1; then + echo " WARN: signature present but cosign not found — skipping signature check" >&2 + return 0 + fi + if [ -n "$PUBKEY" ]; then + # Key-based (the recommended air-gap path — distribute the public key OOB). + cosign verify-blob --key "$PUBKEY" --signature SHA256SUMS.sig SHA256SUMS + elif [ -f SHA256SUMS.pem ]; then + # Keyless (Fulcio cert). cosign 2.x also needs your --certificate-identity / + # --certificate-oidc-issuer policy, which is deployment-specific, so we don't + # guess it — verify manually, e.g.: + echo " keyless signature present; verify with your identity policy:" >&2 + echo " cosign verify-blob --certificate SHA256SUMS.pem \\" >&2 + echo " --certificate-identity --certificate-oidc-issuer \\" >&2 + echo " --signature SHA256SUMS.sig SHA256SUMS" >&2 + else + echo " WARN: signature present but no --key / SHA256SUMS.pem to verify with" >&2 + fi +} + +if [ "$SKIP_VERIFY" -eq 0 ]; then + echo "==> verifying bundle integrity" + sha_check + verify_signature + echo " integrity OK" +fi + +if [ "$VERIFY_ONLY" -eq 1 ]; then + echo "verify-only: not loading images or installing." + exit 0 +fi + +command -v docker >/dev/null 2>&1 || { echo "ERROR: docker is required" >&2; exit 1; } +command -v helm >/dev/null 2>&1 || { echo "ERROR: helm is required" >&2; exit 1; } + +if [ ! -f images.tar ]; then + echo "ERROR: images.tar missing (a --dry-run bundle cannot be installed)" >&2 + exit 1 +fi + +echo "==> loading container images" +docker load -i images.tar + +CHART="$(ls chart/*.tgz 2>/dev/null | head -n1 || true)" +if [ -z "$CHART" ]; then + echo "ERROR: no packaged chart (chart/*.tgz) found in the bundle" >&2 + exit 1 +fi + +HELM_ARGS=(upgrade --install "$RELEASE" "$CHART" --namespace "$NAMESPACE" --create-namespace) +[ -f rag.yaml ] && HELM_ARGS+=(-f rag.yaml) + +echo "==> helm ${HELM_ARGS[*]}" +helm "${HELM_ARGS[@]}" + +echo "==> installed release '$RELEASE' in namespace '$NAMESPACE'"