feat(TC-887): at-rest key escrow crate and ceremony CLI by samgbafa · Pull Request #325 · TinyCloudLabs/tinycloud-node

samgbafa · 2026-10-17T06:27:24Z

TC-887 (S1, PR 1 of 3). Adds the escrow format and the ceremony/recovery tool. The node binary doesn't use them yet, so behaviour is unchanged. Merge only after the current main release is cut (TC-887 merge order).

What's added

tinycloud-at-rest-escrow is a node workspace member; nothing uses it yet. It provides:
- a 2-of-3 Shamir split of the at-rest root key (vsss-rs 5.4.0, GF(256));
- each share age-sealed (age 0.12.1) to a pinned recipient. Accepted kinds are age1yubikey1 (the piv-p256 format that released age-plugin-yubikey 0.5.1 handles) and age1tag1 (the tool warns that 0.5.1 can't decrypt age1tag1). X25519 is accepted only with an explicit software-recipient policy;
- the bundle and its fingerprint (EFP), share commitments and the canary;
- the ARK-MACed verification record;
- the recovery binding and per-boot recovery identity;
- version-first parsing of the bundle, the record and the recovery response.
tinycloud-at-rest-cli (tinycloud-at-rest binary) is its own Cargo workspace with its own Cargo.lock. It is excluded from the node workspace and has its own CI job: fmt, clippy, tests, and interop with a sha256-pinned stock age v1.3.2. Commands:
- recipients, self-test, recipient-key, nonce;
- inspect, extract-share, check-share;
- verify-compose, verify-node, share-wrap, record-efp.
verify-node and share-wrap check, in order:
- local checks first (bundle/EFP, nonce, binding);
- full DCAP verification with collateral (dcap-qvl 0.7.0);
- TCB status as a fixed class, with pinned advisories for the hardening classes;
- the debug bit;
- MRTD and RTMR0-2;
- RTMR3, replayed from the event log, with the identity-event rules;
- REPORTDATA.
docs/at-rest-escrow.md: formats, commands, exit codes, and what is checked before a share is released.
Build plumbing: the Dockerfile copies the escrow crate (a workspace member), and release-plz.toml marks it release = false.

Node dependency graph

cargo tree -p tinycloud-node -e features is identical to main.
The root lockfile only adds the escrow crate's dependencies. No existing locked version changes.
Workspace-wide feature unification adds only additive features to 11 shared crates (alloc/std/default/ecdh/derive/const-generics); serde_json is unchanged.

Tests (local, --locked)

Escrow crate: 32 tests (including the strict compose reader).
CLI: 21 unit + 7 binary-level + 2 stock-age interop.
RUSTFLAGS=-Dwarnings cargo clippy --workspace --all-targets is clean.
cargo test --workspace --no-fail-fast against a local PG14 matches main exactly. On both main and this head, the same 5 environment-specific tests fail: the TC-320 collation test and 4 postgres_pool_timeouts blackhole tests.
CI is the authoritative run.

Next in the stack: #326 (core keyring/store/migration, DataKeyProvider), then #327 (node wiring: config, gate, locked mode, recovery routes). Both stay off by default. This branch includes current main (merged in).

tinycloud_at_rest_escrow::compose reads app-compose.json strictly. The CLI's verify-compose and the node use the same reader. It requires:

runner is docker-compose;
the compose text is newlines and printable ASCII only, since YAML parsers disagree on Unicode line breaks;
one document, with no anchors, aliases, merge keys, tags or duplicate keys;
every service has an image pinned as <name>[:tag]@sha256:<64 hex>, with no build, extends or include;
the at-rest escrow recipients and pin, if set, are literal environment values.

verify-compose prints those escrow values and every other app-compose setting in full. All untrusted text the CLI echoes is escaped. The production compose's one non-ASCII comment character is replaced.

samgbafa · 2026-10-17T06:40:07Z

REQUEST_CHANGES: see TC-887 (security review of head 1b300a2)

samgbafa · 2026-10-17T06:53:32Z

REQUEST_CHANGES: see TC-887 (correctness/operability review of head 1b300a2)

@sha256

Workspace isolation: - tinycloud-at-rest-cli is excluded from the node workspace, with its own Cargo.lock and CI job (fmt, clippy, tests incl. stock age v1.3.2 interop, sha256-pinned). dcap-qvl no longer enters the node's dependency graph: `cargo tree -p tinycloud-node -e features` is identical to main and the root lockfile only adds the escrow crate's dependencies. Escrow crate: - Accept age1yubikey1 (piv-p256) recipients, the format released age-plugin-yubikey 0.5.1 prints and decrypts, sealed exactly as the plugin's wrap_file_key; age1tag1 stays supported. - RecipientPolicy: X25519 software recipients are refused unless explicitly allowed (tests and self-host). - Bundle, record and recovery response are parsed version-first; RecoveryInfo gains a version. Clearer share/commitment errors. - forbid(unsafe_code). CLI: - TCB status is a fixed class: UpToDate passes; SWHardeningNeeded / ConfigurationNeeded / ConfigurationAndSWHardeningNeeded only if allowed and every advisory ID is pinned; OutOfDate*, TDRelaunch*, Revoked and unknown never pass. Status, advisories and collateral dates are printed. - Local checks (stale bundle, nonce, binding) run before any PCCS/DCAP work; PCCS fetch has a 60 s timeout. - --expect-efp and --allow-software on every bundle command; exit codes 0 OK / 1 REFUSED / 2 usage / 3 ERROR; verify-node shows node state and shares held; verify-record renamed record-efp and marked NOT AUTHENTICATED. - New: self-test (ceremony step 0 round trip with the TEE's sealing code), recipient-key (compare with a YubiKey attestation), and verify-compose (compose_hash + every image pinned by @sha256:). - Tests: every TCB class from a genuine quote and collateral, identity event ordering/duplication, v2 preimage, malformed logs, binary-level exit codes and messages, stock age interop (X25519 round trip and a Go-produced p256tag vector).

samgbafa · 2026-10-17T07:35:36Z

REQUEST_CHANGES: see TC-887 (security review of head 8d124b0)

samgbafa · 2026-10-17T07:35:54Z

APPROVE: see TC-887 (correctness re-review of head 8d124b0; Linear comment defc9998)

@sha256

…#325) - verify-compose parses the compose YAML with yaml-rust2 (=0.13.0, CLI workspace only) and accepts a strict subset: one document, no anchors, aliases, merge keys, tags or duplicate keys, a services mapping without include, and every service with an `image` of exactly `<name>[:tag]@sha256:<64 hex>` and no build/extends. It also lists the other app-compose settings the hash covers. The four reported bypasses (comment, flow mapping, quoted key, build) are tests. - Checked-in piv-p256 vector produced by the released age-plugin-yubikey 0.5.1; the test identity now rejects what the plugin's from_stanza rejects (argument lengths, uncompressed epk, body length). - Warn on age1tag1 holders (0.5.1 can't decrypt p256tag). - Version-first parsing keeps duplicate-key rejection; RecoveryInfo mode and state are validated. - CLI: self-test creates --out; malformed bundle/recovery/record JSON is exit 3 (ERROR); a hardening TCB class without advisories is refused.

@sha256

Workspace isolation: - tinycloud-at-rest-cli is excluded from the node workspace, with its own Cargo.lock and CI job (fmt, clippy, tests incl. stock age v1.3.2 interop, sha256-pinned). dcap-qvl no longer enters the node's dependency graph: `cargo tree -p tinycloud-node -e features` is identical to main and the root lockfile only adds the escrow crate's dependencies. Escrow crate: - Accept age1yubikey1 (piv-p256) recipients, the format released age-plugin-yubikey 0.5.1 prints and decrypts, sealed exactly as the plugin's wrap_file_key; age1tag1 stays supported. - RecipientPolicy: X25519 software recipients are refused unless explicitly allowed (tests and self-host). - Bundle, record and recovery response are parsed version-first; RecoveryInfo gains a version. Clearer share/commitment errors. - forbid(unsafe_code). CLI: - TCB status is a fixed class: UpToDate passes; SWHardeningNeeded / ConfigurationNeeded / ConfigurationAndSWHardeningNeeded only if allowed and every advisory ID is pinned; OutOfDate*, TDRelaunch*, Revoked and unknown never pass. Status, advisories and collateral dates are printed. - Local checks (stale bundle, nonce, binding) run before any PCCS/DCAP work; PCCS fetch has a 60 s timeout. - --expect-efp and --allow-software on every bundle command; exit codes 0 OK / 1 REFUSED / 2 usage / 3 ERROR; verify-node shows node state and shares held; verify-record renamed record-efp and marked NOT AUTHENTICATED. - New: self-test (ceremony step 0 round trip with the TEE's sealing code), recipient-key (compare with a YubiKey attestation), and verify-compose (compose_hash + every image pinned by @sha256:). - Tests: every TCB class from a genuine quote and collateral, identity event ordering/duplication, v2 preimage, malformed logs, binary-level exit codes and messages, stock age interop (X25519 round trip and a Go-produced p256tag vector).

@sha256

…#325) - verify-compose parses the compose YAML with yaml-rust2 (=0.13.0, CLI workspace only) and accepts a strict subset: one document, no anchors, aliases, merge keys, tags or duplicate keys, a services mapping without include, and every service with an `image` of exactly `<name>[:tag]@sha256:<64 hex>` and no build/extends. It also lists the other app-compose settings the hash covers. The four reported bypasses (comment, flow mapping, quoted key, build) are tests. - Checked-in piv-p256 vector produced by the released age-plugin-yubikey 0.5.1; the test identity now rejects what the plugin's from_stanza rejects (argument lengths, uncompressed epk, body length). - Warn on age1tag1 holders (0.5.1 can't decrypt p256tag). - Version-first parsing keeps duplicate-key rejection; RecoveryInfo mode and state are validated. - CLI: self-test creates --out; malformed bundle/recovery/record JSON is exit 3 (ERROR); a hardening TCB class without advisories is refused.

@sha256

Workspace isolation: - tinycloud-at-rest-cli is excluded from the node workspace, with its own Cargo.lock and CI job (fmt, clippy, tests incl. stock age v1.3.2 interop, sha256-pinned). dcap-qvl no longer enters the node's dependency graph: `cargo tree -p tinycloud-node -e features` is identical to main and the root lockfile only adds the escrow crate's dependencies. Escrow crate: - Accept age1yubikey1 (piv-p256) recipients, the format released age-plugin-yubikey 0.5.1 prints and decrypts, sealed exactly as the plugin's wrap_file_key; age1tag1 stays supported. - RecipientPolicy: X25519 software recipients are refused unless explicitly allowed (tests and self-host). - Bundle, record and recovery response are parsed version-first; RecoveryInfo gains a version. Clearer share/commitment errors. - forbid(unsafe_code). CLI: - TCB status is a fixed class: UpToDate passes; SWHardeningNeeded / ConfigurationNeeded / ConfigurationAndSWHardeningNeeded only if allowed and every advisory ID is pinned; OutOfDate*, TDRelaunch*, Revoked and unknown never pass. Status, advisories and collateral dates are printed. - Local checks (stale bundle, nonce, binding) run before any PCCS/DCAP work; PCCS fetch has a 60 s timeout. - --expect-efp and --allow-software on every bundle command; exit codes 0 OK / 1 REFUSED / 2 usage / 3 ERROR; verify-node shows node state and shares held; verify-record renamed record-efp and marked NOT AUTHENTICATED. - New: self-test (ceremony step 0 round trip with the TEE's sealing code), recipient-key (compare with a YubiKey attestation), and verify-compose (compose_hash + every image pinned by @sha256:). - Tests: every TCB class from a genuine quote and collateral, identity event ordering/duplication, v2 preimage, malformed logs, binary-level exit codes and messages, stock age interop (X25519 round trip and a Go-produced p256tag vector).

@sha256

…#325) - verify-compose parses the compose YAML with yaml-rust2 (=0.13.0, CLI workspace only) and accepts a strict subset: one document, no anchors, aliases, merge keys, tags or duplicate keys, a services mapping without include, and every service with an `image` of exactly `<name>[:tag]@sha256:<64 hex>` and no build/extends. It also lists the other app-compose settings the hash covers. The four reported bypasses (comment, flow mapping, quoted key, build) are tests. - Checked-in piv-p256 vector produced by the released age-plugin-yubikey 0.5.1; the test identity now rejects what the plugin's from_stanza rejects (argument lengths, uncompressed epk, body length). - Warn on age1tag1 holders (0.5.1 can't decrypt p256tag). - Version-first parsing keeps duplicate-key rejection; RecoveryInfo mode and state are validated. - CLI: self-test creates --out; malformed bundle/recovery/record JSON is exit 3 (ERROR); a hardening TCB class without advisories is refused.

Adds two workspace crates that the node does not use yet: - tinycloud-at-rest-escrow: 2-of-3 Shamir split of the at-rest root key (vsss-rs 5.4.0, GF(256)), shares sealed with age 0.12.1 to pinned age1tag1 (hardware) or age1 recipients, the escrow bundle and its fingerprint, the canary, the ARK-MACed verification record and the recovery wire format. - tinycloud-at-rest-cli (tinycloud-at-rest): recipients config, nonce, inspect, extract-share, check-share, verify-node, share-wrap and verify-record. share-wrap verifies the node's TDX quote with DCAP collateral (dcap-qvl 0.7.0) and a pinned trust policy before it releases a share; shares are read from stdin only. Fixtures: real TDX quotes and collateral from dcap-qvl 0.7.0 (MIT). Lockfile: dcap-qvl raises anyhow, borsh, chrono and typenum within their semver ranges. docs/at-rest-escrow.md describes the formats.

@sha256

Workspace isolation: - tinycloud-at-rest-cli is excluded from the node workspace, with its own Cargo.lock and CI job (fmt, clippy, tests incl. stock age v1.3.2 interop, sha256-pinned). dcap-qvl no longer enters the node's dependency graph: `cargo tree -p tinycloud-node -e features` is identical to main and the root lockfile only adds the escrow crate's dependencies. Escrow crate: - Accept age1yubikey1 (piv-p256) recipients, the format released age-plugin-yubikey 0.5.1 prints and decrypts, sealed exactly as the plugin's wrap_file_key; age1tag1 stays supported. - RecipientPolicy: X25519 software recipients are refused unless explicitly allowed (tests and self-host). - Bundle, record and recovery response are parsed version-first; RecoveryInfo gains a version. Clearer share/commitment errors. - forbid(unsafe_code). CLI: - TCB status is a fixed class: UpToDate passes; SWHardeningNeeded / ConfigurationNeeded / ConfigurationAndSWHardeningNeeded only if allowed and every advisory ID is pinned; OutOfDate*, TDRelaunch*, Revoked and unknown never pass. Status, advisories and collateral dates are printed. - Local checks (stale bundle, nonce, binding) run before any PCCS/DCAP work; PCCS fetch has a 60 s timeout. - --expect-efp and --allow-software on every bundle command; exit codes 0 OK / 1 REFUSED / 2 usage / 3 ERROR; verify-node shows node state and shares held; verify-record renamed record-efp and marked NOT AUTHENTICATED. - New: self-test (ceremony step 0 round trip with the TEE's sealing code), recipient-key (compare with a YubiKey attestation), and verify-compose (compose_hash + every image pinned by @sha256:). - Tests: every TCB class from a genuine quote and collateral, identity event ordering/duplication, v2 preimage, malformed logs, binary-level exit codes and messages, stock age interop (X25519 round trip and a Go-produced p256tag vector).

@sha256

…#325) - verify-compose parses the compose YAML with yaml-rust2 (=0.13.0, CLI workspace only) and accepts a strict subset: one document, no anchors, aliases, merge keys, tags or duplicate keys, a services mapping without include, and every service with an `image` of exactly `<name>[:tag]@sha256:<64 hex>` and no build/extends. It also lists the other app-compose settings the hash covers. The four reported bypasses (comment, flow mapping, quoted key, build) are tests. - Checked-in piv-p256 vector produced by the released age-plugin-yubikey 0.5.1; the test identity now rejects what the plugin's from_stanza rejects (argument lengths, uncompressed epk, body length). - Warn on age1tag1 holders (0.5.1 can't decrypt p256tag). - Version-first parsing keeps duplicate-key rejection; RecoveryInfo mode and state are validated. - CLI: self-test creates --out; malformed bundle/recovery/record JSON is exit 3 (ERROR); a hardening TCB class without advisories is refused.

samgbafa · 2026-10-17T10:15:24Z

REQUEST_CHANGES: see TC-887 (security review round 2, head 4222883bbe4ad1ca85ddc64cf96b21885d62a578).

…; review round 3 (#325) - The strict app-compose reader moves to tinycloud_at_rest_escrow::compose (yaml-rust2 =0.13.0), shared by the CLI and the node. - docker_compose_file may contain only newlines and printable ASCII: Unicode line breaks (U+0085, U+2028, U+2029), CR, tabs, BOM and control characters are refused before parsing. The review's four line-break PoCs are fixtures; the production compose is a test. - runner must be docker-compose. - literal_env: the escrow recipients and pin, if set, must be literal environment values (no interpolation or pass-through). - verify-compose prints the escrow literals and every other app-compose setting in full (pre_launch_script included); all untrusted text the CLI echoes is escaped (\u{..}), including REFUSED/ERROR lines. - docker-compose.dstack-postgres.yaml: an em dash in a comment replaced, so the production compose is ASCII-only.

samgbafa · 2026-10-17T11:07:29Z

APPROVE: see TC-887 (head 4222883)

samgbafa · 2026-10-17T13:02:18Z

APPROVE: see TC-887 (security review round 3, head 8083ca268c46d45b437784c87f1afa68c51b51f8).

…elling (#325) - app-compose.json with an object key repeated at any depth is refused (serde_json would keep the last; dstack's struct parse refuses it). - Null, NULL, null, ~ and empty escrow values are pass-through, refused explicitly rather than failing later.

samgbafa · 2026-10-17T13:59:55Z

APPROVE: see TC-887 (correctness round 3, head a0d57fa8fa0a59d9c1abe2006bdd0fc4860a62dd)

samgbafa · 2026-10-17T14:42:00Z

APPROVE: see TC-887 (security review round 3 delta, final head ca6bfb3b7b64e7be410678c208bd4a6cd3dd4793).

samgbafa · 2026-10-17T17:42:36Z

APPROVE: see TC-887 (security review round 4, head ca6bfb3b7b64e7be410678c208bd4a6cd3dd4793).

samgbafa · 2026-10-17T18:36:45Z

APPROVE: see TC-887 (correctness round 4, head ca6bfb3b7b64e7be410678c208bd4a6cd3dd4793)

samgbafa added a commit that referenced this pull request Oct 17, 2026

fix(TC-887): build the ARK only from zeroizing buffers (#325)

8d124b0

samgbafa mentioned this pull request Oct 17, 2026

feat(TC-887): at-rest keyring, key store and migration #326

Draft

samgbafa added a commit that referenced this pull request Oct 17, 2026

fix(TC-887): build the ARK only from zeroizing buffers (#325)

e37ace9

samgbafa force-pushed the skgbafa/tc-887-s1-escrow branch from 05c5200 to 65d3f4c Compare October 17, 2026 07:58

This was referenced Oct 17, 2026

feat(TC-887): node wiring for at-rest keys and escrow recovery #327

Draft

CI only: TC-887 S1 stack #325+#326+#327 (do not merge) #328

Closed

feat(TC-887): sealed KV block helpers and plaintext sizes by space tag #329

Draft

samgbafa force-pushed the skgbafa/tc-887-s1-escrow branch from 65d3f4c to 91c5411 Compare October 17, 2026 08:47

samgbafa added a commit that referenced this pull request Oct 17, 2026

fix(TC-887): build the ARK only from zeroizing buffers (#325)

c6fe193

samgbafa added 4 commits October 17, 2026 09:08

fix(TC-887): build the ARK only from zeroizing buffers (#325)

397f21c

samgbafa force-pushed the skgbafa/tc-887-s1-escrow branch from 91c5411 to 4222883 Compare October 17, 2026 09:11

This was referenced Oct 17, 2026

feat(TC-887): seal KV blocks in the FileSystem store; prepare writes before the invoke lock #333

Draft

test(TC-887): keep the at-rest key mirror out of KV pairing; check it for raw keys #335

Merged

Merge remote-tracking branch 'origin/main' into skgbafa/tc-887-s1-escrow

8083ca2

Merge remote-tracking branch 'origin/main' into skgbafa/tc-887-s1-escrow

ca6bfb3

Navigation Menu

Repository navigation

feat(TC-887): at-rest key escrow crate and ceremony CLI - #325

feat(TC-887): at-rest key escrow crate and ceremony CLI#325
samgbafa wants to merge 8 commits into
mainfrom
skgbafa/tc-887-s1-escrow

samgbafa commented Oct 17, 2026 •

edited

Loading

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

samgbafa commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

samgbafa commented Oct 17, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

samgbafa commented Oct 17, 2026 •

edited

Loading