Skip to content

docs(TC-887): at-rest interfaces in the repo, one generation ceiling, harness follow-ups - #331

Merged
samgbafa merged 4 commits into
mainfrom
skgbafa/tc-887-followups
Oct 17, 2026
Merged

samgbafa merged 4 commits into
mainfrom
skgbafa/tc-887-followups

Conversation

@samgbafa

Copy link
Copy Markdown
Contributor

Part of TC-887 (encryption at rest): follow-ups to #322 and #324. Nothing here changes node behaviour.

Includes 126adcb (the workflow_dispatch trigger), which is also #330 on its own. Whichever merges first, the other picks it up as a no-op.

What changed

  • docs/at-rest/INTERFACES.md (new): the normative interface and format for tinycloud_core::at_rest and sealed KV storage. It covers the key types, DataKeyProvider, the envelope API, the TCE1 format, the TCE2 versioning rule, legacy-vs-sealed detection, notes for storage callers (header fields are unverified until a segment decrypts; Unsupported is unauthenticated and must fail closed), accepted key-material residuals, and the KV storage decisions. The envelope.rs and provider.rs docs now point to it.
  • One generation ceiling: PayloadEncoding::from_column reads 2..=9 as an unsupported later generation, the same range the header's ASCII digit can express. 0, negative values and anything above 9 are invalid. Previously 10..=255 also mapped to Unsupported.
  • Harness (scripts/at-rest-harness/, tests/at_rest_harness.rs):
    • The escrow acknowledgement is now TC887_AT_REST_ESCROW_NONE=true|false → TINYCLOUD_AT_REST__ESCROW_NONE, the node's real config name, replacing the scalar …__ESCROW. Mode and escrow values are validated before anything starts.
    • Size floor: every blob that must hold the 64 KiB value (the KV object, every SQL payload/delta_payload) must be at least that large, because TCE1 never shrinks data. This catches a compress-only transform.
    • KV pairing no longer assumes one object per put. Counts must match across the two spaces, then all pairs are compared.
    • Per-check expectations: test red --expect kv=green,sql=red STATE (green/red still mean both). Each check is judged and reported separately; exit 0 = all matched, 3 = a per-check mismatch, 4 = errored. With TC887_AT_REST_MODE=encrypt|strict the default is kv=green,sql=red. CI still runs --expect red.
  • .github/workflows/rust.yml: the workflow_dispatch trigger, and a comment in the at-rest-harness job.

How it's tested

  • cargo test -p tinycloud-core --lib at_rest: 30 pass. The PayloadEncoding test covers 2, 3, 9 → Unsupported and 0, −1, 10, 255, 256, i16 limits → invalid.
  • cargo test -p tinycloud-node --test at_rest_harness: the offline self-test passes. It now also shows that a salted deflate-9 of the 64 KiB value is caught by the size floor, while sealed-like blobs pass.
  • Real local harness run (native PostgreSQL 14 + MinIO + node): up → test green (0) → test red --expect red (0, "RED as expected": 20 KV + 16 SQL findings) → --expect kv=red,sql=red (0) → --expect kv=green,sql=red (3, KV-specific "still RED"; SQL as expected) → down --purge (0, nothing left running). Invalid --expect, mode or escrow values exit 2. With MODE=encrypt and ESCROW_NONE=true, node.env carries both variables and test red defaults to kv=green,sql=red.
  • cargo fmt --all -- --check, cargo clippy -p tinycloud-node --test at_rest_harness -- -D warnings, bash -n: clean. The workflow YAML parses (triggers: push, pull_request, workflow_dispatch).

…ceiling

docs/at-rest/INTERFACES.md carries the normative TCE1 format, the TCE2
versioning rule, the fail-closed and header-trust notes and the KV
storage decisions; the module docs now point to it. PayloadEncoding
reads 2..=9 as an unsupported later generation (the range the header
can express) and anything above 9 as invalid.
Review notes on #324 (N2, N3):
- Every blob that must hold a plaintext (each KV put's objects in
  total, every SQL payload and delta) must be at least that large:
  TCE1 never shrinks data, so a smaller blob is compressed, not sealed.
  Catches xz/bz2-style compression of a salted WAL delta that the
  shared-run and compressibility checks miss.
- KV cross-space comparison requires equal object counts per put and
  compares every pair, instead of zipping objects sorted by name.
- The offline selftest also proves a salted deflate-9 blob is flagged
  by the floor and sealed-like blobs pass it.
Review notes on #324 (N1, N4):
- TC887_AT_REST_ESCROW_NONE=true|false maps to S1's boolean
  TINYCLOUD_AT_REST__ESCROW_NONE (replaces the scalar ESCROW, which S1's
  deny_unknown_fields section rejects); mode is validated as
  off|encrypt|strict. State schema v3.
- test --expect takes green|red or kv=green|red,sql=green|red. Each
  check runs alone and is judged on its own; exit 0 only if all match,
  3 per mismatch (KV/SQL-specific UNEXPECTEDLY PASSED / still RED), 4 if
  any check errored. With mode encrypt|strict, test red defaults to
  kv=green,sql=red (S3 before S4); CI stays --expect red.
- README: SIGKILL of up can't be trapped; run down STATE.
@samgbafa

Copy link
Copy Markdown
Contributor Author

APPROVE: see TC-887 (head efe1f85)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant