TC-959: pass validated N2 correlation fields through the log sidecar

.github/workflows/docker.yml

@@ -658,7 +658,7 @@ jobs:
               raise SystemExit('::error::Deployment destination is not the fixed reviewed path')
           native = 'ghcr.io/tinycloudlabs/tinycloud-node'
           reviewed = native + '@sha256:4bdae4fc232d886bb56bfbce4ec62ed4b8fb4e1369a01f6afb73d06dd68c188f'
-          n0_hash = 'ca110b0bae6d5a06a1002fb80df3f71cffec11659c9f9dbfa5d72624735546f6'
+          n0_hash = '6563d51bffe4b537d49b0c087a50e86753aaaf5ba533f1e52c7c2214f7f5f04e'
           pre_n0_hash = 'df239749ca0a9c09893a855ae2a8abb94090b8556515ad002105c58ac91f4c91'
           pre_n0_template_hash = '28302d6761d841b7cc5c11314aed2d377735e6bbdcaf85972a752e03bc9fdc93'
           rollback = os.environ.get('ROLLBACK_PRE_N0') == 'true'

CHANGELOG.md

@@ -5,6 +5,7 @@
 - New migration `m20261017_000000_epoch_order_space_index` adds `idx_epoch_order_space_parent_child ON epoch_order (space, parent, child)` for the head-epoch anti-join that every write runs under its space lock (TC-1274). On a fresh database the build is instant. On PostgreSQL, if a valid index of that name and shape already exists, the migration issues no DDL and takes no lock on `epoch_order`. If the index of that name is INVALID or has a different shape, the boot fails, and nothing is applied, with an error naming the fix. On an existing database **without** the index, the migration builds it inside the boot's migration transaction. That holds a SHARE lock that blocks writes for the whole build, and the lock wait is bounded by the 30 s `lock_timeout` (TC-1176). **Runbook (PostgreSQL, before upgrading a node with history):** in autocommit, run `CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_epoch_order_space_parent_child ON epoch_order (space, parent, child);`. Then confirm `SELECT indisvalid FROM pg_index WHERE indexrelid = 'idx_epoch_order_space_parent_child'::regclass;` returns `t`. If it returns `f`, run `DROP INDEX CONCURRENTLY idx_epoch_order_space_parent_child;` and build it again. `down` keeps the index. MySQL is covered only by the generated statement, because the full MySQL migration chain does not run (TC-1314). Rolling back to v1.20.x: the older node refuses to start while `seaql_migrations` lists this migration (`Migration file of version 'm20261017_000000_epoch_order_space_index' is missing`). Delete that row first, and keep the index: v1.20.x runs the same anti-join, and a later upgrade finds the valid index and does nothing.
 - Writes no longer scan a space's whole epoch history to find its head epochs (TC-1274). A new `epoch_head` table (migration `m20261018_000000_epoch_head`) caches each space's heads. Every write maintains it in the same transaction, under the space's existing sequence lock. A write now reads O(heads) rows, normally one, instead of running the `epoch`/`epoch_order` anti-join, and usually costs one more statement, an `UPDATE` of the cached head. The cache is used only when its newest `seq` is the space's newest. Otherwise that space falls back to the old anti-join once and the write repairs the cache, so epochs appended by older releases (a rolling deploy, or a rollback and upgrade) never produce wrong parents. Heads are taken in `id` byte order. Previously the anti-join returned them in no defined order. This affects only a merge after a pre-TC-732 fork, and nothing re-derives an epoch hash from its parents. The migration only creates the empty table, without foreign keys, so it takes no locks on existing tables. After the node starts serving, a gentle online backfill checks each space once. It runs in one short transaction per space, under that space's lock, at no more than `epoch_head_backfill.spaces_per_second` (default 2) and pauses after each space for at least as long as the space took. It stops at its first error with one log line, and logs progress every `progress_every` spaces (default 500). Turn it off with `[epoch_head_backfill] enabled = false` or `TINYCLOUD_EPOCH_HEAD_BACKFILL__ENABLED=false`. Writes stay correct without it. Keep the `idx_epoch_order_space_parent_child` index: the fallback and the backfill use it. Rolling back to v1.20.x: as for `m20261017_000000_epoch_order_space_index`, the older node refuses to start until its rows are gone. Run `DELETE FROM seaql_migrations WHERE version IN ('m20261017_000000_epoch_order_space_index', 'm20261018_000000_epoch_head');` first. Keep the index and the `epoch_head` table: v1.20.x ignores the table. A later upgrade re-applies both migrations as no-ops (the index check and `CREATE TABLE IF NOT EXISTS`), and the `seq` check repairs every space the older node wrote to.
 - Request events include validated/echoed UUID correlation, typed replacement flags, node-keyed target account/space pseudonyms and per-request database acquisition/execution timings and attempt counts. Driver/actor instrumentation includes failures and request-awaited audit batches; forwarding requires explicit first-party origin classification. A shared core `Pseudonymizer` preserves existing account/space hashes, adds the service/space-scoped database hash primitive for F6, and is derived once at startup for injection into consumers. See `docs/request-telemetry.md` for the wire contract, key custody, restart joinability and coverage/exclusions (TC-959 N2).
+- The log sidecar now forwards the validated N2 correlation fields on `node_request` events: `request_id`, `request_id_replaced`, `account_hash` and `space_hash` (TC-959). Each is copied only when it has exactly the node's emitted shape; a malformed value is dropped and the event still ships. Ingress events and diagnostics never carry them, and `db_hash` stays on the F6 typed events. The embedded Vector configuration is regenerated; byte budgets now count the widest valid node request with these fields.
 - CACAO/SIWE delegation ingestion normalizes ECDSA recovery IDs 0/1 to 27/28 before hashing and persisting the canonical DAG-CBOR bytes (TC-1071). Equivalent low-s signatures now share one delegation CID, including the CID used for revocation lookup. High-s signatures are rejected during decoding, matching the existing EIP-191 verifier and EIP-2 low-s convention; UCAN/JWS bytes remain unchanged. Existing stored delegation IDs are not rewritten.
 - Synchronize the standalone M1, W5 and load-signer Cargo locks with CACAO's `k256` dependency so `--locked` dependency resolution succeeds (TC-1071). Keep the load signer compatible with the SDK's default invocation lifetime.
 

docs/releasing.md

@@ -294,8 +294,10 @@ gh workflow run docker.yml -R TinyCloudLabs/tinycloud-node --ref main \
 ```
 
 The served/OCI identity remains **1.20.6-dstack / e18578ff9162986ddd813259c60f73866e3e60d1**.
-The N0 raw lock is unchanged:
-`ca110b0bae6d5a06a1002fb80df3f71cffec11659c9f9dbfa5d72624735546f6`,
+The N0 raw lock **changed in #346 (TC-959)**, which passes the validated N2 correlation fields through the log sidecar:
+`6563d51bffe4b537d49b0c087a50e86753aaaf5ba533f1e52c7c2214f7f5f04e`,
+from #346 final reviewed head.
+It supersedes the #309 lock `ca110b0bae6d5a06a1002fb80df3f71cffec11659c9f9dbfa5d72624735546f6`,
 from #309 reviewed head `44eeb20a1461ef66b5af492e3be4c42be2728b67`,
 [approval](https://github.com/TinyCloudLabs/tinycloud-node/pull/309#issuecomment-6224650050).
 Current pre-N0 tag/pinned equivalents are separately hash-checked; arbitrary

docs/runbooks/tc-959-n0-log-shipping.md

@@ -28,11 +28,13 @@ Parsing failures become fixed counters; no log-level suppression is used.
 
 | Event | Fields beyond `ts`, `level`, `event`, `svc` | Delivery policy |
 | --- | --- | --- |
-| `node_request`, `svc=api` | `method`, canonical `route` template, integer `status`, nonnegative integer `latency_us` | Individually when valid and eligible under the selected request profile; no sampling |
+| `node_request`, `svc=api` | `method`, canonical `route` template, integer `status`, nonnegative integer `latency_us`; plus, only when present and exactly well-formed: `request_id` (lowercase UUID, version 1-8), `request_id_replaced` (boolean), `account_hash` and `space_hash` (64 lowercase hex, keyed pseudonyms). A malformed value is dropped and the request still ships. | Individually when valid and eligible under the selected request profile; no sampling |
 | `ingress_request`, `svc=ingress` | Same method/route/status, distinct numeric `request_time_seconds` | Individually when valid and eligible under the selected request profile; no sampling |
 | `node_warning` / `node_error`, `svc=api` | Fixed `event_code`, integer `event_count` | All accepted typed application WARN/ERROR records remain eligible under either profile; grouped counts approximately every 60 seconds |
 | `logship_gate_summary_total`, `svc=logship` | Fixed `reason`, integer `count` | Count every rejection that reaches the boundary; emit approximately every 60 seconds per fixed reason |
 
+**`request_id` provenance:** `request_id` is the caller's `X-Request-ID` when that header is a single valid UUID (version 1-8, RFC 4122 variant), and a node-generated UUID otherwise (`request_id_replaced=true`). The value therefore carries caller-chosen bits, and a version-1 UUID can embed the client's MAC address and a timestamp, per the #140 contract. The sidecar validates only the shape; it cannot tell a caller-chosen id from a random one. `account_hash` and `space_hash` are the node's keyed pseudonyms, never raw identifiers.
+
 ### Request profiles
 
 Sam approved **`errors_slow` as the deployment default**. Compose exposes `TINYCLOUD_LOGSHIP_PROFILE`, defaulting to `errors_slow`; the sidecar entrypoint accepts only `errors_slow` or `all_requests`. The profile is selected at Vector startup, not by per-record environment lookup.
@@ -144,7 +146,7 @@ Keep these identities distinct:
 | --- | --- |
 | Before: verifier `docker_compose_file` / hash | Raw preserved pre-N0 compose: `df239749ca0a9c09893a855ae2a8abb94090b8556515ad002105c58ac91f4c91` |
 | Before: application `compose_hash` | `bfe9028a58b23238188aa7d70b82282a7cd6414cd83dcc47a3b295c6f64fe2f8`; a distinct reported identity, not the raw-file hash |
-| After: current profile-enabled raw compose | `ca110b0bae6d5a06a1002fb80df3f71cffec11659c9f9dbfa5d72624735546f6`; bind this artifact to the final reviewed PR head before any promotion |
+| After: current profile-enabled raw compose | `6563d51bffe4b537d49b0c087a50e86753aaaf5ba533f1e52c7c2214f7f5f04e`; bind this artifact to the final reviewed PR head before any promotion |
 
 The raw after hash identifies the profile-enabled compose, not deployment approval. Record its final reviewed PR head and required CI evidence in the officer handoff. Do not infer a raw-file hash from the distinct application-reported compose hash.
 

log-shipper/budget.mjs

@@ -10,6 +10,11 @@ export const MINUTES_30_DAYS = 30 * 24 * 60;
 export const REQUEST_ALLOCATION_BYTES = 480_000_000;
 
 const common = { level: 'info', ts: '9999-12-31T23:59:59.999Z' };
+// Worst-valid N2 correlation fields on a node request: longest request id, both full-width pseudonyms.
+const correlation = {
+  request_id: 'ffffffff-ffff-8fff-bfff-ffffffffffff', request_id_replaced: true,
+  account_hash: 'f'.repeat(64), space_hash: 'f'.repeat(64),
+};
 const request = {
   method: longest(schema.fields.method.values), route: longest(schema.fields.route.values),
   status: schema.fields.status.max,
@@ -18,12 +23,13 @@ const request = {
 /**
  * Representative worst-valid N0 wire records: longest schema strings, maximum
  * integer widths, and a long round-trippable fractional ingress duration.
- * N2 UUIDs/hashes are reserved by the canonical schema but are NOT emitted in N0.
+ * Only node_request carries N2 correlation fields (request_id, request_id_replaced, account_hash,
+ * space_hash); the ingress log carries none. db_hash is emitted by the F6 typed events only.
  * Bytes below are exact for these records, not a proven Vector float-serializer
  * upper bound or a fixed event-size guard. Actual ingestion/billing can differ.
  */
 export const representativeEvents = Object.freeze({
-  node: Object.freeze({ ...common, event: 'node_request', svc: 'api', ...request, latency_us: schema.fields.latency_us.max }),
+  node: Object.freeze({ ...common, event: 'node_request', svc: 'api', ...request, latency_us: schema.fields.latency_us.max, ...correlation }),
   ingress: Object.freeze({ ...common, event: 'ingress_request', svc: 'ingress', ...request, request_time_seconds: 0.0000010000000000000002 }),
   diagnostic: Object.freeze({ level: 'warn', ts: common.ts, event: 'node_warning', svc: 'api', event_code: longest(schema.fields.event_code.values), event_count: schema.fields.event_count.max }),
   gate: Object.freeze({ level: 'warn', ts: common.ts, event: 'logship_gate_summary_total', svc: 'logship', reason: longest(schema.fields.reason.values), count: schema.fields.count.max }),

log-shipper/budget.test.mjs

@@ -6,7 +6,7 @@ import { calculateBudget, representativeEvents, serializedEventBytes, MINUTES_30
 const fields = JSON.parse(await readFile(new URL('./fields.json', import.meta.url), 'utf8'));
 const nodeSchema = fields.services['tinycloud-node'];
 
-test('request representatives use the schema longest route and method, with no N2 identifiers', () => {
+test('request representatives use the schema longest route and method; only node requests carry correlation fields', () => {
   const routes = nodeSchema.fields.route.values;
   const methods = nodeSchema.fields.method.values;
   for (const row of [representativeEvents.node, representativeEvents.ingress]) {
@@ -15,7 +15,11 @@ test('request representatives use the schema longest route and method, with no N
     assert.ok(routes.includes(row.route));
     assert.ok(methods.includes(row.method));
     assert.equal(row.ts.length, 24);
-    for (const key of ['request_id', 'account_hash', 'space_hash']) assert.equal(Object.hasOwn(row, key), false);
+    for (const key of ['request_id', 'request_id_replaced', 'account_hash', 'space_hash']) {
+      assert.equal(Object.hasOwn(row, key), row === representativeEvents.node);
+    }
+    if (row === representativeEvents.node) for (const key of ['request_id', 'account_hash', 'space_hash']) assert.match(row[key], new RegExp(nodeSchema.fields[key].regex));
+    assert.equal(Object.hasOwn(row, 'db_hash'), false);
   }
   assert.equal(representativeEvents.node.event, 'node_request');
   assert.equal(representativeEvents.node.latency_us, Number.MAX_SAFE_INTEGER);