[samgbafa]

Linear: TC-1423 (part 1 of 3)

Policy/v3 registration records which of the owner's delegations granted what the policy roots carry, together with that delegation's ancestor closure. #360 adds the root-lifetime cap and the record/enforce mode. #362 checks the record whenever a policy is used.

Changes

• Migration m20261019_000000_policy_v3_owner_authority. Adds two new tables, policy_v3_owner_binding and policy_v3_owner_authority (policy_cid, delegation_id, depth), and an index on (delegation_id, policy_cid). There is no ALTER and no foreign key, and both tables are created empty. down drops both tables. The migration is the last entry of Migrator::migrations().
• Registration selects an authority for each root capability (policy_v3/owner_authority.rs):
  • Root authority needs no delegation.
  • Otherwise it first looks at delegations to the owner from the owner of the capability's space or encryption network (the 64 newest).
  • Then it looks at the general window: the 256 newest delegations to the owner that carry an ability on the capability's space or network (an abilities subquery), at most 32 tried.
  • It skips any candidate whose closure contains a delegation issued by the policy owner, without using up a try.
  • Among the candidates that grant the capability, it picks the first, newest first, whose closure outlives the root. If none does, it picks the newest.
  • The choice is stored in the registration transaction with state bound, bound-uncapped or root-authority.
• Registrations from before the migration are classified in the background the first time authorize_roots sees them. Nothing is added to the request: policies whose stored classification is final are remembered in process (bounded); others get one background task per policy, at most three at once. The classification runs as of registered_at (only grants issued no later than it). One transaction inserts the binding row first and the closure after it; if another writer stored a row first, its row is used. A result that is not final (a failed write, or starved) holds off the next attempt for 60 s. unknown rows store a reason (revoked, missing, starved or foreign_network). starved is re-evaluated on later uses; the others are re-evaluated when CLASSIFIER_VERSION is bumped.
• Stored policies over networks that are not the content owner's. The classifier applies content_owner_networks (from Check the owner and existence of policy source networks at registration #363) to the stored policy before it looks for delegations. A policy that names an encryption network, or a kv space, that does not belong to the owner of its content source is classified unknown with reason foreign_network, in both modes, whatever delegations the owner holds. The rule is a pure check of the stored policy; it does not look up whether the network exists. foreign_network is final. No schema change.
• Opt-in sweep: policy_v3.owner_binding_backfill.enabled, default off, paced by policies_per_second (default 1).
• policy_v3.legacy_cutover. Only registrations before the cutover count as legacy. It defaults to this migration's seaql_migrations.applied_at. The effective value and its source are logged at boot.
• POST /policy/v3/authorized {cid, cursor} lists the policies whose recorded closure contains cid or one of its alias CIDs.
  • Each entry has policyCid, policyRootCid, enforcementRootCid, registeredAt, expiresAt, rootState and ownerAuthorityState. Policy bytes are never returned.
  • Authorization follows the /delegation/status rule on urn:cid:<cid>; everyone else gets 403.
  • Pages are keyset pages of 100 on policy_cid, with an opaque cursor and no total count. Unclassified legacy policies are not listed. Database errors return 503 policy-listing-unavailable.
• Core: tinycloud_core::policy_owner_authority adds a one-round-trip closure loader (auth_graph::load_closure_edges), batched alias-aware revoked IDs (revocation::effective_rows), alias equivalents and the migration applied-at lookup. revocation.rs is not edited.
• docs/policy-owner-authority.md and a CHANGELOG entry. The rollback section carries the SQL statements and a checklist, including the policy_v3.legacy_cutover step for the re-upgrade.

Stack and merge order

• Base: main. This branch contains Check the owner and existence of policy source networks at registration #363 (merged in), because the classifier uses its content_owner_networks; Check the owner and existence of policy source networks at registration #363 merges first.
• Merge order: Check the owner and existence of policy source networks at registration #363, record owner authority for policy registrations #358, cap policy root lifetime and add authority mode #360, check owner authority on policy share use #362, Check configured issuer set at policy registration #396, Allow policy registration under a space-owner register grant #399, TC-1500: fixed DID method allow-list for UCAN resolution #392.
• owner_authority_tests create the content owner's mainnet network after boot_app, since registration requires the source network to exist (Check the owner and existence of policy source networks at registration #363).
• CHANGELOG: one ### Changes heading under [Unreleased] with all pending bullets.

Rollback

Run the three statements in docs/policy-owner-authority.md (DROP TABLE IF EXISTS on both tables and DELETE FROM seaql_migrations for this migration), then deploy 1.22.0. On the re-upgrade, set policy_v3.legacy_cutover to this release's first deploy time. The checklist in the document has one step for each.

Tests

• policy_v3/owner_authority_tests.rs boots the production app (app_with_control). Each test runs once on SQLite (file DB) and once on PostgreSQL (own schema, TINYCLOUD_TEST_POSTGRES_URL; PostgreSQL runs are CI-only).
  • registration records the authority (UCAN, SIWE, root authority, uncapped);
  • loop-back delegations are skipped in the owner-first step and in the general window, and do not use up tries;
  • delegations from other issuers do not displace the owner's grant, at registration or in the legacy backfill, for UCAN and SIWE owners;
  • the general window only holds delegations touching the capability's space or network;
  • starved is not final; a window full of skipped rows is starved;
  • a legacy classifier ignores grants issued after registration;
  • an older grant that outlives the root is preferred;
  • a classification that cannot be stored is served from memory;
  • concurrent first uses store one classification;
  • rollback and re-upgrade respect the configured cutover; the sweep runs only when enabled;
  • listing: authorization, paging, states.
• Unit tests in owner_authority.rs (in-memory SQLite):
  • a stored registration over another owner's network is classified unknown / foreign_network in backfill and record mode, and a policy over the content owner's network goes on to the delegation search (stored_registrations_over_foreign_networks_are_unknown);
  • foreign_network is read back as a final classification, and every UnknownReason round-trips;
  • the in-flight claim is released when a background task panics.
• Mutation check for the new classification: inverting the content_owner_networks check fails stored_registrations_over_foreign_networks_are_unknown.

Verification

At this head, on SQLite: cargo fmt --all -- --check; cargo clippy --workspace --all-targets and cargo clippy -p tinycloud-core --features duckdb --all-targets, both with RUSTFLAGS=-Dwarnings; node scripts/gen-capabilities.mjs --check; and every test binary of tinycloud-core, tinycloud-node and tinycloud-auth pass. Two sets of cases did not pass in the combined run: the three startup_identity cases, which start the node binary and refuse to create an identity because the host has a dstack socket, and the three account_alias_tests, which time out when other jobs share the host and pass when the tinycloud-node lib tests run alone. PostgreSQL-backed tests run in CI.

Details: TC-1423

[samgbafa]

Review at 0ccb45e54f02821a7bbe25b0ec9cc23056c196e8 (TC-1423 N1). No approval at this head; the findings below are minor.

Verified locally on SQLite and a native PostgreSQL 14:

• PR tests: 32/32. RED at 0c17677: 0/32, because the tables and route are missing.
• fmt and clippy -D warnings: clean. tinycloud-node lib: 568/568.
• Mutations: owner-first removed, loop-back filter removed, loop-back filter applied to ancestors only, outlives-root preference removed, listing authorization bypassed. Each one is caught by a test on both backends.
• Trial merge with fix(TC-1436): tighten revocation authorization #353/fix(TC-1423): typed scope for revoke controls #356 + TC-1432: add no-transform to ETag-bearing responses #354: the only conflicts are in CHANGELOG, and the 32 tests pass on the merged result.

Findings:

1. Read path work. authorize_roots now awaits owner_binding (policy_v3.rs:2811-2819).

   • Every policy read, challenge and mint pays +1 primary-key lookup on policy_v3_owner_binding, forever.
   • The first use of a legacy row runs the selection plus a write inline. On SQLite that includes a writer-lock wait of up to 250 ms.
   • After a failed write, the selection reruns on every request.

   None of this gates anything in N1. Suggest:

   • a process-local cache of policy CIDs whose classification is final (the planned N3 cache, introduced now);
   • running the lazy classification off the request (spawned, deduplicated per policy), or at least try_lock on SQLite;
   • a short per-policy backoff after a failed write.

2. Test gap. Dropping the IssuedAt <= registered_at bound in the lazy classifier passes all tests. Suggest a did:pkh-session test where the only owner grant was issued after registration (expect unknown/missing), and a did:pkh variant of the backdated-iat case with issue times between the grant's and the registration's.

3. Test gap. Removing the "general window full => starved" marker (owner_authority.rs:448-450) passes all tests: the starved test gets there through the 32-try cap. Pin it or drop it.

4. General-window selection, before N3 gates on it:

   • restrict step-2 candidates in SQL to delegations that carry an ability on the capability's space or network;
   • don't count rows skipped by the loop-back filter toward the 32-try budget;
   • make starved retryable rather than persisted;
   • bump CLASSIFIER_VERSION when these land.

5. N3 scope. policy_v3.legacy_unknown_allow (default false, logged) isn't in this PR. It is inert here, because nothing denies yet, so it should land with N3's denial.

6. Nit. The listing's DB-error 503 returns the raw DB error text (db_error). Map it to the fixed policy-listing-unavailable.

7. Nit. Consider logging the effective cutover at warn when source=migration and the migration was applied recently. A forgotten override after a rollback is then easier to spot.

8. CHANGELOG. fix(TC-1436): tighten revocation authorization #353, TC-1432: add no-transform to ETag-bearing responses #354 and fix(TC-1423): typed scope for revoke controls #356 add the same ### Changes heading. On merge, keep one heading with all bullets.

Details: TC-1423.

[samgbafa]

Delta review at 65df9d5ed7ead119d40f8f0ab3451a5ce3cf010d (TC-1423 N1). No approval at this head; the remaining items are small.

The previous findings are fixed in code:

• Classification is off the request path. A final policy, or one in backoff, now adds no query and no await to the request: verified by reading the code and by the counted-connection unit tests, which the mutations kill.
• The did:pkh tests are in.
• The window-full starved marker is pinned.
• Step 2 is scoped to the capability's space or network.
• CLASSIFIER_VERSION is 2.
• The listing's 503 has a fixed body, and the cutover log level follows its age.
• The escape flag is stated for N3, and the CHANGELOG order is stated.

Local reruns on SQLite and native PostgreSQL 14:

• RED on main + tests: 0/40. GREEN: 42/42.
• fmt and clippy -D warnings: clean.
• Mutations M1, M2, M3, M5, M6 and M8 each fail a test on both backends.

Remaining:

1. CI test (workspace, PostgreSQL-backed) fails on delegations_back_to_the_owner_are_never_recorded::sqlite with /delegate: 401 … Child delegation expiry exceeds parent expiry. This is a fixture race at owner_authority_tests.rs:1275-1287. The authority's expiry is computed before an HTTP round trip, and the link's window after it; expiries have second granularity, so the link can end up one second past its parent. Compute one window up front and use it for all four delegations. The same failure also showed up in two of my mutation runs.
2. Two behaviours are not pinned by tests. Mutating "rows skipped by the loop-back filter don't use a try" and "starved is re-classified" both pass 42/42. Suggest:
   • about 40 loop-back rows (more than 32, fewer than 256) sorted ahead of a re-delegated authority, expecting bound;
   • a stored starved row at the current classifier version for a policy whose authority is findable: reboot, use the policy, and expect bound.
3. Concurrency of background classifications is capped only per policy, and at 65,536 overall. Suggest a small global semaphore (2-4 permits) with try_acquire; when no permit is free, skip without backing off.
4. Low. a_general_window_full_of_skipped_rows_is_starved::sqlite failed once in about 70 local runs at this head. Suggest driving the background classification deterministically in tests: await the spawned handle through a test hook, or call owner_binding directly.
5. Nit. A panicking background task leaves its policy marked in-flight until restart. Use a drop guard.
6. Info. On PostgreSQL the step-2 scope plans as a full scan of ability. On a local synthetic set of 1.2M ability rows it took about 80 ms, against 31 ms for the old unscoped delegation scan. That is the same trade-off core documents at db.rs:3974-3982. It runs only when step 1 misses. Track it with the deferred ability(resource COLLATE "C") index.

Details: TC-1423.

[samgbafa]

Delta review at 09d7e336bb7c68bd3b9a0c615a3ac570745d38fd (TC-1423 N1). No approval at this head; one test line remains.

Verified locally on SQLite and native PostgreSQL 14:

• RED on main + tests: 0/42. GREEN: 46/46. fmt and clippy -D warnings: clean.
• Mutations M1, M2, M3, M5, M6, M8, M9 (loop-backs use tries) and M10 (starved final) each fail a test. So do removing the final-cache check, the backoff, the drop-guard release, or the slot limit.
• The 3-slot semaphore releases its permit on every path (early None, completion, panic, cancellation), never waits while holding a lock, and a skipped use leaves no mark or back-off, so a later use retries.
• The D1 window fix holds; no other test derives delegations from a separately timed grant.

Remaining:

1. loop_back_delegations_are_skipped_in_the_general_window (owner_authority_tests.rs:1355-1375) still builds its authority without high_id. Each sorting_before call there fails when that id is the smallest of 513, about 0.4% per backend run. Under 8-way load I reproduced it 1 time in 504 runs, with no delegation sorted before the target. Build that authority with high_id. For targets that already use high_id, a failure needs probability 1/C(576,64), independent of the id layout.
2. Info: CI's PostgreSQL test job is red on tinycloud-core postgres_concurrent_historical_backfills_produce_ready_databases (its 30 s overlap observation timed out). This delta doesn't touch core: that test passed in CI at 65df9d5 and passes 6/6 locally here. But cargo stopped there, so N1's node tests didn't run in CI at this head. Please rerun CI with the fix above.

Details: TC-1423.

[samgbafa]

APPROVE: see TC-1423
058eee0

[samgbafa]

APPROVE: see TC-1423 (main-merge only; PR change unchanged vs 058eee0: added/removed lines identical for all own files; migrations/mod.rs and CHANGELOG.md resolved additively with N1's migration last; head 9364237)

[samgbafa]

APPROVE: see TC-1423 (reviewed exact head 4df9331)