Repository navigation
feat(TC-1274): add epoch_order (space, parent, child) index migration - #341
Merged
Merged
Conversation
Adds m20261017_000000_epoch_order_space_index, matching the index operators build online with CREATE INDEX CONCURRENTLY. On PostgreSQL it reads pg_index first and issues no DDL (so takes no SHARE lock on epoch_order) when a valid btree index of that name on (space, parent, child) exists; an INVALID or differently shaped index of that name fails the boot with the fix. Without the index it builds it in the migration transaction. down() keeps the index. The m20261015 lock-timeout test no longer assumes it is the last migration.
Contributor
Author
|
APPROVE: see TC-1274 (head |
…ogress The pg_index pre-check now requires indisvalid, indisready and indislive. A same-name index whose CREATE INDEX CONCURRENTLY is still running fails the boot with 'still being built ... wait', not the DROP INDEX fix. It is found by its pg_stat_progress_create_index row: by index_relid, or, when PostgreSQL hides the row's columns from this role, by that backend's SHARE UPDATE EXCLUSIVE lock on epoch_order. The m20261015 lock test starts from migrated_through(that migration) instead of rolling back with down().
Contributor
Author
|
APPROVE: see TC-1274 (delta |
samgbafa
added a commit
that referenced
this pull request
Oct 17, 2026
TC-735's later kv_orphan_candidates migration made Migrator::down(1) undo it instead of the cursor index. Identical to #341's fix.
samgbafa
added a commit
that referenced
this pull request
Oct 17, 2026
Rename the unshipped kv_orphan_candidates migration from m20261015_020000 to m20261017_020000 so it sorts after main's m20261017_000000_epoch_order_space_index, per the second-merger rule. Take main's cursor-index lock test (migrated_through), dropping ours.
samgbafa
added a commit
that referenced
this pull request
Oct 17, 2026
…lit (#326) - D1: a mirrored data key that doesn't authenticate is never ignored and never overwritten. It may be the genuine key under a generation that didn't load (a restore plus a failed boot read). restore_dek fails closed for that space (AtRestError::DekUnavailable, transient, reason dek_unavailable) instead of letting create_dek and mirror_dek replace it; mirror_dek never overwrites an existing object with other bytes; an oversized DEK object fails closed too. Test reproduces the review's D1 chain and checks the original key returns once the object reads. - KeyringConfig::allow_resplit_before_verify: a holder change may re-split at boot only with the flag, no pin, an unverified active escrow and no data key ever made; otherwise EscrowHoldersChanged. - Merges main (#341's m20261017_000000_epoch_order_space_index and its migrated_through m20261015 test).
samgbafa
added a commit
that referenced
this pull request
Oct 17, 2026
…ig (#327) - FS mirror get opens with O_NOFOLLOW | O_NONBLOCK and requires a regular file (fstat on the handle) before reading, so a FIFO under an escrow or record name can't hang the boot (test). - at_rest.escrow.allow_resplit_before_verify passes through to the keyring's guarded pre-ceremony re-split. - Merges #326 (D1 fail-closed data keys, main with #341). - Docs: the re-split flag, dek_unavailable, the never-overwrite DEK rule.
samgbafa
added a commit
that referenced
this pull request
Oct 17, 2026
…ory (#343) * feat(TC-1274): maintain epoch_head so writes stop scanning epoch history transact found a space's head epochs (a new epoch's parents) with an epoch/epoch_order anti-join over the space's whole history on every write, under the space's sequence lock. It now reads them from a new epoch_head table (PK (space, id)) and records the new epoch as the space's only head in the same transaction: one UPDATE in the usual one-row case. The cache is trusted only when its newest seq is the space's newest (MAX(event_order.seq) + 1, which transact already computes). Every epoch, including one appended by a release that does not write epoch_head, outranks all existing ones under the same lock, so a stale cache is always detected; that space falls back to the anti-join once and the write repairs it. Heads are taken in id byte order; nothing re-derives an epoch hash from its parents. m20261018_000000_epoch_head creates only the empty table, with no foreign keys, so it takes no lock on existing tables. A gentle online backfill ([epoch_head_backfill], on by default at 2 spaces/s, pausing at least as long as each space took, stopping at its first error) fills spaces written before the table existed; writes are correct without it. The TC-951 round-trip budgets rise by the measured cache write: sign-in 75 -> 77, kv put 37 -> 38. Also folds in two #341 review nits: the in-progress check's pg_locks is limited to the current database, and a failed index plus a non-building SHARE UPDATE EXCLUSIVE holder still gets the DROP advice. * fix(TC-1274): epoch_head review delta: doc reflow, rollback note, disabled pacing - Reflow the epoch_head module doc so no line starts with '+ 1' (clippy doc_lazy_continuation). - CHANGELOG: rolling back to v1.20.x first needs the seaql_migrations rows of m20261017 and m20261018 deleted (sea-orm-migration refuses unknown applied versions); keep the index and the table. - A disabled backfill boots whatever its pacing fields say; an enabled one still needs spaces_per_second and progress_every of at least 1.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TC-1274. Adds the node migration that matches the production index Sam is building online,
idx_epoch_order_space_parent_child ON epoch_order (space, parent, child). With it, main matches prod, and fresh databases get the index.Behaviour
m20261017_000000_epoch_order_space_indexdoes(space, parent, child)already exists (operator-built)pg_indexand issues no DDL. It takes no lock onepoch_order, and the index keeps its OID.CONCURRENTLY), or has a different shape (columns, method, predicate or expressions)DROP INDEX CONCURRENTLY/CREATE INDEX CONCURRENTLYfix.CREATE INDEXinside the boot migration transaction. The SHARE lock onepoch_orderblocks writes for the whole build, and holds until every pending migration commits. Waiting for that lock is bounded by the 30 slock_timeoutfrom #316 (TC-1176). The build itself is not bounded.has_index, then creates the index. MySQL has noCREATE INDEX IF NOT EXISTS, and sea-query silently drops the clause.The pg_index pre-check exists because a bare
CREATE INDEX IF NOT EXISTSstill takesepoch_order's SHARE lock before it checks the name, and the migration transaction holds that lock until commit. The test below shows this: the naiveif_not_existsversion blocks behind an open writer.down()keeps the index. It may be operator-built, and older releases run the same anti-join.This does not copy m20261015's drop-then-create pattern.
Runbook (PostgreSQL, before upgrading any node that has history)
Run this in autocommit (psql, no
BEGIN):If you upgrade without it, the boot builds the index under a write-blocking lock.
Tests (RED first)
generated_create_index_statements: the exact PG, SQLite and MySQL DDL. This is MySQL's only coverage, because the full MySQL migration chain fails on init_tables with error 1074 (TC-1314, a known release caveat; I reproduced it on MySQL 8.4.7).rootpage).db::migrateentry point:space,parent,child);CREATE INDEX CONCURRENTLYindex kept, same OID, while a writer holds ROW EXCLUSIVE onepoch_order(finishes in under 5 s);DbErrnaming the index andDROP INDEX CONCURRENTLY, migration not recorded.RED evidence, native PG 18.6:
create_index(..if_not_exists()),postgres_operator_built_index_is_kept_without_blocking_writersfails (blocked behind the writer) andpostgres_unusable_operator_index_fails_the_migrationfails (the INVALID index is accepted).cargo test -p tinycloud-core --lib migrationspasses 18/18.cargo clippy -p tinycloud-core --all-targets -D warningsandfmt --checkare clean.Also:
m20261015's lock-timeout test usedMigrator::down(Some(1))and so assumed it was the last migration. It now rolls back to just before itself.No merge or deploy is implied.