Skip to content

fix(database): unblock Postgres boot after edited migrations (WI-919) - #170

Merged
stefan-ernst merged 1 commit into
mainfrom
fix/wi-919-migration-checksums
Aug 4, 2026
Merged

fix(database): unblock Postgres boot after edited migrations (WI-919)#170
stefan-ernst merged 1 commit into
mainfrom
fix/wi-919-migration-checksums

Conversation

@stefan-ernst

Copy link
Copy Markdown
Contributor

Every Postgres install stamped before 2026-05-17 refused to boot on main. Live was left in a 10-minute crash-rollback loop: the update timer pulled main, the container was recreated, boot failed, and update.sh rolled back.

Two commits months apart combined into this. ac1e4f1 (17 May) switched TIMESTAMP to TIMESTAMPTZ inside ten migrations that had already shipped and been applied. That was silent, because those bodies are CREATE TABLE IF NOT EXISTS and no-op against an existing table, so only fresh databases ever got the timezone-aware type. 54cff11 (4 Aug, unreleased) then added checksum validation and made a mismatch fatal, turning the latent drift into a hard boot failure.

Reverting the ten bodies is not a fix: it would restore databases stamped between 14 and 17 May while breaking every database stamped after 17 May, which would then compute the old checksum and mismatch in the other direction. Instead, record the pre-edit checksum on each affected migration as Superseded. A stored checksum listed there is accepted and re-stamped to the current value, so the entry matters only once per database, and an unrecognized mismatch stays fatal.

Superseded absorbs history; it does not make edits safe. The schema drift is converged for real by 20260804_pg_timestamptz_convergence, which alters the 27 affected columns across 11 tables to TIMESTAMPTZ. It is Postgres-only, self-guarding on data_type so it no-ops on already-converged databases and tolerates an absent table, and uses an explicit USING ... AT TIME ZONE 'UTC' so stored values are read as UTC rather than reinterpreted through the session TimeZone.

Verified against a throwaway Postgres by reverting the 27 columns and stamping the ten pre-edit checksums: boot completes, all 27 columns convert, and no stale checksums remain. A deliberately unknown checksum still aborts startup with exit 1.

Every Postgres install stamped before 2026-05-17 refused to boot on main.
Live was left in a 10-minute crash-rollback loop: the update timer pulled
main, the container was recreated, boot failed, and update.sh rolled back.

Two commits months apart combined into this. ac1e4f1 (17 May) switched
TIMESTAMP to TIMESTAMPTZ inside ten migrations that had already shipped and
been applied. That was silent, because those bodies are CREATE TABLE IF NOT
EXISTS and no-op against an existing table, so only fresh databases ever got
the timezone-aware type. 54cff11 (4 Aug, unreleased) then added checksum
validation and made a mismatch fatal, turning the latent drift into a hard
boot failure.

Reverting the ten bodies is not a fix: it would restore databases stamped
between 14 and 17 May while breaking every database stamped after 17 May,
which would then compute the old checksum and mismatch in the other
direction. Instead, record the pre-edit checksum on each affected migration
as Superseded. A stored checksum listed there is accepted and re-stamped to
the current value, so the entry matters only once per database, and an
unrecognized mismatch stays fatal.

Superseded absorbs history; it does not make edits safe. The schema drift is
converged for real by 20260804_pg_timestamptz_convergence, which alters the
27 affected columns across 11 tables to TIMESTAMPTZ. It is Postgres-only,
self-guarding on data_type so it no-ops on already-converged databases and
tolerates an absent table, and uses an explicit USING ... AT TIME ZONE 'UTC'
so stored values are read as UTC rather than reinterpreted through the
session TimeZone.

Verified against a throwaway Postgres by reverting the 27 columns and
stamping the ten pre-edit checksums: boot completes, all 27 columns convert,
and no stale checksums remain. A deliberately unknown checksum still aborts
startup with exit 1.
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@stefan-ernst
stefan-ernst merged commit b100cce into main Aug 4, 2026
16 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant