Skip to content

db:apply: fix the two preflight checks that made every dry run fail, and stage the held September migrations - #1903

Open
njrini99-code wants to merge 1 commit into
mainfrom
agent/db-apply-preflight
Open

njrini99-code wants to merge 1 commit into
mainfrom
agent/db-apply-preflight

Conversation

@njrini99-code

Copy link
Copy Markdown
Owner

Why

Every npm run db:apply dry run against the September migrations failed at preflight. Neither failure was about the migrations.

(a) The origin/main reachability check could never pass — for any file.
It shelled out to git log origin/main --name-only --pretty=format:, which emits every path of every commit: 2.28 MB in this repo, against execFileSync's 1 MB default maxBuffer. It threw ENOBUFS on every invocation and the catch turned that into FAIL. Fail-closed, but a check that cannot pass verifies nothing.

Replaced with a scoped git rev-list -1 origin/main -- <path> — bounded by one commit id regardless of repo size. Raising maxBuffer would only move the cliff.

(b) The HELD.md gate let held migrations through.
It matched a literal **HOLD** anchored to a | immediately before the basename. Real register rows carry qualified statuses (**HOLD — R3, not yet reviewed**) and group files into one row (A.sql + B.sql + C.sql), so merged-but-held migrations passed the gate. That is the dangerous direction — it admits an unreviewed privileged migration.

isHeldInRegister now parses rows cell by cell, splits the name cell on +/,, matches whole filename tokens, and anchors the status to the start of the status cell so **APPLIED … — hold discharged** is not a false positive.

(c) A third defect that (a) had been masking.
extractVerifyQueries treated each -- VERIFY: line as a separate query, so a multi-line assertion degraded into a bare unfiltered select 1 from information_schema.columns — which returns rows against any database at all. Continuation lines are now joined until ;. APPLY_PATH.md and check-migration-headers.mjs updated to match.

Also staged

-- VERIFY: (and -- ROLLBACK: where missing) headers on the held helm_debug / helm_jobs / pgaudit migrations, so --apply can verify itself.

Every assertion was extracted with the shipped extractVerifyQueries and executed read-only against production as its literal string:

  • All are valid SQL.
  • Every discriminating assertion returns false today — the correct pre-apply answer.
  • Signature checks use ::regprocedure, not pg_get_function_identity_arguments, which returns parameter names alongside types and would have failed post-apply.
  • The has_function_privilege(role, 'schema.fn(sig)', 'execute') form was shape-checked against functions that exist today (single- and multi-argument), confirming the role names and the = false comparison.

Two assertions were rewritten after that pass caught them asserting nothing:

  • 20260906115900 — the signature already exists in production, so both existence checks pass before and after. Added a prosrc like '%min_exec_ms%' body discriminator.
  • 20260906142000pg_db_role_setting.setconfig quotes values Postgres considers to need it (search_path="$user", public sits in this very table), so an ilike on the raw spelling would miss pgaudit.log="ddl, role". Now a whitespace- and quote-tolerant regex.

Header ratchet re-baselined downward.

What this does NOT do

These migrations stay HELD. This makes them applyable and self-verifying; it does not clear them to apply. Three independent owner-only prerequisites remain:

  1. Eight of the nine are R3 and have not been through db-migration-reviewer, which APPLY_PATH.md step 4 makes mandatory. A grep scan is not that review.
  2. --apply is blocked for agents by permissions.deny; APPLY_PATH.md step 8: "Only the owner runs --apply."
  3. supabase link has never been run in this checkout, so the ledger check cannot run.

Test plan

  • scripts/__tests__/db-apply-preflight.test.mjs — 33 tests, registered explicitly in vitest.config.ts (that directory has no glob, so an unlisted file is decorative). Includes the pre-fix regex kept as a regression witness, a hold discharged false-positive test, whole-token matching, and a source-level assertion that the unbounded git log is gone.
  • npm run typecheck, npm run lint, npm run check:migration-headers — pass.

Unrelated, pre-existing: scripts/__tests__/create-workspace.test.ts fails 13/16 on main as well — the machine is at 11 GiB free against createWorkspace's 12 GiB reserve, so it refuses. Not touched here.

🤖 Generated with Claude Code

Every dry run against the September migrations failed at preflight, and
neither failure was about the migrations.

(a) The origin/main reachability check shelled out to
    `git log origin/main --name-only --pretty=format:`. That emits every
    path of every commit — 2.28 MB in this repo — against execFileSync's
    1 MB default maxBuffer, so it threw ENOBUFS on every invocation and
    the catch turned that into a permanent FAIL. Fail-closed, but a check
    that cannot pass for any file verifies nothing. Replaced with a
    scoped `git rev-list -1 origin/main -- <path>`, bounded by one commit
    id no matter how large the repo gets. Raising maxBuffer would only
    move the cliff.

(b) The HELD.md check matched a literal `**HOLD**` anchored to a `|`
    immediately before the basename. Real register rows carry qualified
    statuses (`**HOLD — R3, not yet reviewed**`) and group several files
    into one row (`A.sql + B.sql + C.sql`), so merged-but-held migrations
    passed the hold gate. That is the dangerous direction — it lets an
    unreviewed privileged migration through. `isHeldInRegister` now parses
    table rows cell by cell, splits the name cell on `+`/`,` and matches
    whole filename tokens, anchoring the status to the START of the status
    cell so `hold discharged` is not a false positive.

Fixing (a) made a third defect reachable for the first time:
`extractVerifyQueries` treated each `-- VERIFY:` line as its own query, so
a multi-line assertion degraded into a bare unfiltered SELECT that returns
rows against any database. Continuation lines are now joined until `;`.
APPLY_PATH.md and check-migration-headers.mjs updated to match.

Also stages `-- VERIFY:` (and `-- ROLLBACK:` where missing) headers on the
nine held helm_debug/helm_jobs/pgaudit migrations, so `--apply` can verify
itself. Every assertion was executed read-only against production first:
all are valid SQL and all discriminators return 0 today, which is the
correct pre-apply answer. Signature checks use `::regprocedure` rather
than `pg_get_function_identity_arguments`, which returns parameter names
alongside types. Header ratchet re-baselined downward.

These files stay HELD. This change makes them applyable and
self-verifying; it does not clear them to apply. Eight of the nine are R3
and have not been through db-migration-reviewer, `--apply` is owner-only
per docs/operations/APPLY_PATH.md, and `supabase link` has never been run
in this checkout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant