From 7423bd987469d827ee0c5734f4d7d8fb019f24b7 Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Sat, 12 Sep 2026 08:22:43 -0600 Subject: [PATCH] test: seed the mutation ledger with differential, 204 checks (#752) suites_not_covered 250 -> 249 checks_never_observed_red 951 -> 1155 covered harness_selftest, native_join_runtime_filter, differential BOTH NUMBERS ARE DERIVED FROM THE FILES, and this change is its own argument for that rule. Written against an earlier base the same seed produced 913 -> 1117. #983 then landed forty rows and pruned two, and the census became 1155. Carrying 1117 forward would have been arithmetic that was true when written and false when it shipped -- so the ledger was rebuilt by re-running `merge` against the new base rather than by rebasing the file, and both numbers were re-derived after. WHY THIS SUITE, measured rather than chosen by taste: product-central heap-versus-columnar differential correctness, so a check that cannot fail there is a wrong answer nobody sees churn 16 of the last 300 commits touch it, so the gate fires cheap 19 seconds uncontended no open change touches it STABILITY ACROSS RUNS IS WHAT DECIDED IT. A suite whose checks move between runs churns the ledger and fires the gate on nothing. Two consecutive runs on PG17: 204 records, 204 distinct names, 0 duplicate keys, both runs the two sets identical in NAME and in VERDICT The verdict half is @OffgridwithJD's, and it is not covered by the name half: a flipped verdict churns `last observed red` while the keys stay still. PROOF THAT SEEDING CHANGED BEHAVIOUR, re-run against this base: after seeding, a log with one unseen differential check rc=1, REFUSED before seeding, the same log against main's ledger rc=0, not refused after seeding, the real log rc=0, no false red The middle row is the point: the gate refuses an unseen check only in a suite it covers, so before this those 204 checks were invisible to it. The third stops the first being bought with a gate that refuses everything. The tax is the gate working: adding a check to differential now needs the ledger regenerated in the same commit, which is a reviewable diff. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01EbyGSaU93XYQr8aH4NrUiw --- CHANGELOG.md | 38 +++++++ test/check_ledger.tsv | 204 +++++++++++++++++++++++++++++++++++ test/check_ledger_budget.txt | 4 +- 3 files changed, 244 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index aada155c..6c11bb48 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,44 @@ true until the next version shipped. ### Added +- The mutation ledger covers a third suite: `differential`, 204 checks (#752). + + suites_not_covered 250 -> 249 + checks_never_observed_red 951 -> 1155 + covered harness_selftest, native_join_runtime_filter, differential + + BOTH NUMBERS ARE DERIVED FROM THE FILES, never computed from the old ones, and this + change is its own argument for that rule. Written against an earlier base the same + seed produced `913 -> 1117`; #983 then landed forty rows and pruned two, and the + census became 1155. Carrying 1117 forward would have been arithmetic that was true + when it was written and false when it shipped. The census is `grep -c` over the + ledger, re-run after the rebase; the ceiling is the registered list minus the + ledger's own suites. + + WHY THIS SUITE, measured rather than chosen by taste. It is the heap-versus-columnar + differential correctness suite, so a check that cannot fail there is a wrong answer + nobody sees. 16 of the last 300 commits touch it, so the gate will fire. It runs in + 19 seconds, and no open change touches it. + + THE RISK THAT DECIDED IT WAS STABILITY ACROSS RUNS, because a suite whose checks move + between runs churns the ledger and fires the gate on nothing. Two consecutive runs on + PG17: 204 records, 204 distinct names, zero duplicate keys, and the two sets identical + in NAME AND IN VERDICT -- the second half matters because a flipped verdict churns the + `last observed red` column while the keys stay still. + + Proof that seeding changed behaviour, re-run against this base: + + after seeding, a log with one unseen differential check rc=1, REFUSED + before seeding, the same log against main's ledger rc=0, not refused + after seeding, the real log rc=0, no false red + + The middle row is the point: the gate refuses an unseen check only in a suite it + covers, so before this those 204 checks were invisible to it. The third stops the + first from being bought with a gate that refuses everything. + + The tax is the gate working: a change adding a check to `differential` now needs the + ledger regenerated in the same commit, which is a reviewable diff. + - `test/selftest/470` now has the pytest half it shipped without (#994). #998 added the shell part and no pytest twin, against the owner's rule that a test diff --git a/test/check_ledger.tsv b/test/check_ledger.tsv index d2e29c8a..d0e4769f 100644 --- a/test/check_ledger.tsv +++ b/test/check_ledger.tsv @@ -1,3 +1,207 @@ +differential differential agg avg never - +differential differential agg count never - +differential differential agg minmax never - +differential differential agg sum never - +differential differential allnull chunk isnull never - +differential differential allnull chunk range never - +differential differential allnull chunk scan never - +differential differential allnull column count never - +differential differential allnull column isnull never - +differential differential allnull column minmax never - +differential differential allnull column scan never - +differential differential bloom absent correct never - +differential differential bloom k present never - +differential differential bloom k range never - +differential differential bloom u eq never - +differential differential c_arr count never - +differential differential c_arr eq never - +differential differential c_arr is null never - +differential differential c_arr not null never - +differential differential c_arr project never - +differential differential c_big count never - +differential differential c_big is null never - +differential differential c_big min/max never - +differential differential c_big not null never - +differential differential c_big project never - +differential differential c_big range never - +differential differential c_big sum/avg never - +differential differential c_bool count never - +differential differential c_bool eq never - +differential differential c_bool is null never - +differential differential c_bool not null never - +differential differential c_bool project never - +differential differential c_bytea count never - +differential differential c_bytea is null never - +differential differential c_bytea not null never - +differential differential c_bytea project never - +differential differential c_bytea range never - +differential differential c_char count never - +differential differential c_char is null never - +differential differential c_char min/max never - +differential differential c_char not null never - +differential differential c_char project never - +differential differential c_date count never - +differential differential c_date is null never - +differential differential c_date min/max never - +differential differential c_date not null never - +differential differential c_date project never - +differential differential c_date range never - +differential differential c_f4 count never - +differential differential c_f4 is null never - +differential differential c_f4 min/max never - +differential differential c_f4 not null never - +differential differential c_f4 project never - +differential differential c_f4 sum/avg never - +differential differential c_f8 count never - +differential differential c_f8 is null never - +differential differential c_f8 min/max never - +differential differential c_f8 not null never - +differential differential c_f8 project never - +differential differential c_f8 range never - +differential differential c_f8 sum/avg never - +differential differential c_int count never - +differential differential c_int eq never - +differential differential c_int is null never - +differential differential c_int min/max never - +differential differential c_int not null never - +differential differential c_int project never - +differential differential c_int range never - +differential differential c_int sum/avg never - +differential differential c_iv count never - +differential differential c_iv is null never - +differential differential c_iv min/max never - +differential differential c_iv not null never - +differential differential c_iv project never - +differential differential c_jsonb count never - +differential differential c_jsonb eq never - +differential differential c_jsonb is null never - +differential differential c_jsonb not null never - +differential differential c_jsonb project never - +differential differential c_num count never - +differential differential c_num is null never - +differential differential c_num min/max never - +differential differential c_num not null never - +differential differential c_num project never - +differential differential c_num range never - +differential differential c_num sum/avg never - +differential differential c_small count never - +differential differential c_small is null never - +differential differential c_small min/max never - +differential differential c_small not null never - +differential differential c_small project never - +differential differential c_small sum/avg never - +differential differential c_text count never - +differential differential c_text is null never - +differential differential c_text min/max never - +differential differential c_text not null never - +differential differential c_text project never - +differential differential c_text range never - +differential differential c_ts count never - +differential differential c_ts is null never - +differential differential c_ts min/max never - +differential differential c_ts not null never - +differential differential c_ts project never - +differential differential c_ts range never - +differential differential c_tstz count never - +differential differential c_tstz is null never - +differential differential c_tstz min/max never - +differential differential c_tstz not null never - +differential differential c_tstz project never - +differential differential c_tstz range never - +differential differential c_uuid count never - +differential differential c_uuid eq never - +differential differential c_uuid is null never - +differential differential c_uuid not null never - +differential differential c_uuid project never - +differential differential c_uuid range never - +differential differential c_vc count never - +differential differential c_vc eq never - +differential differential c_vc is null never - +differential differential c_vc min/max never - +differential differential c_vc not null never - +differential differential c_vc project never - +differential differential c_vc range never - +differential differential c_ztext count never - +differential differential c_ztext is null never - +differential differential c_ztext min/max never - +differential differential c_ztext not null never - +differential differential c_ztext project never - +differential differential cg boundary N=100 groups never - +differential differential cg boundary N=100 range never - +differential differential cg boundary N=100 scan never - +differential differential cg boundary N=101 groups never - +differential differential cg boundary N=101 range never - +differential differential cg boundary N=101 scan never - +differential differential cg boundary N=200 groups never - +differential differential cg boundary N=200 range never - +differential differential cg boundary N=200 scan never - +differential differential cg boundary N=201 groups never - +differential differential cg boundary N=201 range never - +differential differential cg boundary N=201 scan never - +differential differential cg boundary N=250 groups never - +differential differential cg boundary N=250 range never - +differential differential cg boundary N=250 scan never - +differential differential cg boundary N=99 groups never - +differential differential cg boundary N=99 range never - +differential differential cg boundary N=99 scan never - +differential differential compound never - +differential differential control: the set oracle is order-blind by design never - +differential differential count meta=off never - +differential differential count meta=on never - +differential differential dict text agg never - +differential differential dict text eq never - +differential differential dict text group never - +differential differential dict whole-row never - +differential differential empty agg never - +differential differential empty count never - +differential differential empty scan never - +differential differential empty-vs-null empties never - +differential differential empty-vs-null nulls never - +differential differential empty-vs-null scan never - +differential differential enc aggregate never - +differential differential enc const scan never - +differential differential enc lowcard eq never - +differential differential enc seq range never - +differential differential enc whole-row never - +differential differential enc+nocompress agg never - +differential differential enc+nocompress scan never - +differential differential i4 float agg never - +differential differential i4 ts minmax never - +differential differential i4 ts range never - +differential differential i4 whole-row never - +differential differential matrix chunk groups>=12 never - +differential differential matrix row count never - +differential differential matrix stripes>=2 never - +differential differential matrix whole-row never - +differential differential order limit head never - +differential differential order limit tail never - +differential differential premise: the ordered oracle agrees with itself never - +differential differential premise: the ordered oracle is order-sensitive never - +differential differential single count never - +differential differential single scan never - +differential differential stripe boundary N=1000 agg never - +differential differential stripe boundary N=1000 scan never - +differential differential stripe boundary N=1000 stripes never - +differential differential stripe boundary N=1001 agg never - +differential differential stripe boundary N=1001 scan never - +differential differential stripe boundary N=1001 stripes never - +differential differential stripe boundary N=2000 agg never - +differential differential stripe boundary N=2000 scan never - +differential differential stripe boundary N=2000 stripes never - +differential differential stripe boundary N=2001 agg never - +differential differential stripe boundary N=2001 scan never - +differential differential stripe boundary N=2001 stripes never - +differential differential textbloom C absent never - +differential differential textbloom C eq never - +differential differential textbloom absent never - +differential differential textbloom collate-mismatch never - +differential differential textbloom present never - +differential differential wide most never - +differential differential wide nomatch never - +differential differential wide point never - +differential differential wide range never - +differential differential wide row proj never - +differential differential wide row scan never - harness_selftest 030-assertions nothing leaked into the squatter never - harness_selftest 030-assertions pgc_port_free says the squatter's port is busy never - harness_selftest 030-assertions squatter survived untouched never - diff --git a/test/check_ledger_budget.txt b/test/check_ledger_budget.txt index fb09efcd..9847dd7f 100644 --- a/test/check_ledger_budget.txt +++ b/test/check_ledger_budget.txt @@ -15,7 +15,7 @@ # # Adding a check to a suite that is already covered does not move it, which is # what makes it safe to bound. -suites_not_covered 250 +suites_not_covered 249 # # checks_never_observed_red -- A CENSUS. NOT a ceiling, and it must not become # one. @@ -34,4 +34,4 @@ suites_not_covered 250 # Without that it is a hand-maintained count that drifts, which is the failure # this repository has spent a day proving. It is not a ceiling; it is a # measurement that must be true. -checks_never_observed_red 951 +checks_never_observed_red 1155