Skip to content

test: seed the mutation ledger with differential, 204 checks (#752) #1002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jdatcmd merged 1 commit into from
Sep 12, 2026
+244 −2
Merged

test: seed the mutation ledger with differential, 204 checks (#752) #1002

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,44 @@ true until the next version shipped.

### Added

- The mutation ledger covers a third suite: `differential`, 204 checks (#752).

suites_not_covered 250 -> 249
checks_never_observed_red 951 -> 1155
covered harness_selftest, native_join_runtime_filter, differential

BOTH NUMBERS ARE DERIVED FROM THE FILES, never computed from the old ones, and this
change is its own argument for that rule. Written against an earlier base the same
seed produced `913 -> 1117`; #983 then landed forty rows and pruned two, and the
census became 1155. Carrying 1117 forward would have been arithmetic that was true
when it was written and false when it shipped. The census is `grep -c` over the
ledger, re-run after the rebase; the ceiling is the registered list minus the
ledger's own suites.

WHY THIS SUITE, measured rather than chosen by taste. It is the heap-versus-columnar
differential correctness suite, so a check that cannot fail there is a wrong answer
nobody sees. 16 of the last 300 commits touch it, so the gate will fire. It runs in
19 seconds, and no open change touches it.

THE RISK THAT DECIDED IT WAS STABILITY ACROSS RUNS, because a suite whose checks move
between runs churns the ledger and fires the gate on nothing. Two consecutive runs on
PG17: 204 records, 204 distinct names, zero duplicate keys, and the two sets identical
in NAME AND IN VERDICT -- the second half matters because a flipped verdict churns the
`last observed red` column while the keys stay still.

Proof that seeding changed behaviour, re-run against this base:

after seeding, a log with one unseen differential check rc=1, REFUSED
before seeding, the same log against main's ledger rc=0, not refused
after seeding, the real log rc=0, no false red

The middle row is the point: the gate refuses an unseen check only in a suite it
covers, so before this those 204 checks were invisible to it. The third stops the
first from being bought with a gate that refuses everything.

The tax is the gate working: a change adding a check to `differential` now needs the
ledger regenerated in the same commit, which is a reviewable diff.

- `test/selftest/470` now has the pytest half it shipped without (#994).

#998 added the shell part and no pytest twin, against the owner's rule that a test
Expand Down
204 changes: 204 additions & 0 deletions test/check_ledger.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,207 @@
differential differential agg avg never -
differential differential agg count never -
differential differential agg minmax never -
differential differential agg sum never -
differential differential allnull chunk isnull never -
differential differential allnull chunk range never -
differential differential allnull chunk scan never -
differential differential allnull column count never -
differential differential allnull column isnull never -
differential differential allnull column minmax never -
differential differential allnull column scan never -
differential differential bloom absent correct never -
differential differential bloom k present never -
differential differential bloom k range never -
differential differential bloom u eq never -
differential differential c_arr count never -
differential differential c_arr eq never -
differential differential c_arr is null never -
differential differential c_arr not null never -
differential differential c_arr project never -
differential differential c_big count never -
differential differential c_big is null never -
differential differential c_big min/max never -
differential differential c_big not null never -
differential differential c_big project never -
differential differential c_big range never -
differential differential c_big sum/avg never -
differential differential c_bool count never -
differential differential c_bool eq never -
differential differential c_bool is null never -
differential differential c_bool not null never -
differential differential c_bool project never -
differential differential c_bytea count never -
differential differential c_bytea is null never -
differential differential c_bytea not null never -
differential differential c_bytea project never -
differential differential c_bytea range never -
differential differential c_char count never -
differential differential c_char is null never -
differential differential c_char min/max never -
differential differential c_char not null never -
differential differential c_char project never -
differential differential c_date count never -
differential differential c_date is null never -
differential differential c_date min/max never -
differential differential c_date not null never -
differential differential c_date project never -
differential differential c_date range never -
differential differential c_f4 count never -
differential differential c_f4 is null never -
differential differential c_f4 min/max never -
differential differential c_f4 not null never -
differential differential c_f4 project never -
differential differential c_f4 sum/avg never -
differential differential c_f8 count never -
differential differential c_f8 is null never -
differential differential c_f8 min/max never -
differential differential c_f8 not null never -
differential differential c_f8 project never -
differential differential c_f8 range never -
differential differential c_f8 sum/avg never -
differential differential c_int count never -
differential differential c_int eq never -
differential differential c_int is null never -
differential differential c_int min/max never -
differential differential c_int not null never -
differential differential c_int project never -
differential differential c_int range never -
differential differential c_int sum/avg never -
differential differential c_iv count never -
differential differential c_iv is null never -
differential differential c_iv min/max never -
differential differential c_iv not null never -
differential differential c_iv project never -
differential differential c_jsonb count never -
differential differential c_jsonb eq never -
differential differential c_jsonb is null never -
differential differential c_jsonb not null never -
differential differential c_jsonb project never -
differential differential c_num count never -
differential differential c_num is null never -
differential differential c_num min/max never -
differential differential c_num not null never -
differential differential c_num project never -
differential differential c_num range never -
differential differential c_num sum/avg never -
differential differential c_small count never -
differential differential c_small is null never -
differential differential c_small min/max never -
differential differential c_small not null never -
differential differential c_small project never -
differential differential c_small sum/avg never -
differential differential c_text count never -
differential differential c_text is null never -
differential differential c_text min/max never -
differential differential c_text not null never -
differential differential c_text project never -
differential differential c_text range never -
differential differential c_ts count never -
differential differential c_ts is null never -
differential differential c_ts min/max never -
differential differential c_ts not null never -
differential differential c_ts project never -
differential differential c_ts range never -
differential differential c_tstz count never -
differential differential c_tstz is null never -
differential differential c_tstz min/max never -
differential differential c_tstz not null never -
differential differential c_tstz project never -
differential differential c_tstz range never -
differential differential c_uuid count never -
differential differential c_uuid eq never -
differential differential c_uuid is null never -
differential differential c_uuid not null never -
differential differential c_uuid project never -
differential differential c_uuid range never -
differential differential c_vc count never -
differential differential c_vc eq never -
differential differential c_vc is null never -
differential differential c_vc min/max never -
differential differential c_vc not null never -
differential differential c_vc project never -
differential differential c_vc range never -
differential differential c_ztext count never -
differential differential c_ztext is null never -
differential differential c_ztext min/max never -
differential differential c_ztext not null never -
differential differential c_ztext project never -
differential differential cg boundary N=100 groups never -
differential differential cg boundary N=100 range never -
differential differential cg boundary N=100 scan never -
differential differential cg boundary N=101 groups never -
differential differential cg boundary N=101 range never -
differential differential cg boundary N=101 scan never -
differential differential cg boundary N=200 groups never -
differential differential cg boundary N=200 range never -
differential differential cg boundary N=200 scan never -
differential differential cg boundary N=201 groups never -
differential differential cg boundary N=201 range never -
differential differential cg boundary N=201 scan never -
differential differential cg boundary N=250 groups never -
differential differential cg boundary N=250 range never -
differential differential cg boundary N=250 scan never -
differential differential cg boundary N=99 groups never -
differential differential cg boundary N=99 range never -
differential differential cg boundary N=99 scan never -
differential differential compound never -
differential differential control: the set oracle is order-blind by design never -
differential differential count meta=off never -
differential differential count meta=on never -
differential differential dict text agg never -
differential differential dict text eq never -
differential differential dict text group never -
differential differential dict whole-row never -
differential differential empty agg never -
differential differential empty count never -
differential differential empty scan never -
differential differential empty-vs-null empties never -
differential differential empty-vs-null nulls never -
differential differential empty-vs-null scan never -
differential differential enc aggregate never -
differential differential enc const scan never -
differential differential enc lowcard eq never -
differential differential enc seq range never -
differential differential enc whole-row never -
differential differential enc+nocompress agg never -
differential differential enc+nocompress scan never -
differential differential i4 float agg never -
differential differential i4 ts minmax never -
differential differential i4 ts range never -
differential differential i4 whole-row never -
differential differential matrix chunk groups>=12 never -
differential differential matrix row count never -
differential differential matrix stripes>=2 never -
differential differential matrix whole-row never -
differential differential order limit head never -
differential differential order limit tail never -
differential differential premise: the ordered oracle agrees with itself never -
differential differential premise: the ordered oracle is order-sensitive never -
differential differential single count never -
differential differential single scan never -
differential differential stripe boundary N=1000 agg never -
differential differential stripe boundary N=1000 scan never -
differential differential stripe boundary N=1000 stripes never -
differential differential stripe boundary N=1001 agg never -
differential differential stripe boundary N=1001 scan never -
differential differential stripe boundary N=1001 stripes never -
differential differential stripe boundary N=2000 agg never -
differential differential stripe boundary N=2000 scan never -
differential differential stripe boundary N=2000 stripes never -
differential differential stripe boundary N=2001 agg never -
differential differential stripe boundary N=2001 scan never -
differential differential stripe boundary N=2001 stripes never -
differential differential textbloom C absent never -
differential differential textbloom C eq never -
differential differential textbloom absent never -
differential differential textbloom collate-mismatch never -
differential differential textbloom present never -
differential differential wide most never -
differential differential wide nomatch never -
differential differential wide point never -
differential differential wide range never -
differential differential wide row proj never -
differential differential wide row scan never -
harness_selftest 030-assertions nothing leaked into the squatter never -
harness_selftest 030-assertions pgc_port_free says the squatter's port is busy never -
harness_selftest 030-assertions squatter survived untouched never -
Expand Down
4 changes: 2 additions & 2 deletions test/check_ledger_budget.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
#
# Adding a check to a suite that is already covered does not move it, which is
# what makes it safe to bound.
suites_not_covered 250
suites_not_covered 249
#
# checks_never_observed_red -- A CENSUS. NOT a ceiling, and it must not become
# one.
Expand All @@ -34,4 +34,4 @@ suites_not_covered 250
# Without that it is a hand-maintained count that drifts, which is the failure
# this repository has spent a day proving. It is not a ceiling; it is a
# measurement that must be true.
checks_never_observed_red 951
checks_never_observed_red 1155
Loading