diff --git a/CHANGELOG.md b/CHANGELOG.md index d6f5579e..2779b8c5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,98 @@ true until the next version shipped. ### Added +- The gate refuses two checks that share one ledger key, instead of printing a note + about them (#982). + + AND THE SAME CLASS ONE LEVEL DOWN, found while building the arm for the first. `read_ledger` + did `rows[(f[0], f[1], f[2])] = [...]`, so a duplicated key in the TRACKED file collapsed + silently and the last line won. Measured on a two-line fixture, both orders: + + never first, then 2026-09-01 survivor last_red='2026-09-01' + 2026-09-01 first, then never survivor last_red='never' the red is GONE + + Line order decided whether a recorded red observation survived. A merge that keeps both + sides of a changed row turns `ever red` back into `never`. That is what #918 and #925 exist + to prevent, arriving from the opposite direction. + + NOTHING ELSE COULD CATCH IT, and bounding the census cannot. `check_ledger_budget.txt` says + `checks_never_observed_red` is a CENSUS and must not become a ceiling, because every new + check enters as `never` and bounding it deadlocks. The gate compares the budget's number + with the ledger's, and both come from the same dict, so they agree either way. Measured: + with the budget regenerated alongside, an erased red passes the gate at rc=0. + + It is refused as an INTEGRITY FAILURE (rc=2) rather than a gate verdict, beside the other + inputs that do not parse. A ledger that cannot be trusted is not a gate result. + + So the two halves are the same shape at two levels. A SET hid two checks in one run; a DICT + hid two rows in one file. The question that found both is what the input canonicalises + before the guard sees it. + + AND A THIRD, IN THE ARM THAT POLICES THE OTHER TRACKED KEY-VALUE FILE. + `test_harness_deps.py` read `expected_tests.txt` with `nums[f[0]] = int(f[1])`, so a + duplicated key collapsed and the last line won, exactly as `read_ledger` did. Measured on + one fixture read both ways: + + the line form names guard_tests as duplicated + the dict form sees two keys and keeps 280, the LAST line + + THIS IS NOT HYPOTHETICAL. Three PRs were open at once, each moving `guard_tests`, and + resolving all three keep-both produced three of those lines. `ci.yml` reads the value with + `awk '$1=="guard_tests"{print $2}'`, which prints one line per match. So `WANT` becomes + multi-line, `test -n "$WANT"` still passes, and the flag refuses it at exit 4: + + pytest: error: argument --pgc-expect-tests: invalid int value: '284\n280\n283' + + It FAILS CLOSED, so this is legibility rather than a hole. What exit 4 does not say is that a + line is duplicated. Two arms now say it. The removal proof on the real file reddens the new + arm and leaves the pre-existing one green, which is the point. + + Keep-both is right for a changelog and wrong for a key-value file. Nothing in the tree said + so. + + `cluster_tests` 205 -> 207, not `guard_tests`: `test_harness_deps.py` DEFINES `NO_CLUSTER` + and is not in it. I got that wrong first and the mechanism caught it, which is the argument + for the mechanism. + + A ledger row is keyed on `(suite, part, name)`, so two checks with the same name in one + part share a row. Nothing is mis-recorded while both pass. The hazard is exact: when one + goes red the row records `ever red`, and its namesake inherits a red observation nothing + attacked. `checks_never_observed_red` then falls by one for a check nobody attacked, and + that census is what #918 and #925 exist to make trustworthy. + + `merge` has detected this since #982 was filed, and returns 0. That is how three of them + sat in one part of `selftest/400` for a day. The instance was fixed by `c3b13aed`; this is + the mechanism, which that issue called the more valuable half. + + THE GATE COULD NOT SEE IT AT ALL, and the reason is worth recording. `cmd_gate` builds its + records as `sorted({(s, p, n, m) for ...})`. A set collapses the duplicate before any arm + can count it. So the same canonicalisation that makes the rest of the gate correct made + this one class unreachable. The count now comes from the raw records through `_by_run`. + + PER LOG, because one check observed in two logs is two RUNS of it. That is the normal case + and the way the ledger accumulates evidence at all. Only a repeat inside one log is a + collision. An arm pins the distinction, because written over the logs together the refusal + would reject every multi-day merge. + + EVERY SUITE, DELIBERATELY UNLIKE THE NEW-CHECK REFUSAL. That one is restricted to suites + the ledger covers because it cannot know which of an uncovered suite's checks are new. + This one needs no history: two records, one key, one log is decidable from the log alone. + The ledger covers four suites of 253. Copying the restriction would close the class in four + places only. The next collision would then sit in one of the other 249 until that suite was + seeded. + + MEASURED BEFORE WIDENING IT, because a gate that reddens 250 unmeasured suites is a gate + somebody turns off. A full PG 18 matrix run with the refusal armed for every suite: + + suites that ran 247 (6 skipped, 0 incomplete) + matrix verdict ALL VERSIONS PASSED, RC=0 + shared ledger keys found 0 + + Check names are static, so one major's matrix measures this class completely rather than + sampling it. A real `harness_selftest` log says the same end to end. Run against the + COMMITTED ledger and budget, the gate returns 0 with no shared-key line, over 934 records + and 934 distinct keys. + - `projection_privilege.sh` has a pytest twin, and both halves now attribute a refusal by SQLSTATE instead of by error text (#432, #562, #563). diff --git a/test/pgc_ledger.py b/test/pgc_ledger.py index 0322cf12..25304ee4 100755 --- a/test/pgc_ledger.py +++ b/test/pgc_ledger.py @@ -311,7 +311,34 @@ def read_ledger(path): f"{path}:{n}: major {m!r} is neither a number nor " f"{MAJOR_UNKNOWN!r}, so this row names no version it applies to") muts = set() if f[5] == NONE else {m for m in f[5].split(";") if m} - rows[(f[0], f[1], f[2])] = [majors, f[4] or NEVER, muts] + key = (f[0], f[1], f[2]) + # A REPEATED KEY, REFUSED RATHER THAN OVERWRITTEN. This was + # `rows[key] = [...]`, so a duplicated row collapsed silently and the LAST line + # won. Measured on a two-line fixture, both orders: + # + # never first, then 2026-09-01 survivor last_red='2026-09-01' + # 2026-09-01 first, then never survivor last_red='never' <- the red is GONE + # + # So line order decided whether a recorded red observation survived, and a merge + # that keeps both sides of a changed row turns `ever red` back into `never` -- + # the corruption #918 and #925 exist to prevent. + # + # NOTHING ELSE CAN CATCH IT, and bounding the census cannot. The budget file says + # `checks_never_observed_red` is a CENSUS and must not become a ceiling, because + # every new check enters as `never` and bounding it deadlocks. The gate compares + # the budget's number with the ledger's, and both come from this dict, so they + # agree either way. Measured: with the budget regenerated alongside, an erased red + # passes the gate at rc=0. + # + # The same shape as the shared-key refusal in `cmd_gate`, one level down: there a + # set hid two checks in one RUN, here a dict hid two rows in one FILE. + if key in rows: + raise LedgerError( + f"{path}:{n}: a ledger row repeats a key already in this file: " + f"{f[0]}\t{f[1]}\t{f[2]}. One key is one check, and the later row would " + f"silently replace the earlier -- which loses a recorded red if the later " + f"row says {NEVER!r}. Keep one row per check.") + rows[key] = [majors, f[4] or NEVER, muts] return rows @@ -866,6 +893,60 @@ def cmd_gate(args): rc = 0 + covered_suites = {k[0] for k in rows} + # AND THE SAME ARGUMENT FOR THE MAJOR (#1010). The gate cannot refuse a new check on a + # major it holds no rows for, for the identical reason it cannot in a suite it has + # never seen: it has no idea which of that major's checks are new. Adding PG20 to the + # matrix would otherwise redden every check at once, which is a gate somebody turns + # off -- the failure this issue family exists to prevent. It tightens on its own the + # moment one run on that major is merged. + covered_majors = set().union(*(v[0] for v in rows.values())) if rows else set() + + # TWO CHECKS SHARING ONE LEDGER KEY, refused rather than noted (#982). + # + # A row is keyed on (suite, part, name), so two checks with the same name in one + # part share a row. Nothing is mis-recorded while both pass, and the hazard is + # exact: when one goes red the row records `ever red` and its namesake inherits a + # red observation nothing attacked. `checks_never_observed_red` then falls by one + # for a check nobody attacked, and that census is what #918 and #925 exist to make + # trustworthy. + # + # `merge` has printed this since #982 was filed and returns 0, which is how three + # of them sat in one part of selftest/400 for a day. THE GATE COULD NOT SEE IT AT + # ALL: `records` above is a set, and a set collapses the duplicate before any arm + # can count it. The same canonicalisation that makes the rest of this function + # correct made this one class unreachable, so the count comes from the raw records. + # + # PER LOG, via _by_run, because one check observed in two logs is two RUNS of it -- + # the normal case, and how the ledger accumulates evidence at all. Only a repeat + # inside one log is a collision. + # + # EVERY SUITE, DELIBERATELY UNLIKE THE NEW-CHECK REFUSAL BELOW. That one is + # restricted to suites the ledger covers because it cannot know which of an + # uncovered suite's checks are new. This one needs no history: two records, one + # key, one log is decidable from the log alone. Refusing everywhere is what makes + # the class impossible rather than impossible in the four suites seeded so far, and + # the next collision is likelier to arrive in one of the other 249. + # + # MEASURED BEFORE WIDENING IT, because a gate that reddens 250 unmeasured suites is + # a gate somebody turns off. A full PG 18 matrix run with this refusal armed for + # every suite: 247 suites ran, RC=0, ALL VERSIONS PASSED, zero shared keys. Check + # names are static, so one major's matrix is a complete measurement of this class + # rather than a sample of it. + shared = [] + for path, seen in _by_run(args.logs): + for (suite, part, name), verdicts in sorted(seen.items()): + if len(verdicts) > 1: + shared.append((path, suite, part, name, [v for v, _m in verdicts])) + for path, suite, part, name, verdicts in shared: + print(f" one ledger key covers {len(verdicts)} checks in {path}: " + f"{suite}\t{part}\t{name}\t({', '.join(verdicts)})") + if shared: + print(f" {len(shared)} ledger key(s) cover more than one check. Give each " + f"check a name that says which it is; a shared key records one check's " + f"red against the other.") + rc = 1 + # THE REFUSAL: a check the committed ledger has never seen, IN A SUITE THE # LEDGER COVERS. # @@ -878,14 +959,6 @@ def cmd_gate(args): # # It tightens on its own as suites are seeded, and the ceiling is what forces # that direction. - covered_suites = {k[0] for k in rows} - # AND THE SAME ARGUMENT FOR THE MAJOR (#1010). The gate cannot refuse a new check on a - # major it holds no rows for, for the identical reason it cannot in a suite it has - # never seen: it has no idea which of that major's checks are new. Adding PG20 to the - # matrix would otherwise redden every check at once, which is a gate somebody turns - # off -- the failure this issue family exists to prevent. It tightens on its own the - # moment one run on that major is merged. - covered_majors = set().union(*(v[0] for v in rows.values())) if rows else set() unknown = [] for key in records: if key[0] not in covered_suites or key[3] not in covered_majors: diff --git a/test/pytest/TESTS.md b/test/pytest/TESTS.md index 2c88f5c1..93bf7fa4 100644 --- a/test/pytest/TESTS.md +++ b/test/pytest/TESTS.md @@ -1599,6 +1599,8 @@ fixtures are read off `conftest.py` rather than named in the classifier. | `test_the_job_installs_no_database_driver` | the job asserts psycopg is absent rather than assuming it | | `test_the_cluster_job_runs_the_other_half_and_derives_it` | the complement of `NO_CLUSTER` is RUN, derived not listed, and asserts the driver IS present | | `test_both_pytest_jobs_assert_how_many_tests_they_collected` | both jobs pass `--pgc-expect-tests` from the tracked file, and each guards the read | +| `test_each_expected_count_is_stated_exactly_once` | `expected_tests.txt` states each key on one line — a keep-both merge duplicates it, `awk` then hands the flag a multi-line value, and the arm above could not see it because it builds a dict | +| `test_a_duplicated_count_is_caught_and_the_dict_form_is_not` | **removal proof**: the same fixture read both ways, so the dict form is shown keeping the LAST line while the line form names the duplicate | | `test_the_shell_reference_detector_sees_code_and_not_prose` | the premise: a docstring is prose, a string passed to bash is a reference, an f-string counts once | | `test_the_harness_independence_inventory_is_exactly_what_the_corpus_does` | CONTEXT.md's inventory, asserted in both directions | @@ -1612,6 +1614,23 @@ passed, so the numbers reach a reader from the run. A written count is a hand-maintained derived value, and the one in the job's comment was wrong the day it was written (#908). +**A keep-both merge of this file duplicates a key.** CI then says something unrecognisable. +`ci.yml` reads the value with `awk '$1=="guard_tests"{print $2}'`, which prints one line per +match. Two matches make `WANT` multi-line, `test -n "$WANT"` still passes, and the flag +refuses it: + + pytest: error: argument --pgc-expect-tests: invalid int value: '284\n280\n283' + exit 4 + +So it **fails closed**, which is why these arms are about legibility rather than a hole. What +exit 4 does not say is that a line is duplicated. The reader has to work back from an int +parse error to a merge resolution. + +**Not hypothetical.** Three PRs were open at once, each moving `guard_tests`, and resolving all +three keep-both produced exactly that. Keep-both is right for a changelog and wrong for a +key-value file, and nothing in the tree said so. The arm that read this file built +`nums[f[0]] = int(f[1])`, which is the same shape `read_ledger` had before #982 — one file over. + BUT THE ARMS IN THIS FILE DO NOT RUN IN THE GATE, and that is why `test/selftest/350-the-pytest-corpus-must-be.sh` runs the membership decision through this module's command line. The corpus is not in `SUITES` (README.md), and @@ -2364,6 +2383,88 @@ it the gate refuses every check of every uncovered suite and reddens the whole m on its first run. It tightens on its own as suites are seeded, and the deadlock that shipped is pinned as its own arm — regenerating the ledger lets a new check through. +### `test_the_ledger_refuses_two_rows_sharing_one_key` + +The same class one level down, found while building the arm below. `read_ledger` did +`rows[(f[0], f[1], f[2])] = [...]`, so a duplicated key in the **tracked file** collapsed +silently and the **last line won**. Measured on a two-line fixture, both orders: + + never first, then 2026-09-01 survivor last_red='2026-09-01' + 2026-09-01 first, then never survivor last_red='never' <- the red is GONE + +So line order decided whether a recorded red observation survived. A merge that keeps both +sides of a changed row turns `ever red` back into `never`. That is the corruption #918 and +#925 exist to prevent, arriving from the opposite direction to #982's. + +**Nothing else could catch it, and bounding the census cannot.** +`check_ledger_budget.txt` says `checks_never_observed_red` is a CENSUS and must not become a +ceiling. Every new check enters as `never`, so bounding it deadlocks. The gate compares the +budget's number with the ledger's, and both come from the same dict, so they agree either +way. Measured: with the budget regenerated alongside, an erased red passes the gate at +**rc=0**. + +It is refused as an **integrity failure** (rc=2) rather than a gate verdict, beside the other +inputs that do not parse. A ledger that cannot be trusted is not a gate result. + +### `test_a_ledger_with_no_duplicate_key_still_loads` + +The false-positive budget, and the premise the refusal needs. Two rows differing only in the +NAME are two checks and must load, which is the ordinary case for every part in the tree. The +arm also asserts `rows=3`, so the refusal cannot pass by eating a row. + +### `test_the_gate_refuses_two_checks_sharing_one_ledger_key` + +A row is keyed on `(suite, part, name)`, so two checks with the same name in one part +share a row. Nothing is mis-recorded while both pass. The hazard is exact: when one goes +red the row records `ever red` and its namesake inherits a red observation nothing +attacked. `checks_never_observed_red` then falls by one for a check nobody attacked, +which is the census #918 and #925 exist to make trustworthy. + +**`merge` already printed this and returned 0**, which is how three of them sat in one +part of `selftest/400` for a day (#982). The instance was fixed by `c3b13aed`; this is +the mechanism. + +**The gate could not see it at all, by construction.** `cmd_gate` builds its records as +`sorted({(s, p, n, m) for ...})`. A set collapses the duplicate before any arm can count +it. So the same canonicalisation that makes the rest of the gate correct made this one +class unreachable. The count now comes from the raw records, through `_by_run`. + +Measured on main at `03c6c9c8`. A real `harness_selftest` run emits **934 RESULT records +over 934 distinct keys, 0 collisions**. The gate run against the COMMITTED ledger and +budget returns 0 with no shared-key line. A full PG 18 matrix says the same for the whole +corpus: 247 suites, RC=0, zero shared keys. + +### `test_a_shared_key_is_refused_in_a_suite_the_ledger_does_not_cover` + +**The refusal covers every suite, unlike the new-check refusal.** That one is restricted +for a reason. It cannot know which of an uncovered suite's checks are new. This one needs +no history. Two records, one key, one log is decidable from the log alone. + +Why that matters rather than being a detail. The ledger covers **four suites of 253**. A +refusal restricted the same way would close the class in four places only. The next +collision would then arrive in one of the other 249 and sit there until that suite is +seeded. This arm is what stops the restriction being copied in by habit. + +**Measured before widening it**, because a gate that reddens 250 unmeasured suites is a gate +somebody turns off. A full PG 18 matrix ran with the refusal armed for every suite: + + 247 suites ran, RC=0, ALL VERSIONS PASSED, zero shared keys + +Check names are static, so one major's matrix measures this class completely rather than +sampling it. + +### `test_the_same_check_in_two_runs_is_not_a_shared_key` + +The distinction the refusal must not lose. One check observed on two days is two records +for one key and is the normal case. It is how the ledger accumulates evidence at all. +Only a repeat inside ONE log is a collision, which is why the count goes through +`_by_run` rather than over the logs together. Written the other way it would reject +every multi-day merge. + +### `test_a_clean_run_is_not_refused_for_a_shared_key` + +The false-positive budget: two different names in one part pass, and the gate returns 0. + ### `test_the_ceiling_may_only_fall_and_that_is_enforced` The tracked file says the ceiling may only fall. Without a mechanism that is prose, and diff --git a/test/pytest/expected_tests.txt b/test/pytest/expected_tests.txt index 39e5213b..92192835 100644 --- a/test/pytest/expected_tests.txt +++ b/test/pytest/expected_tests.txt @@ -51,7 +51,12 @@ # 290 -> 298 when test_compare_to_bash.py landed: eight arms over the parity tool's own # extractors. Re-derived by collection on the merged tree, per the recipe above: # `298 tests collected`. -guard_tests 298 +# 298 -> 304: six arms in test_mutation_ledger.py about one ledger key covering +# more than one check (#982) -- the refusal, its uncovered-suite control, the two-runs +# distinction, the false-positive budget, and two for the same class one level down, where a +# duplicated row in the tracked ledger file collapsed silently and let the last line win. +# Re-derived by collection on the merged tree, per the recipe above: `304 tests collected`. +guard_tests 304 # The complement: tests that need the driver and a throwaway cluster. Until #1016 these ran # in no CI job at all -- a quarter of the corpus, green when somebody ran them by hand and @@ -74,4 +79,14 @@ guard_tests 298 # 205 -> 217 when test_projection_privilege.py landed: twelve arms over the two projection # read helpers, parametrised per function so each carries the bash suite's own check name. # Re-derived by collection. -cluster_tests 217 +# 217 -> 219: two arms in test_harness_deps.py asserting THIS FILE states +# each key exactly once (#982). A keep-both merge of three PRs each moving guard_tests produced +# three of those lines, and the arm that read this file built a dict, so it could not see the +# duplicate. +# +# IN THE CLUSTER HALF, NOT THE GUARD HALF, which I got wrong first and this mechanism caught +# with `collected 283 test(s) but expected 285`: test_harness_deps.py DEFINES NO_CLUSTER and +# is not IN it. +# +# Re-derived by collection on the merged tree: `219 tests collected`. +cluster_tests 219 diff --git a/test/pytest/test_harness_deps.py b/test/pytest/test_harness_deps.py index 30c166bf..4ef25dd3 100644 --- a/test/pytest/test_harness_deps.py +++ b/test/pytest/test_harness_deps.py @@ -983,6 +983,83 @@ def test_both_pytest_jobs_assert_how_many_tests_they_collected(expect): "and each guards the read, because an empty value would fail open") +# A TRACKED KEY-VALUE FILE MUST NOT REPEAT A KEY, and the arm above could not see one. +# It builds `nums[f[0]] = int(f[1])`, so a duplicated key collapses and the last line +# wins -- the same shape as `read_ledger` before #982, one file over. A seam so the rule +# has fixtures, because the real file has no duplicate to redden on. +def _count_lines(text): + """-> [(key, value)] for every `name N` line, REPEATS INCLUDED.""" + out = [] + for line in text.splitlines(): + if line.startswith("#"): + continue + f = line.split() + if len(f) == 2 and f[1].isdigit(): + out.append((f[0], int(f[1]))) + return out + + +def test_each_expected_count_is_stated_exactly_once(expect): + """A merge that keeps both sides of this file duplicates a key, and CI says something + unrecognisable instead of saying that. + + `ci.yml` reads the value with `awk '$1=="guard_tests"{print $2}'`, which prints one + line per match. Two matches make `WANT` multi-line, `test -n "$WANT"` still passes, + and the flag then refuses it. Measured, with two and with three duplicated lines: + + pytest: error: argument --pgc-expect-tests: invalid int value: '284\n280\n283' + exit 4 + + SO IT FAILS CLOSED, which is why this is an arm about legibility rather than a hole. + Exit 4 reddens the job. What it does not do is say that a line is duplicated, and the + person reading it has to work back from an int parse error to a merge resolution. + + THIS IS NOT HYPOTHETICAL. Three PRs of mine were open at once, each moving + `guard_tests`, and a keep-both resolution across all three produced exactly this: + + guard_tests 284 + guard_tests 280 + guard_tests 283 + cluster_tests 205 + + Keep-both is the right resolution for a changelog and the wrong one for a key-value + file, and nothing in the tree said so. + """ + counts = HERE / "expected_tests.txt" + pairs = _count_lines(counts.read_text()) + expect.at_least(len(pairs), 2, "premise: the file states counts to check") + keys = [k for k, _v in pairs] + dupes = sorted({k for k in keys if keys.count(k) > 1}) + expect.text(", ".join(dupes) or "none", "none", + "no key is stated twice in expected_tests.txt") + expect.num(len(keys), len(set(keys)), + "so the lines and the distinct keys are the same count") + + +def test_a_duplicated_count_is_caught_and_the_dict_form_is_not(expect): + """The removal proof, and it shows WHY the arm above is not the one that existed. + + The older reading collapses the duplicate into a dict and reports a healthy file, so + the two forms are run side by side on the same fixture. + """ + fixture = ("# a comment\n" + "guard_tests 284\n" + "guard_tests 280\n" + "cluster_tests 205\n") + pairs = _count_lines(fixture) + keys = [k for k, _v in pairs] + expect.num(len(pairs), 3, "premise: the line reading sees all three lines") + expect.text(", ".join(sorted({k for k in keys if keys.count(k) > 1})), "guard_tests", + "the duplicated key is named") + + collapsed = {} + for k, v in pairs: + collapsed[k] = v + expect.num(len(collapsed), 2, "while the dict form sees only two keys") + expect.num(collapsed["guard_tests"], 280, + "and keeps the LAST line, which is how the duplicate stayed invisible") + + def test_the_job_installs_no_database_driver(expect): """The job's value is that it runs where there is no database. diff --git a/test/pytest/test_mutation_ledger.py b/test/pytest/test_mutation_ledger.py index b6e3bfe4..79436161 100644 --- a/test/pytest/test_mutation_ledger.py +++ b/test/pytest/test_mutation_ledger.py @@ -444,6 +444,197 @@ def test_the_committed_ledger_and_budget_agree(expect): "and the ceiling matches the suites with no rows") +def test_the_ledger_refuses_two_rows_sharing_one_key(tmp_path, expect): + """The same class one level down, found while building the arm above. + + `read_ledger` does `rows[(f[0], f[1], f[2])] = [...]`, so a duplicated key in the + TRACKED file collapsed silently and the LAST line won. Measured on a two-line + fixture, both orders: + + never first, then 2026-09-01 survivor last_red='2026-09-01' + 2026-09-01 first, then never survivor last_red='never' <- the red is GONE + + So line order decides whether a recorded red observation survives, and a merge that + keeps both sides of a changed row can turn `ever red` back into `never`. That is the + exact corruption #918 and #925 exist to prevent, arriving from the opposite direction + to #982's. + + NOTHING CAUGHT IT, and bounding the census cannot. `checks_never_observed_red` is a + CENSUS and the budget file says it must not become a ceiling, because every new check + enters as `never` and bounding it deadlocks. The gate only checks that the budget's + number equals the ledger's, and both are derived from the collapsed dict, so they + agree. Measured: with the budget regenerated alongside, the erasure passes at rc=0. + """ + dup = _w(tmp_path, "l.tsv", + "demo\tpart1\ta check\t18\t2026-09-01\tmut-A\n" + "demo\tpart1\ta check\t18\tnever\t-\n") + budget = _w(tmp_path, "b.txt", "suites_not_covered 0\n") + reg = _w(tmp_path, "reg", "demo\n") + log = _w(tmp_path, "r.log", + "RESULT\tdemo\tpart1\ta check\tPASS\t18\t\nchecks run: 1\n") + out, rc = _run("gate", "--ledger", dup, "--budget", budget, "--registered", reg, log) + expect.num(int("a ledger row repeats a key" in out), 1, + "a duplicated ledger key is refused by name") + expect.num(int("a check" in out), 1, "and the row is named") + expect.num(rc, 2, "as an integrity failure, not a gate verdict") + + +def test_a_ledger_with_no_duplicate_key_still_loads(tmp_path, expect): + """The false-positive budget, and the premise the refusal above needs. + + Two rows that differ only in the NAME are two checks and must load, which is the + ordinary case for every part in the tree. + """ + ok = _w(tmp_path, "l.tsv", + "demo\tpart1\tfirst check\t18\tnever\t-\n" + "demo\tpart1\tsecond check\t18\t2026-09-01\tmut-A\n" + "demo\tpart2\tfirst check\t18\tnever\t-\n") + budget = _w(tmp_path, "b.txt", "suites_not_covered 0\n") + reg = _w(tmp_path, "reg", "demo\n") + log = _w(tmp_path, "r.log", + "RESULT\tdemo\tpart1\tfirst check\tPASS\t18\t\nchecks run: 1\n") + out, rc = _run("gate", "--ledger", ok, "--budget", budget, "--registered", reg, log) + expect.num(int("a ledger row repeats a key" in out), 0, + "three distinct keys are not a duplicate") + expect.num(int("ledger census: rows=3" in out), 1, + "and all three rows loaded, so the refusal did not eat one") + + +def test_the_gate_refuses_two_checks_sharing_one_ledger_key(tmp_path, expect): + """#982's remaining half. `merge` PRINTS this and nothing fails on it. + + A ledger row is keyed on (suite, part, name), so two checks with the same name in + one part share a row. Neither is mis-recorded while both pass. The hazard is + conditional and exact: if one goes red the row records `ever red`, and its + namesake inherits a red observation nothing attacked -- which is the census + `checks_never_observed_red` exists to make trustworthy. + + THE GATE WAS STRUCTURALLY BLIND TO IT, which is why the note was not enough. + `cmd_gate` builds its records as `sorted({(s, p, n, m) for ...})`, and a set + collapses the duplicate before any arm can count it. The same canonicalisation + that makes the rest of the gate correct made this one class unreachable. + + Measured on main at `03c6c9c8`: a real `harness_selftest` run emits 934 RESULT + records over 934 distinct keys, 0 collisions, so this refusal is green on the tree + it lands in. The instance #982 reported was fixed by `c3b13aed`; this is the + mechanism that stops the next one. + """ + # THE FIXTURE HAS TO DEFEAT THREE OTHER ROUTES TO rc=1, and an arm asserting rc == 1 + # is worth nothing until it has. Each was measured reaching 1 on its own. + # + # 1. THE COVERAGE CEILING. With `suites_not_covered 0` the gate returns 1 for + # `suites_not_covered: 1 exceeds the ceiling of 0`. The budget names 1 instead. + # + # 2. THE NEW-CHECK REFUSAL, which is the route I missed and @jdatcmd found by + # mutation. The ledger must NAME `shared name`, not merely some other check in + # the part. With only `some other check` listed, `demo` is a covered suite whose + # log carries a check the ledger has never seen, and the pre-existing refusal + # sets rc by itself: + # + # not in the ledger: demo part1 shared name (on major 18) + # + # Measured: with `rc = 1` deleted from the shared-key block, the old fixture's + # arm STILL PASSED and this one fails. That is the whole difference between an + # arm about this refusal and an arm about the gate returning 1. + # + # 3. AN UNHANDLED EXCEPTION, which also exits 1 -- an earlier draft referenced + # `covered_suites` before it was defined, which compiles and fails at runtime. + # Hence the `Traceback` arm below. + # + # The ledger covering the suite is NOT why it is shaped this way: the refusal applies + # to every suite, which the next arm asserts. + ledger = _w(tmp_path, "l.tsv", + "demo\tpart1\tsome other check\t18\tnever\t-\n" + "demo\tpart1\tshared name\t18\tnever\t-\n") + budget = _w(tmp_path, "b.txt", "suites_not_covered 1\n") + reg = _w(tmp_path, "reg", "demo\n") + dup = _w(tmp_path, "dup.log", + "RESULT\tdemo\tpart1\tshared name\tPASS\t18\t\n" + "RESULT\tdemo\tpart1\tshared name\tPASS\t18\t\n" + "checks run: 2\n") + out, rc = _run("gate", "--ledger", ledger, "--budget", budget, + "--registered", reg, dup) + expect.at_least(len(out), 20, "premise: the gate produced output to read") + expect.num(out.count("one ledger key covers 2 checks"), 1, + "a key covering two checks in one run is refused, and named") + expect.num(int("part1" in out), 1, "with the part, since the part is half the key") + expect.num(rc, 1, "and the gate fails rather than noting it") + expect.num(int("Traceback" in out), 0, + "premise: rc came from the refusal, not from a crash") + + +def test_a_shared_key_is_refused_in_a_suite_the_ledger_does_not_cover(expect, tmp_path): + """The refusal covers EVERY suite, deliberately unlike the new-check refusal. + + `test_the_gate_refuses_a_new_check_only_in_a_suite_it_covers` is restricted because + it cannot know which of an uncovered suite's checks are new. This one needs no + history at all: two records, one key, one log is decidable from the log alone. + + WHY THAT MATTERS rather than being a detail. The ledger covers four suites of 253, so + a refusal restricted the same way would close the class in four places and leave the + next collision to arrive in one of the other 249 and sit there until that suite is + seeded. This arm is what stops the restriction being copied in by habit. + + MEASURED BEFORE WIDENING IT, because a gate that reddens 250 unmeasured suites is a + gate somebody turns off. A full PG 18 matrix with this refusal armed for every suite: + 247 suites ran, RC=0, ALL VERSIONS PASSED, zero shared keys. Check names are static, + so one major's matrix measures this class completely rather than sampling it. + """ + ledger = _w(tmp_path, "l.tsv", "") # covers NOTHING + budget = _w(tmp_path, "b.txt", "suites_not_covered 1\n") + reg = _w(tmp_path, "reg", "demo\n") + dup = _w(tmp_path, "dup.log", + "RESULT\tdemo\tpart1\tshared name\tPASS\t18\t\n" + "RESULT\tdemo\tpart1\tshared name\tPASS\t18\t\n" + "checks run: 2\n") + out, rc = _run("gate", "--ledger", ledger, "--budget", budget, + "--registered", reg, dup) + expect.num(out.count("one ledger key covers 2 checks"), 1, + "a shared key in an uncovered suite is named") + expect.num(rc, 1, "and refused, with no row in the ledger for that suite") + expect.num(int("Traceback" in out), 0, + "premise: rc came from the refusal, not from a crash") + expect.num(out.count("not in the ledger:"), 0, + "premise: and not from the new-check refusal, which this suite escapes") + + +def test_the_same_check_in_two_runs_is_not_a_shared_key(tmp_path, expect): + """The control the refusal needs, and the distinction it must not lose. + + One check observed on two days is two records for one key and is the NORMAL case -- + it is how the ledger accumulates evidence at all. Only a repeat WITHIN one log is a + collision. Without this arm the refusal could be written as a count over all logs + together and would reject every multi-day merge. + """ + ledger = _w(tmp_path, "l.tsv", "demo\tpart1\tshared name\t18;19\tnever\t-\n") + budget = _w(tmp_path, "b.txt", "suites_not_covered 0\n") + reg = _w(tmp_path, "reg", "demo\n") + one = _w(tmp_path, "a.log", + "RESULT\tdemo\tpart1\tshared name\tPASS\t18\t\nchecks run: 1\n") + two = _w(tmp_path, "b.log", + "RESULT\tdemo\tpart1\tshared name\tPASS\t19\t\nchecks run: 1\n") + out, rc = _run("gate", "--ledger", ledger, "--budget", budget, + "--registered", reg, one, two) + expect.num(out.count("one ledger key covers"), 0, + "the same check in two logs is not a shared key") + expect.num(rc, 0, "so a two-run merge is not refused") + + +def test_a_clean_run_is_not_refused_for_a_shared_key(tmp_path, expect): + """The false-positive budget: two DIFFERENT names in one part must pass.""" + ledger = _w(tmp_path, "l.tsv", + "demo\tpart1\tfirst check\t18\tnever\t-\n" + "demo\tpart1\tsecond check\t18\tnever\t-\n") + budget = _w(tmp_path, "b.txt", "suites_not_covered 0\n") + reg = _w(tmp_path, "reg", "demo\n") + clean = _w(tmp_path, "c.log", GREEN) + out, rc = _run("gate", "--ledger", ledger, "--budget", budget, + "--registered", reg, clean) + expect.num(out.count("one ledger key covers"), 0, + "two distinct names in one part are not a shared key") + expect.num(rc, 0, "and a clean run passes the gate") + + def test_the_gate_refuses_a_census_that_contradicts_its_own_ledger(tmp_path, expect): """#952. The gate PRINTED the census and never compared it, so rc=0 on a lie.