From 9553e4dc76bfd8c8fa963a633bdcb4706b6e84dc Mon Sep 17 00:00:00 2001 From: "Joshua (D) Drake" <136637981+ChronicallyJD@users.noreply.github.com> Date: Sat, 12 Sep 2026 17:13:43 -0600 Subject: [PATCH] test: refuse a duplicated entry in the vacuity inventory (#432) VACUITY_MODES.md section 3.4 carried the same six-id bullet twice, verbatim, two lines each. It is deleted, and three arms in test_docs_cover_the_corpus.py now refuse a duplicated entry. WHY NOTHING CAUGHT IT, MEASURED. Every count this document states is checked, and every one of those checks is blind to this by construction: _named_modes_in builds set(MODE_ID.findall(chunk)) per section, so each total is over distinct ids. Measured with the second copy present and then deleted: with the duplicate (28, 44, 72) without the duplicate (28, 44, 72) So the totals arm, the prose-totals arm and the sum arm were all green with a duplicated entry in the file. Deduping ids is right, and a total must not move because a line was pasted twice. The cost falls on the reader instead: one group of open modes reads as two. So the new arm is about entries, and the counting rule is unchanged. MY OWN FIRST SWEEP MISSED IT, which decided the unit the rule uses. An adjacent duplicate-LINE sweep over every document in the directory reported nothing: the duplicate is a two-line bullet, so line 1 of the first copy and line 1 of the second are not adjacent. Re-keyed on the bullet ENTRY, the same sweep found it, and found exactly one tree-wide. Both fixture arms therefore use a two-line bullet rather than a one-line one, and the rule carries its false-positive budget measured at the 40-character floor. AND THE SECTION THAT CHECKS FOR STALE DOCUMENTS CARRIED A STALE COUNT. TESTS.md section 6 said "the five fixture arms". Five was right at 3d6e1216 (2026-09-09) and counted the arms taking tmp_path; there are eight of those now and thirteen fixture arms in total. Nothing read the number. It is removed rather than corrected, which is what the paragraph above it already decided for the same reason: test_the_document_states_no_totals_for_a_merge_to_get_wrong exists to keep a totals line out. guard_tests 277 -> 280, re-derived by collection rather than by adding three. Verified: 280 passed / 689 checks with --pgc-expect-tests 280 armed; the new real-document arm red first at "lines 231 and 233" before the deletion; docs_style.sh PASSED (11 checks); selftest 350 5 PASS 0 FAIL. The long-sentence count is unchanged in both documents (TESTS.md 299, CHANGELOG 857). Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a REBASED onto 03c6c9c8, which also removed a stale number this branch was carrying. `cluster_tests` said 177 against main's 205: the branch never changed that line, so a three-way merge would have taken main's value and nothing would have gone wrong, but the file on the branch was stating a count that was three merges out of date. It now reads what main reads. guard_tests 280, re-derived by collection on the new base (280 collected) rather than carried. Re-verified: 280 passed / 698 checks with --pgc-expect-tests 280 armed; docs_style.sh PASSED (11 checks); the duplicate this PR removes is still present on main, 2 occurrences there against 1 here, so the fix still applies to the tree it is merging into. --- CHANGELOG.md | 44 +++++++++ test/pytest/TESTS.md | 19 +++- test/pytest/VACUITY_MODES.md | 2 - test/pytest/expected_tests.txt | 5 +- test/pytest/test_docs_cover_the_corpus.py | 108 ++++++++++++++++++++++ 5 files changed, 172 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index aa25d49d..9ef498bb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -72,6 +72,50 @@ true until the next version shipped. Verified on PG 18 in the container: `local_open_race_free.sh` PASSED, 11 checks; `native_recluster.sh` PASSED, 12 checks; both mutations red; main's arm green under the same mutation. +- The vacuity inventory named one entry twice, and no arm could fail on it (#432). + + `VACUITY_MODES.md` section 3.4 carried the same six-id bullet twice, verbatim, on two + lines each. It is deleted, and three arms in `test_docs_cover_the_corpus.py` now refuse + a duplicated entry. + + WHY NOTHING CAUGHT IT, measured rather than guessed. Every count this document states + is checked. Every one of those checks is blind to this by construction: + `_named_modes_in` builds `set(MODE_ID.findall(chunk))` per section, so each total is + over distinct ids. Measured with the second copy present and then deleted: + + with the duplicate (28, 44, 72) + without the duplicate (28, 44, 72) + + So `test_the_mode_inventory_states_its_own_totals_correctly`, the prose-totals arm and + the sum arm were all green with a duplicated entry in the file. Deduping ids is right. + A total must not move because a line was pasted twice. The cost falls on the reader + instead: one group of open modes reads as two. So the new arm is about entries, and the + counting rule is unchanged. + + MY OWN FIRST SWEEP MISSED IT, which decided the unit the rule uses. An adjacent + duplicate-LINE sweep over every document in the directory reported nothing. The + duplicate is a two-line bullet, so line 1 of the first copy and line 1 of the second + are not adjacent. Re-keyed on the bullet ENTRY, the same sweep found it, and found + exactly one tree-wide. Four sweeps here have now failed by keying on how something is + written rather than on what it contains. So the unit is named in the code, and both + fixture arms use a two-line bullet rather than a one-line one. + + The rule carries its false-positive budget as an arm. The inventory legitimately repeats + short bullets, so entries under a 40-character floor are not compared. The budget is + measured at the boundary: the same bullet passes below the floor and is refused above + it. + +- The section that checks for stale documents carried a stale count (#432). + + `TESTS.md` section 6 said "the five fixture arms". Five was correct at `3d6e1216` + (2026-09-09) and counted the arms taking `tmp_path`; there are eight of those now and + thirteen fixture arms in total. Nothing read the number, so it went stale in the + document whose whole subject is documents going stale. + + It is removed rather than corrected. The paragraph above it already decided that for the + same reason: `test_the_document_states_no_totals_for_a_merge_to_get_wrong` exists to keep + a totals line OUT. A count in prose that no arm reads is a claim waiting to go wrong. The + arms themselves are listed in the table above, where a reader can count them. - `native_ownership.sh` has a pytest twin, and it asserts the SQLSTATE (#432). diff --git a/test/pytest/TESTS.md b/test/pytest/TESTS.md index 279e9317..a9267d7d 100644 --- a/test/pytest/TESTS.md +++ b/test/pytest/TESTS.md @@ -1026,6 +1026,9 @@ many times. | `test_the_inventory_accounts_for_every_mode_the_run_found` | the admitted gap row is the run's total minus what is written down | | `test_the_prose_totals_match_the_counted_modes` | every sentence stating what the layer refuses today carries the counted number, not just the table | | `test_the_two_halves_of_the_refused_sentence_sum_to_the_named_total` | TESTS.md states the split twice in one sentence, and BOTH halves are checked against the inventory's own count — the gated half alone let 26 + 47 = 73 past a named total of 72 | +| `test_the_inventory_names_no_entry_twice` | no bullet entry in VACUITY_MODES.md is written twice — the count guards dedupe ids, so a duplicated entry moves no total and nothing could fail on it | +| `test_a_duplicated_entry_is_caught_on_a_fixture` | **removal proof**: the shape that got through, which is a TWO-LINE bullet, on a fixture with its clean control | +| `test_a_short_repeated_bullet_is_not_flagged` | the rule's false-positive budget, measured at the length floor: below it a repeated bullet is ordinary, above it is a duplicate | | `test_an_undocumented_file_is_caught_with_the_tests_inside_it` | how 29 tests went missing at once | | `test_a_document_with_no_totals_line_states_none` | absent totals report `None`, which must not read as "they match" | | `test_a_stated_total_that_disagrees_with_disk_is_visible` | the count arm's own red | @@ -1060,9 +1063,19 @@ mode id, and that no un-struck entry names an id section 2 already claims. With every entry now struck, the second arm has nothing to refuse on the real document. That is what the fixture arm is for. -The five fixture arms exist because everything above them passes on a healthy tree, -which is exactly what a guard that does nothing also does. They run the identical -functions over a corpus built to be wrong. +The fixture arms exist because everything above them passes on a healthy tree, which +is exactly what a guard that does nothing also does. They run the identical functions +over a corpus built to be wrong. + +**That sentence said "the five fixture arms" and nothing counted them.** Five was right +when it was written, at `3d6e1216` on 2026-09-09. It counted the arms taking `tmp_path`. +There are eight of those now, and thirteen fixture arms in total. So the number had gone +stale in the document whose subject is stale documents. + +It is removed rather than corrected. One paragraph up, +`test_the_document_states_no_totals_for_a_merge_to_get_wrong` already decided that for the +same reason. A count in prose that no arm reads is a claim waiting to go wrong. The arms +are listed in the table above, where a reader can count them. ### The twin, and which half has teeth diff --git a/test/pytest/VACUITY_MODES.md b/test/pytest/VACUITY_MODES.md index 74b6acf1..a02d3190 100644 --- a/test/pytest/VACUITY_MODES.md +++ b/test/pytest/VACUITY_MODES.md @@ -230,8 +230,6 @@ binds the exception and the body pins its SQLSTATE. See section 2. @jdatcmd on review. See TESTS.md section 20. - `same-broken-helper-both-sides`, `truthy-error-string`, `assert-not-unset-error`, `zero-on-both-arms`, `tuple-assert-always-true`, `approx-of-nothing` -- `same-broken-helper-both-sides`, `truthy-error-string`, `assert-not-unset-error`, - `zero-on-both-arms`, `tuple-assert-always-true`, `approx-of-nothing` ### 3.5 The fixture built the wrong situation diff --git a/test/pytest/expected_tests.txt b/test/pytest/expected_tests.txt index fda3537e..30e7a20e 100644 --- a/test/pytest/expected_tests.txt +++ b/test/pytest/expected_tests.txt @@ -36,7 +36,10 @@ # test_mutation_ledger.py. That is the mechanism doing its job rather than a nuisance: had # it not moved, the job would have failed with "collected 274 test(s) but expected 272" and # named the drift instead of running a different suite than the one declared. -guard_tests 277 +# 277 -> 280: three arms in test_docs_cover_the_corpus.py about duplicated entries in +# VACUITY_MODES.md. Re-derived by collection rather than by adding three, per the recipe +# above: `280 tests collected`. +guard_tests 280 # The complement: tests that need the driver and a throwaway cluster. Until #1016 these ran # in no CI job at all -- a quarter of the corpus, green when somebody ran them by hand and diff --git a/test/pytest/test_docs_cover_the_corpus.py b/test/pytest/test_docs_cover_the_corpus.py index 1d8590b4..5e9fec84 100644 --- a/test/pytest/test_docs_cover_the_corpus.py +++ b/test/pytest/test_docs_cover_the_corpus.py @@ -480,6 +480,114 @@ def test_the_two_halves_of_the_refused_sentence_sum_to_the_named_total(expect): "and the refused half is the count of ids section 2 claims") +# A bullet ENTRY is the unit, not a line. The duplicate that motivated this is a +# two-line bullet, and a line-keyed sweep cannot see it: line 1 of the first copy +# and line 1 of the second are not adjacent. Four sweeps in this tree have now +# failed by keying on the wrong unit, so the unit is named here and fixtured below. +def _bullet_entries(text, floor=40): + """-> [(1-based start line, the entry joined)] for every bullet in TEXT. + + A continuation is an indented non-blank line under a bullet, which is how every + multi-line entry in the inventory is written. Entries shorter than FLOOR + characters are dropped: the inventory legitimately repeats short bullets such as + a bare id, and a rule that flagged those would be switched off. + """ + out, cur, start = [], None, 0 + for i, line in enumerate(text.splitlines()): + if re.match(r"^\s*[-*] ", line): + if cur is not None: + out.append((start, cur)) + cur, start = [line.strip()], i + 1 + elif cur is not None and line.strip() and line[:1] in " \t": + cur.append(line.strip()) + elif cur is not None: + out.append((start, cur)) + cur = None + if cur is not None: + out.append((start, cur)) + return [(s, " ".join(b)) for s, b in out if len(" ".join(b)) >= floor] + + +def _duplicated_entries(text): + """-> [(first line, repeat line, the entry)] for every entry written twice.""" + seen, dupes = {}, [] + for start, entry in _bullet_entries(text): + if entry in seen: + dupes.append((seen[entry], start, entry)) + else: + seen[entry] = start + return dupes + + +def test_the_inventory_names_no_entry_twice(expect): + """A duplicated entry double-states the inventory, and the count guard is blind + to it BY CONSTRUCTION rather than by accident. + + `_named_modes_in` builds `set(MODE_ID.findall(chunk))` per section, so every + total it states is over distinct ids. Measured on the document that motivated + this, with the second copy of a six-id bullet present and then deleted: + + with the duplicate (28, 44, 72) + without the duplicate (28, 44, 72) + + So no existing arm here can fail on it, and none did: the duplicate sat in + section 3.4 while `test_the_mode_inventory_states_its_own_totals_correctly`, + `test_the_prose_totals_match_the_counted_modes` and the sum arm above were all + green. The cost is to the reader rather than to the totals -- a six-id bullet + written twice reads as two distinct groups of open modes -- which is why this is + an arm over entries and not a correction to the counting rule. Deduping ids is + right; the totals must not move because someone pasted a line twice. + + The counterpart to `test_no_test_name_is_defined_twice_in_the_corpus`, for the + document rather than the corpus. + """ + doc = MODES_DOC.read_text() + entries = _bullet_entries(doc) + expect.at_least(len(entries), 20, "premise: the rule finds entries to compare") + dupes = _duplicated_entries(doc) + expect.text( + "; ".join(f"lines {a} and {b}" for a, b, _ in dupes) or "none", "none", + "the inventory names no entry twice") + + +def test_a_duplicated_entry_is_caught_on_a_fixture(expect): + """Prove the rule fires, and fires on the shape that got through. + + Two lines, not one, because a one-line fixture would pass against a sweep keyed + on adjacent identical LINES -- the sweep that missed the real duplicate. + """ + entry = ("- `a-mode-named-once`, `a-second-mode-here`,\n" + " `a-third-mode-on-the-continuation-line`\n") + clean = "## 3.4 A section\n\n" + entry + "\n- `something-else-entirely-here`, `and-another-mode-id`\n" + expect.num(len(_duplicated_entries(clean)), 0, + "premise: the clean fixture is not flagged") + spliced = clean.replace(entry, entry + "\n" + entry, 1) + dupes = _duplicated_entries(spliced) + expect.num(len(dupes), 1, "the duplicated two-line entry is caught") + expect.num(dupes[0][0], 3, "and the FIRST copy's line number is reported") + expect.at_least(dupes[0][1], 4, "with the repeat's line after it") + + +def test_a_short_repeated_bullet_is_not_flagged(expect): + """The rule's false-positive budget, stated rather than assumed. + + The inventory repeats short bullets -- a bare id under two headings is ordinary + -- and a guard that reddened on those would be removed, taking the real rule + with it. Measured at the floor: the same bullet below it passes, above it fails. + """ + short = "## 3.4 A section\n\n- `a-b-c`\n\n- `a-b-c`\n" + expect.num(len(_duplicated_entries(short)), 0, + "a repeated bullet under the length floor is not a duplicate") + long_id = "- `" + "a-b-c-" * 9 + "d`\n" + doubled = "## 3.4 A section\n\n" + long_id + "\n" + long_id + expect.at_least(len(_bullet_entries(doubled)), 2, + "premise: the long fixture clears the floor") + expect.num(len(_duplicated_entries(doubled)), 1, + "and the same bullet above the floor IS a duplicate") + + + + # --------------------------------------------------------------------------- # The counting rule's own edges, and the row reader's. #