From f80bb8ab67289f475f45123a9b4e1f175dded16a Mon Sep 17 00:00:00 2001 From: OffgridwithJD Date: Mon, 14 Sep 2026 15:51:24 +0000 Subject: [PATCH] test: a test file documented outside the numbering is invisible (#1024) #1024 reports that two PRs adding a test file both take the next section number, the merge keeps both, and nothing notices. MEASURED ON `52511a7`, THE FIRST HALF IS NO LONGER TRUE: `test_the_contents_list_is_numbered_in_order` landed after #1023 and catches exactly that. Planted a duplicate and it reddens, naming the inversion. So this does not drop the numbers, which was the issue's preferred option. Dropping them would remove the subject of a guard that works, and it costs more than the issue estimated -- 37 headings, 37 contents links, and THREE prose self-references the "one sed plus the anchors" figure misses. The other 26 `section N` references in the file name VACUITY_MODES.md's sections and are unaffected, which is why they have to be classified rather than counted. WHAT IS STILL OPEN IS A DIFFERENT SHAPE, and I shipped it. A test file whose section is written as an unnumbered `###` instead of a numbered `##` is invisible to every arm: not in the numbering so `1..N with no gap` never sees it not in the contents so the link arms never see it still NAMED so the coverage arm is satisfied `test_iceberg_fdw.py` went in that way in #1057 and sat undetected. Planted as a mutation it is caught by nothing before this change: 36 passed. It is also a cheaper failure than the collision. The collision needs two PRs in flight; this needs one person writing a heading at the wrong level. test files on disk 33 with a NUMBERED ## section 32 with an unnumbered ### section 1 <- test_iceberg_fdw.py, the defect So the rule was already true everywhere else, which is why it can be asserted rather than declared as a goal. `test_iceberg_fdw.py` becomes section 37 where it already sits in the body, and `test_hilbert_cluster.py` moves to 38 -- body order and numbering must agree, because the #1023 arm checks for inversions as well as gaps. Removal proof: CONTROL 37 passed M1 a section demoted to an unnumbered ### 1 failed <- the new arm, alone M2 a DUPLICATE section number 2 failed <- #1023's arm, still working TESTS.md restored to 70a78a69 M2 is in the proof deliberately: this change must not weaken the guard it found already working. Asserted in both directions, so a section cannot outlive the file it documents. guard leg 341 passed, 870 checks, 0 fail `guard_tests` 340 -> 341 by collection. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_012RSw4qMHS7ByE7PY8Ns4cs --- CHANGELOG.md | 13 ++++++++ test/pytest/TESTS.md | 8 +++-- test/pytest/expected_tests.txt | 5 +++- test/pytest/test_docs_cover_the_corpus.py | 36 +++++++++++++++++++++++ 4 files changed, 58 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 33e28fb0..1aca9e8d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,19 @@ true until the next version shipped. ### Added +- A test file documented as an unnumbered `###` section was invisible to every arm + that checks `TESTS.md` (#1024). + + Not in the numbering, so `1..N with no gap` never saw it. Not in the contents, so the + link arms never saw it. Still NAMED in the document, so the coverage arm was + satisfied. `test_iceberg_fdw.py` shipped that way in #1057 and sat undetected. + + #1024's own report -- two PRs each taking the next section number -- is no longer + open: `test_the_contents_list_is_numbered_in_order` landed after #1023 and catches a + duplicate and an inversion in one rule. Planted, it reddens. This change keeps that + guard and closes the remaining shape, which is cheaper to hit: the collision needs two + PRs in flight, a heading at the wrong level needs one person. + - A bash suite that unrolls a family as literals graded MISSING against the port that parametrises it (#1045 class 3). diff --git a/test/pytest/TESTS.md b/test/pytest/TESTS.md index d482ef3c..99cf489b 100644 --- a/test/pytest/TESTS.md +++ b/test/pytest/TESTS.md @@ -82,7 +82,8 @@ behaviour, the source of that number is named. - [34. test_docs_stripe_floor.py: the stripe floor is below a vector](#34-test_docs_stripe_floorpy-the-stripe-floor-is-below-a-vector) - [35. test_projection_privilege.py: the projection read helpers are a privilege boundary](#35-test_projection_privilegepy-the-projection-read-helpers-are-a-privilege-boundary) - [36. test_compare_to_bash.py: the parity tool reads the NAME](#36-test_compare_to_bashpy-the-parity-tool-reads-the-name) -- [37. test_hilbert_cluster.py: the Hilbert clustering SQL surface](#37-test_hilbert_clusterpy-the-hilbert-clustering-sql-surface) +- [37. test_iceberg_fdw.py: the Iceberg FDW's pruning surface](#37-test_iceberg_fdwpy-the-iceberg-fdws-pruning-surface) +- [38. test_hilbert_cluster.py: the Hilbert clustering SQL surface](#38-test_hilbert_clusterpy-the-hilbert-clustering-sql-surface) ## 1. How to read a test in here @@ -1066,6 +1067,7 @@ many times. | `test_an_anchor_that_strips_the_underscores_is_caught` | the exact broken link that shipped, with a control | | `test_every_in_document_link_in_this_directory_reaches_a_heading` | every contents-list link resolves, with a coverage premise | | `test_the_contents_list_is_numbered_in_order` | the contents list and the sections both count 1..N with no gap or inversion — the link arms above ask only whether a link RESOLVES, and a shuffled list resolves perfectly | +| `test_every_test_file_has_a_NUMBERED_section_of_its_own` | a section written as an unnumbered `###` is invisible to every other arm: not in the numbering, not in the contents, and the file is still NAMED so the coverage arm is satisfied — `test_iceberg_fdw.py` shipped that way in #1057 | | `test_a_shuffled_contents_list_is_caught_on_a_fixture` | **removal proof**: the `29, 31, 30` shape that shipped, with a clean control and an omitted entry named apart from an inversion | | `test_the_next_steps_list_is_anchored_to_the_inventory` | every section 5 entry names a mode id, so the entry can be checked at all | | `test_no_open_next_step_names_work_the_document_calls_done` | an un-struck entry whose id reached section 2 is stale work to do | @@ -3875,7 +3877,7 @@ the tool grades THIS tree. | `test_every_pair_in_the_tree_is_declared` | the declaration is asserted BOTH ways, so a new pair cannot be silently ungraded | | `test_the_ported_suites_in_this_tree_are_graded_one_for_one` | the standing arm: every pair in the tree, graded | -### `test_iceberg_fdw.py` -- the Iceberg FDW's pruning surface (#388, #432) +## 37. test_iceberg_fdw.py: the Iceberg FDW's pruning surface Ports `test/iceberg_fdw.sh`. 74 of its 76 check names, one for one; the two it cannot carry are `pgc_skip`'s refusal names, which are structural and declared in @@ -3911,7 +3913,7 @@ carry are `pgc_skip`'s refusal names, which are structural and declared in | `test_a_plan_with_no_pruning_marker_is_not_read_as_zero` | a plan that never mentions `Files Pruned` is not read as 0; needs no server | -## 37. test_hilbert_cluster.py: the Hilbert clustering SQL surface +## 38. test_hilbert_cluster.py: the Hilbert clustering SQL surface The port of `test/hilbert_cluster.sh` (#432, #889's SQL half). The bash suite pins the SQL surface of `pgcolumnar.cluster_hilbert` and `recluster_hilbert`, the recorded diff --git a/test/pytest/expected_tests.txt b/test/pytest/expected_tests.txt index f7a52680..b97c33b2 100644 --- a/test/pytest/expected_tests.txt +++ b/test/pytest/expected_tests.txt @@ -129,7 +129,10 @@ # right, the additive constraint run against three REAL pairs, the three refusals with # a control, and the name-argument restriction. # Re-derived by collection: `340 tests collected`. -guard_tests 340 +# 340 -> 341: a test file documented as an unnumbered `###` is invisible to every +# other arm -- outside the numbering, outside the contents, and still NAMED, so the +# coverage arm passes (#1024). Re-derived by collection: `341 tests collected`. +guard_tests 341 # The complement: tests that need the driver and a throwaway cluster. Until #1016 these ran # in no CI job at all -- a quarter of the corpus, green when somebody ran them by hand and diff --git a/test/pytest/test_docs_cover_the_corpus.py b/test/pytest/test_docs_cover_the_corpus.py index c27dbf12..f7617952 100644 --- a/test/pytest/test_docs_cover_the_corpus.py +++ b/test/pytest/test_docs_cover_the_corpus.py @@ -714,6 +714,42 @@ def _unresolved_links(text): return sorted({a for a in _LINK.findall(text) if a not in have}) +def test_every_test_file_has_a_NUMBERED_section_of_its_own(expect): + """A SECTION AT THE WRONG LEVEL IS INVISIBLE TO EVERY OTHER ARM (#1024). + + The arm above catches a section NUMBER taken twice, which is the collision #1024 + describes, and it does catch it -- planted, two arms redden. What nothing caught is + a test file whose section was written as an unnumbered `###` instead of a numbered + `##`: it is not in the numbering, so `1..N with no gap` never sees it; it is not in + the contents, so the link arms never see it; and the file IS named in the document, + so the coverage arm is satisfied. + + That is not hypothetical. `test_iceberg_fdw.py` shipped that way in #1057 and sat + undetected until this arm was written -- one person writing a heading at the wrong + level, where the collision needs two PRs in flight. + + MEASURED BEFORE WRITING IT: 33 test files, 32 with a numbered section, one without, + and that one was the defect. The rule was already true everywhere else, which is + why it can be asserted rather than declared as a goal. + """ + text = (HERE / "TESTS.md").read_text(encoding="utf-8") + files = sorted(p.name for p in HERE.glob("test_*.py")) + numbered = set(re.findall(r"^## \d+\. (test_\w+\.py)", text, re.M)) + + expect.at_least(len(files), 20, + "premise: the corpus was found, so the comparison is not vacuous") + missing = [f for f in files if f not in numbered] + expect.text(", ".join(missing) or "none", "none", + "every test file has a NUMBERED top-level section, so none is " + "documented outside the numbering the arms above check") + + # AND THE OTHER DIRECTION, so a section cannot outlive the file it documents -- + # the same both-ways shape the declaration arms use. + gone = [n for n in sorted(numbered) if n not in files] + expect.text(", ".join(gone) or "none", "none", + "and every numbered section names a file that exists") + + def test_the_anchor_rule_drops_punctuation_and_keeps_underscores(expect): """The derivation, on the heading the defect was found in.""" # Assembled, for the reason given in the arm below: a literal corpus file name