test: the grader reads every check helper lib.sh defines (#432, #1040)

[jdatcmd]

The other half of #1040. #1041 fixed the two hilbert_locality names this was hiding; this makes the grader able to see the class at all.

The defect

The bash side of the parity tool matched check(?:_num|_ratio|_text|_timing)?, so three of the eight helpers lib.sh defines were invisible:

check  check_num  check_text  check_ratio  check_timing        READ
check_unrunnable  check_skip  check_ratio_needs_quiet_machine  INVISIBLE

A property asserted through one of the three was never reported MISSING and could not move rc. So a pair could grade one-for-one on the strength of the grader's blind spot — which hilbert_locality was.

It never drifted out of date. 0cbf574 is the commit that introduced that pattern, and check_unrunnable already had 21 call sites that day. Same shape as #1036 and #1038 on the python side — a rule true of most of a class taken for a property of the class — sitting on the other side of the same tool, unexamined because fixing one end thoroughly did not prompt anyone to look at the other.

Three arms, two of which state what the tool does NOT cover

arm	holds
test_the_extractor_reads_every_check_helper_lib_sh_defines	the list, re-derived from lib.sh's DEFINITIONS, failing with the helper named
test_a_longer_helper_name_is_not_shadowed_by_a_shorter_one	the pattern SHAPE — see the correction below
test_the_suite_local_helpers_are_known_and_excluded	the four suite-local helpers, and that none of their suites is graded today

The helper list is hand-written so the tool stays standalone (ast, re, sys), and it is pinned exactly the way _NAME_ARG is.

A claim of mine that was overstated, corrected after measuring

The entries are longest-first, and the code said a shorter prefix would otherwise shadow a longer helper. That is false in Python. re backtracks across alternatives, so a pure reorder reads both names identically — measured, and the arm stays green under it.

What the OLD pattern could not do was read check_ratio_needs_quiet_machine at all: it matches check_ratio, wants whitespace, finds _needs, backtracks to the empty option, wants whitespace after check, and fails.

pre-#1040 optional-suffix form   ['short']
this form, either order          ['short', 'long']

So the arm pins the shape and the ordering is readability. Corrected in the code comment, the docstring and TESTS.md rather than left reading as load-bearing.

Population

check or check_<something>, not check[a-z_]*. The loose form also matches checks_in in decode_interrupts.sh, a counting utility that returns a number and records nothing. It was in the suite-local list until the arm printed it and I read the definition.

Removal proof

Each mutation asserted to apply by md5, tree restored to baseline after each.

mutation	red
drop check_unrunnable from the list	the drift guard, naming it
drop check_skip from the list	the drift guard, naming it
add a duplicate entry	the drift guard, via its size reconciliation
pure reorder, longest no longer first	nothing — which is the finding above

What this changes for a suite that uses those helpers CORRECTLY

Found by @OffgridwithJD while porting hilbert_cluster, because their port grades against this tool; re-measured here before quoting it.

hilbert_locality was the broken case — two unrunnable names with no runnable twin, which #1041 fixed. hilbert_cluster is the opposite, and it is the first real exercise of this widening:

bash names the grader reads today (5 helpers)	124
check_unrunnable sites, invisible until now	+9
check_skip sites	0
bash names after this PR (8 helpers)	133

All nine of those names are twinned by a runnable check in the same file — confirmed two ways, by the merged sweep tool reporting no MISMATCH for that suite, and by matching each unrunnable name against the runnable names directly: 9 of 9 twinned, 0 orphans.

So the widening adds nine names to that suite's bash side and should produce zero MISSING against a complete port. Nothing reddens here — hilbert_cluster has no pytest twin yet — but a port of it now has a definition of done nine names larger, and that number moves the moment this merges. @OffgridwithJD is targeting 133 rather than 124 for exactly that reason.

The crossing is declared, and why the scan's scope is now written down

The drift guard reads the shell harness's helper library by a hard-coded path, which the harness-independence inventory reddens: a python file naming a shell path. Declared rather than redesigned. The inventory is asserted in both directions, so a declaration that stops being true reddens too, and each entry must carry a mechanism a reader can check. The justification is that the alternative is a hand-maintained copy of the helper list inside pytest — the exact value this issue is about.

While reviewing that, @OffgridwithJD argued that .github/scripts/ is exempt by design, since two tools there read test/*.sh and appear in no inventory — and therefore that moving compare_to_bash.py there would delete the crossing rather than declare it.

The inventory's scan is test/pytest/*.py, so that population is a LOCATION, and absent-from-the-report and outside-the-scan produce identical evidence. Two readings were live and the artefact did not say which.

Both halves of that are worth stating precisely:

• The inference was not available from the evidence, and the rule was genuinely underivable — which is why the ruling now sits in the file instead of in a glob.
• It was also avoidable: the glob was one command away, and it was checked after the argument rather than before it. @OffgridwithJD asked that this half not be left out, and they are right that omitting it makes the episode read better than it was.

jd ruled: the scope is deliberate. The inventory is the pytest corpus's self-guard; .github/scripts/ is CI tooling belonging to neither harness, so a tool there reading both sides is what it is for. Those two tools are out of scope rather than exempt. Recorded in the declaration's comment, along with the consequence a reader acts on: a move to .github/scripts/ is defensible on design grounds, never because it makes the arm stop reporting the file.

Verification

Against main 0d17a87, re-derived there rather than carried from the head it was written on:

all seven pairs           rc=0 missing=0
guard job                 316 collected | 784 checks | 784 pass + 0 fail + 0 unrun
docs_style.sh             14 checks PASSED
expected_tests.txt        313 -> 316, by collection

Which half of the corpus that is, and which I did not run. Everything above is the
GUARD half — the database-free files, run exactly as ci.yml's
pytest (harness guards, no database) job runs them, in a venv pinned from
requirements-test.txt with the driver absent. I did not run the cluster half:
there is no PostgreSQL on this host, so pytest (cluster tests, with the driver) is
CI's evidence and not mine. Stated because a review is more useful when it says which
leg belongs to whom, and because this PR's first CI run failed exactly there.

No .sh changed. pgc_vacuity.py untouched — an earlier draft added a keyword-only name= to cannot_run so the port could mirror bash's four unrunnable records; @OffgridwithJD showed that replicates a bash inconsistency instead of fixing it, and #1041 fixed it at the source. That work is dropped, not parked in this PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK

[linuxhikerpm]

The defect is real and the same shape as #1036/#1038 on the other end of the same tool: a pattern that described most of a class, treated as a property of the class. I checked origin/main's lib.sh — the eight helpers in _BASH_HELPERS are exactly the definitions, including check_skip.

The three new arms do the useful work. The drift guard re-derives from definitions, not from call sites, so a new check_whatever() in lib.sh goes red before a suite using it is silently ungraded. The shadowing arm pins the pattern shape (the old optional-suffix form could not read check_ratio_needs_quiet_machine at all) and the comment now says the longest-first order is not load-bearing, which matches the measurement. The suite-local exclusion names the four helpers and that none of their suites is graded — that is the day-one statement a port of parallel_copy will need.

hilbert_cluster growing from 124 names to 133, with the nine check_unrunnable names already twinned in the same file, is the first real exercise of the widening. Nothing here should redden an existing pair.

CI is green on this head, including the cluster job. I did not merge.

[OffgridwithJD]

Reviewed at ea5a6cb532c7e18dba90d3efdd2f324ca8a23de0, and CI read from the head sha's
check-runs in the same call as the head: 14/14 success, 0 other, MERGEABLE/CLEAN, 0 behind
main. Every claim below was re-run here rather than read; each mutation asserted it applied
and its restore was md5-asserted.

The four things you asked me to check rather than take

1. The null result reproduces. A pure reorder of _BASH_HELPERS to shortest-first --
the arrangement your original claim said would break it -- reddens nothing:

control                       48 checks, 48 pass, 17 passed
shortest-first reorder        48 checks, 48 pass, 17 passed
                              md5 moved, restore asserted

So your correction stands and the ordering is not load-bearing. Python's re alternation
backtracks across branches. A mutation that reddens nothing is a finding about the claim,
not a gap in the mutation -- that is your sentence and it is the best thing in this PR.

2. The drift guard fires in both directions and names the helper.

drop "check_skip" from the tuple
  FAILED test_the_extractor_reads_every_check_helper_lib_sh_defines
  every check helper lib.sh defines is one the extractor reads: got 'check_skip' want 'none'

remove the test_compare_to_bash.py declaration from SHELL_REFERENCES
  FAILED test_the_harness_independence_inventory_is_exactly_what_the_corpus_does
  got [... 'test_compare_to_bash.py' ...]  want [... without it ...]

My first attempt at the second was invalid and I am saying so rather than quietly
re-running it: my regex removed part of the multi-line entry and produced a SyntaxError,
so the arm never ran and "it failed" would have been a claim about my mutation. The valid
version asserts the mutated file still parses before drawing any conclusion from it.

3. The population is right and the checks_in trap is real. The strict class
check(_[a-z_]+)? finds exactly your four suite-local helpers -- check_structure,
check_reconstruct, check_split_happened in parallel_copy.sh and check_float in
parquet_export_stats.sh -- plus the 13 suites that define their own check(). The loose
check[a-z_]* adds exactly one file, decode_interrupts.sh, for checks_in(), whose body
runs an awk that counts COLUMNAR_DECODE_INTERRUPT( occurrences and prints a number. It
records nothing. And none of the three suites has a pytest twin, so none is graded today.

4. The entry says a true thing, and it is not the phrase you gave me in chat. You told
me the justification was "one implementation rather than two", which is pgc_cluster.py's
and would have been wrong here -- no implementation is shared, a list is re-derived. The
entry in the file says the list "cannot become a hand-maintained copy that rots apart from
it (#1040, where five of eight helpers had been unreadable since the pattern shipped)". That
is accurate, it names a mechanism a reader can check, and it is a better justification than
the one I was going to hold you to. The artefact got the care; I only found that by opening
it.

One thing recorded, not asked for

The hilbert_cluster measurement in the body is mine and reproduces: 124 bash checks as the
tool ships, 133 with this change, the gap being that suite's nine check_unrunnable sites
and zero check_skip. It is worth saying why that case matters beyond being a bigger number:
hilbert_locality was the BROKEN case, where the names had no twin and #1041 fixed them;
hilbert_cluster is the WORKING case, where nine correct names were simply invisible. A
change exercised only against the broken case shows that it stops hiding a defect and never
shows that it adds names correctly. My port of that suite grades 0 MISSING against both the
shipping extractor and this one.

Approving

The cluster leg -- the one you cannot run -- is green here on this sha at 219 passed / 622
checks / RC=0, which agrees with CI's. Merging stays yours.