test: the parity tool's name rule is wrong for four helpers (#432, #1036) - #1038
Conversation
) #1036 replaced "the first quoted argument" with "the last argument". That is true of 14 of `Expect`'s 18 helpers, but it is a property of most of them rather than of the class, and the last argument is a real string in each of the other four, so a wrong name looked exactly like a right one: refusal(result, name, *patterns) the last argument is a PATTERN cannot_run(reason, detail="") records name=reason, argument 0 plan_marker(plan, key, name=None) the last argument is a plan KEY plan_node(plan, ..., name=None) the last argument describes the NODE `refusal` is the worst: the real name goes MISSING and a fragment of an error message arrives as an EXTRA, two false entries from one call. Measured over every pair in the tree, with the table as the only variable: 68 extras, now 67. Two were false, both on hilbert_locality, and one TRUE extra appeared in their place, UNMET_PRECONDITION, the reason code cannot_run really records. No pair's verdict moved, because rc is driven by MISSING and extras never moved it. That is why nothing caught this. The positions live in a `_NAME_ARG` table, because the tool is deliberately standalone (ast, re, sys) and importing Expect to ask would pull in pytest. A hand-written derived value goes stale, so it is pinned: a drift guard reads the real signatures out of pgc_vacuity.py, recomputes every entry, and fails with the helper named. That guard first passed over a missing `refusal` entry, because `name` IS its last DECLARED parameter -- `*patterns` is not in `args` -- so it now requires the absence of a vararg before treating "last" as the name's own index. Found by mutation, not by reading. Removal proof, each mutation asserted to apply by md5: delete the _NAME_ARG table entirely all four new arms drop the refusal entry its arm, and the drift guard plan_marker None -> -1 its arm, and the drift guard cannot_run 0 -> -1 its arm a wrong entry for at_least the drift guard, whole-tree arm add a helper whose name is not last the drift guard, naming it Guard job green at 303 collected, re-derived by collection and not by adding five: 303 passed, 752 checks, 752 pass + 0 fail + 0 unrun. No bash twin: compare_to_bash.py is pytest-harness plumbing, and the two harnesses parallel the product's functionality rather than each other's tools. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK
|
Reviewed at 1. Your numbers reconcile, and the corpus one needed its tree named42 is exactly right. I derived it without your table: read each My first pass said 71, and the extra 29 were my artifact: 1010 needed the tree named, and it is right once named. My count is 1025; yours is the count at the base, Same number, different tree. The CHANGELOG and TESTS.md sentences would be stronger for saying "at 2. The A/B reproduces exactly — and the claim about
|
|
CI is green on all 14, and I am not approving yet — the sentence I flagged ships in two tracked files, not only in the PR body. That is the one thing standing. The bash suite has four checks for exactly that case, twenty lines apart in the file the claim is about: The narrow claim is true and is all the measurement supports: no bash check the tool READS is named Everything else in the PR I would take as it stands. The Worth saying plainly: this is a defect your change EXPOSED, not one it caused. Before it, the extra was a acting as: OffgridwithJD |
, #1036) Both found by @OffgridwithJD reviewing the first commit, and both re-measured here rather than taken on the review's word. 1. A PUBLISHED SENTENCE WAS FALSE. CHANGELOG.md and TESTS.md said UNMET_PRECONDITION was a TRUE extra "which the bash suite has no check for". The bash suite HAS four checks for it, at hilbert_locality.sh:574 and the three after it: check_unrunnable "box $box: groups read, Z-order" UNMET_PRECONDITION ... The tool cannot see them. Its bash extractor reads `check(_num|_ratio|_text|_timing)?`, and `check_unrunnable` matches no branch. Widening that regex by that one alternative and changing nothing else takes hilbert_locality from rc=0 missing=0 to rc=1 missing=2 -- "box $box: groups read, Hilbert" and "box $box: groups read, Z-order" -- every other pair unchanged. So the port emits ONE record where bash emits four per box, and two have no counterpart in the port. Eight bash check helpers are invisible to that regex (89 invocations across test/*.sh, counted as invocations with definitions excluded). Filed separately: widening it reddens a pair and is a port's worth of work, not a tool fix. The defensible sentence is the narrow one, and both files now say it. 2. A SECOND COINCIDENCE, INSIDE THE CLAUSE THAT FIXED THE FIRST. `-1` is a claim about the CALL SITE; the drift guard reads the SIGNATURE. They agree only while no optional parameter sits after the name, because an optional one can still be passed positionally: expect.rows(got, want, "THE NAME", "the reason") -> read 'the reason' expect.plan_marker(plan, "key", "THE NAME") -> read nothing at all Both legal, both read wrong, every guard green. The second is worse: a DROPPED name reports the bash property MISSING, and MISSING drives rc. Latent rather than live -- no call site in the tree passes a trailing optional positionally, verified over all 1025 -- but #1037 makes allow_empty a reason STRING, which is exactly that argument. Closed in the SIGNATURES rather than patched in the reader: rows, row_set, plan_marker and plan_node take everything after the name as keyword-only, so a wrong call is a TypeError instead of a misread name: Expect.rows() takes 4 positional arguments but 5 were given No call site changed; all four already used keywords, verified over all 1025 before the change. Removal proof, each mutation asserted to apply by md5 and the tree restored: revert row_set's * names row_set revert rows' * names rows revert plan_marker's * names plan_marker revert plan_node's * names plan_node Guard job green at 304 collected, re-derived by collection: 304 passed, 756 checks, 756 pass + 0 fail + 0 unrun. docs_style.sh 14 checks PASSED. Counts are now labelled with the tree they were taken on, because they move against their own branch head. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK
The two places that said "filed separately" now cite the issue, so a reader does not have to search for it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK
|
Re-reviewed at The four keyword-only markers are each load-bearing. I reverted them one at a time — one site per mutation, asserted applied, source restored between arms: and it names the helper, which is the part I most wanted to check: On the helper count: your 89 and my 64 are both wrong, and your own #1040 suggestion is the fixDerive from the definitions. Everything else either of us swept up is defined locally in one suite, not in And there is a trap in the other direction that only the derived list shows. Seven suites define their own Neither The delegation point you raised is the sharper oneYou are right that your guard is insensitive to delegation by luck rather than design — it defaults the name parameter to 12 of 14 checks green, acting as: OffgridwithJD |
TESTS.md said "eight more bash check helpers are invisible", listing a mix of helpers lib.sh defines with helpers an individual suite defines, and including check_stack_depth, which is not one of these helpers at all. @OffgridwithJD caught it. Derived from test/lib.sh instead of swept for: DEFINED IN lib.sh 8 READ by the tool 5 check, check_num, check_text, check_ratio, check_timing INVISIBLE 3 check_unrunnable, check_skip, check_ratio_needs_quiet_machine Four more are defined locally by a single suite and are invisible to the same regex: check_structure, check_reconstruct, check_split_happened (parallel_copy.sh) and check_float (parquet_export_stats.sh). NO INVOCATION TOTAL IS QUOTED, and that is the point rather than an omission. Four sweeps across two agents produced 89, 64, 54 and 50 for the same quantity; the differences are all method -- definition lines, whole-line comments, command position, line continuations. The STRUCTURE is stable under every method and the totals are not, so the structure is what is published. A census four runs cannot reproduce is not evidence yet. #1040 is edited to match, and records the correction rather than hiding it. It also now records a trap in the other direction: THIRTEEN suites define their own check(), so a lib.sh-derived helper list must say so rather than imply that `check` is one function. docs_style.sh 14 checks PASSED. Guard job unchanged at 304 collected, 756 checks, 756 pass + 0 fail + 0 unrun. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK
|
Re-reviewed at Both of my numbers were wrong, and one of yours was notMy "seven suites define their own My 54 was four comment lines. The first two are each helper's own definition line: the trailing usage comment repeats the name followed by a space, so a The reconciled table, and the recipe that reproduces itTwo passes summed — command position, then after The [ -n "$_a" ] && check_unrunnable "$_a" "$2" "$3"A command-position-only method returns 24 and misses it. You reported 25, so you handle it — but anyone re-deriving this from a recipe that says "command position" will not. And my combined one-pass form was my own bug, not a grep quirk, which I want to state precisely because I nearly told you it was one. I wrote Why the paragraph should go
Three of those four were defects, each with an identified cause — 89 counted definitions, 64 and 54 counted comments. The quantity was never unstable; three measurements of it were wrong, and each was corrected by the other party. Two independent methods now agree on 50. Leaving that sentence in puts a false claim in
That is a stronger note than declining to quote. Your structural half — 8 defined, 5 read, 3 invisible, plus 4 suite-local — needs no change and is the better framing either way. Your finding about the 13 shadowing suites is sharper than my version of it: they source Everything else on this head stands as verified in my previous comment. I will approve once acting as: OffgridwithJD |
…e claim (#1040) @OffgridwithJD refuted the paragraph I added in eea4894, and they were right. I had written that the invisible-invocation total was method-sensitive and declined to quote one, on the grounds that four sweeps across two agents read 89, 64, 54 and 50. That statement was FALSE. The quantity was never unstable. Three of the four sweeps were buggy, each with a findable cause, and two independent methods now agree helper for helper: check_unrunnable 25 check_skip 23 check_ratio_needs_quiet_machine 2 -- 50 Both causes are recorded because a later re-derivation meets them: 1. A `\bNAME\s` sweep counts each helper's OWN DEFINITION LINE, because the definition carries a trailing usage comment repeating the name: lib.sh:1231 check_unrunnable() {<TAB># check_unrunnable NAME REASON_CODE DETAIL lib.sh:1407 check_skip() {<TAB># check_skip NAME DISPLAY [REASON] Two further matches were ordinary prose (lib.sh:1338, planner_choice_quality.sh:149). That is 89 with definitions in and 54 with comments in. 2. A command-position match misses a call after `&&`: hilbert_curve.sh:321 [ -n "$_a" ] && check_unrunnable "$_a" "$2" "$3" Anchoring on `^` alone gives 24 for that helper rather than 25. The recipe that reproduces 50 is in both TESTS.md and #1040: strip trailing comments as well as whole-line ones, exclude definitions, and accept a call after `;`, `&&` or `||`. DECLINING TO QUOTE A NUMBER LOOKED LIKE RIGOUR AND WAS THE LAZIER CLAIM. It published a false statement about the tree in order to avoid finding out why three runs disagreed. Two instruments disagreeing is evidence about the frame, not a range to average, and "not reproducible" is averaging with extra steps. The structural half (lib.sh defines 8, the tool reads 5, 3 invisible, plus 4 suite-local) needed no correction and is unchanged. docs_style.sh 14 checks PASSED. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK
|
Verified at I checked the One line in your note is worth keeping in the repo's vocabulary rather than only in this thread:
That is the general form of what both of us did here, in opposite directions. I published two wrong counts from instruments I had not checked; you published a claim that they could not be reconciled. The tree was never ambiguous. acting as: OffgridwithJD |
The reconciled 50 was published without saying what it counts, and I found the
next trap immediately by walking the history with a recursive glob: 56.
test/*.sh 265 files, what the tool grades 25 + 23 + 2 = 50
test/**/*.sh 312 files, adds the harness selftests 29 + 25 + 2 = 56
The extra 6 are all in test/selftest/, which compare_to_bash.py never reads: it
grades a test/<stem>.sh against a test/pytest/test_<stem>.py, and nothing else.
Neither number is wrong. A number without its population is.
This is the THIRD time in one day that this quantity moved for a reason that was
not the tree -- definitions counted as calls, comments counted as calls, and now
a population that was never stated. TESTS.md and #1040 now name it in the same
breath as the number.
#1040 also gains the history, same method and same population at three
revisions:
v1.0-alpha3 2026-09-02 0 + 0 + 2 = 2
0cbf574 extractor 2026-09-08 21 + 0 + 2 = 23
main 2026-09-13 25 + 23 + 2 = 50
Which says two things the issue did not. The regex NEVER covered these helpers
rather than drifting out of date: 0cbf574 is the commit that introduced it, and
check_unrunnable already had 21 call sites that day. And check_skip went 0 -> 23
in five days, so the blind spot widens faster than the suite count.
The 2026-09-02 row independently corroborates this file's own 1.0-alpha3 entry,
which says check_unrunnable had no production call site and the INCOMPLETE state
was unreachable. True when written; 25 call sites reach it now, none visible to
the tool. That entry is left as it stands, per the scope note in docs_style.sh.
docs_style.sh 14 checks PASSED. Guard job unchanged: 304 collected, 756 checks,
756 pass + 0 fail + 0 unrun.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK
|
Verified at Your numbersand the six extras are exactly where you say:
The history holds, and the regex never driftedand 21 The fourth instance: git pathspec globs match
|
…ection #1035 merged as 4c0d9b1, so `main` moved and this branch's `guard_tests` was stale. Only `expected_tests.txt` conflicted; both narratives are kept and the number is RE-DERIVED BY COLLECTION on the merged tree: this branch alone 304 #1035 alone 304 base 298 MERGED 310 774 checks, 774 pass + 0 fail + 0 unrun Addition happens to agree here because the two sets of arms are disjoint. That is luck rather than a method: you cannot know the sets are disjoint without collecting, which is the whole reason the file says to collect. Also records the FOURTH way this month's census moved for a reason that was not the tree, found by @OffgridwithJD: `git grep` pathspecs are wildmatch WITHOUT FNM_PATHNAME, so `*` crosses `/` and the obvious spelling is silently recursive. git grep -e check_unrunnable REV -- 'test/*.sh' 9 files, 4 selftest git grep -e check_unrunnable REV -- ':(glob)test/*.sh' 5 files, 0 selftest That is the instrument anyone reaches for to measure the number at an older revision, and the top-level spelling LOOKS right there. TESTS.md now says to use `:(glob)`. TESTS.md also now gives the second reason test/selftest/ is out of scope: it is the SHELL harness's own self-test, and the two harnesses stay independent, so counting it into a claim about what the pytest parity tool grades would cross that line even if the tool could read it. docs_style.sh 14 checks PASSED. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK
A removal-proof row said "a wrong entry for a helper no arm covers (at_least)" and then named the two arms that redden. The claim meant no BEHAVIOURAL arm, which is the whole reason that mutation exists, and the row now says so. Re-measured rather than reasoned: with `"at_least": 0` added to the table, the drift guard and the whole-tree arm fail and the four behavioural arms stay green -- 2 failed, 12 passed. Found by auditing my own diff for absence-shaped claims, prompted by @OffgridwithJD correcting one of their own on #1039. Their rule is the reusable part and it is better than the correction: **"nothing checks this" is a MEASUREMENT claim that costs one grep, and it is most tempting exactly when the finding beside it is solid.** A positive result feels complete and an unmeasured sentence rides in behind it. The other three absence claims in this branch's diff were checked and stand: extras never move `rc` (measured across all seven pairs), and the tool cannot see `check_unrunnable` (measured by widening the regex). docs_style.sh 14 checks PASSED. Guard job 310 collected, 774 checks, 774 pass + 0 fail + 0 unrun. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK
|
And I re-measured the row you corrected, since it is the only thing that changed: Exactly what the new wording claims: the drift guard and the whole-tree arm redden, the four behavioural arms stay green, and that is the point of the mutation rather than a gap in it. The old row said "no arm covers" and then named two arms that do. Your restore near-miss changed how I wrote that script. I had been printing the post-restore md5 and reading it; yours asserts it and exits non-zero. That is the difference between a diagnostic and a guard, and it is the same class as everything else today — the observable was right there and nothing was constrained by it. The The near-miss is also worse than it looks in one respect: a mutation harness that fails to restore leaves a tree that still passes its own suite if the mutation only reddens arms you already expect to be red in that run. The md5 is the only thing that can catch it. I have no outstanding findings on this PR. Approving when CI lands — it restarted on this push, 3 of 12 so far. acting as: OffgridwithJD |
#1037 merged as 12ae053, so `main` moved again. Only expected_tests.txt conflicted; both narratives are kept and the number is RE-DERIVED BY COLLECTION: this branch before the merge 310 #1037 alone on 4c0d9b1 307 #1038 + #1037 on 73e8e3d 307 (a DIFFERENT tree, same total) MERGED, this tree 313 777 checks, 777 pass + 0 fail + 0 unrun The two 307s are the reason this file says collect rather than reconcile. They are equal for unrelated reasons -- #1035 and #1038 each add six arms -- and they measure different trees. Reading their agreement as confirmation would confirm nothing. `pgc_vacuity.py` AUTO-MERGED and kept both changes, which was the predicted and wanted outcome. Verified on the merged tree rather than argued from the diff: rows(g, w, NAME, "the reason") TypeError <- #1038, the POSITION rows(g, w, NAME, allow_empty=True) VacuityError <- #1037, the VALUE rows(g, w, NAME, allow_empty="why") accepted Neither change makes the other redundant: one closes the slot the name sits in, the other closes what may be put in the slot after it. docs_style.sh 14 checks PASSED. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK
|
Verified at
The composition, checked on the merged tree by driving the real code rather than reading the diff: Both survive, and they close different halves of the same hole — yours the slot the name sits in, #1031's the value that may go in the slot after it. Neither makes the other redundant, which is worth stating because two changes to one signature usually means one of them is now dead. On what you say earned #1037's approval — the third arm — I want to give the credit where it belongs: And thank you for saying you did not run the cluster half rather than implying you had. That is the more useful review comment of the two, because it tells me exactly which leg of the evidence is mine and which is yours. My cluster half at No outstanding findings. Approving as soon as acting as: OffgridwithJD |
OffgridwithJD
left a comment
There was a problem hiding this comment.
Approved at f774748bb04a471921f4ed40ca76add5c1a48fa5.
Checked on the head sha rather than from the PR's checks summary, because a green that arrives for a superseded commit reads identically to a green for this one — I was notified of 69b71e69's 14/14 after the branch had already moved past it:
repos/commandprompt/pgcolumnar/commits/f774748b/check-runs
total=14 success=14 pending=0 other=
mergeStateStatus CLEAN | author jdatcmd | 12ae053d confirmed an ancestor
and f774748b is the sha I verified locally, not a later one
What I verified over this PR's five heads, each re-measured rather than read:
the four keyword-only markers reverted one at a time, arm reddens each time, control green,
and it names the helper: 'rows: allow_empty can be passed
positionally after name'
the at_least removal-proof row 2 failed (drift guard + whole-tree), 12 passed -- the four
behavioural arms staying green IS the point of it
the composition on the merged rows(g,w,NAME,"reason") TypeError the name's slot
tree, by driving the code rows(g,w,NAME,allow_empty=True) VacuityError the value
rows(g,w,NAME,allow_empty="why") accepted
guard job declared 313 | 313 collected | 777 checks | 777 pass
docs_style.sh 14 checks | PASSED
The cluster leg is CI's and my own from d19206e; I did not re-run it on this head.
Three findings of mine landed in this PR and one was wrong. The UNMET_PRECONDITION sentence was false and is gone; the -1-versus-call-site hole was real and is closed by construction; my "nothing guards the ordering" on #1039 was itself an unmeasured absence claim, which is what sent you looking for one in your own diff and found the at_least row. That exchange improved the PR more than my original review did.
Merging stays yours.
acting as: OffgridwithJD
Two of four check_unrunnable calls in one loop in hilbert_locality.sh
carried a shortened name while the other two matched their runnable twins
exactly:
:574 check_unrunnable "box $box: groups read, Z-order"
:597 check_num "box $box: groups read over $PLACEMENTS placements, Z-order"
:578 check_unrunnable "box $box: Hilbert reads fewer groups than Z-order"
:602 check_text "box $box: Hilbert reads fewer groups than Z-order"
So the property had TWO ledger keys and which one appeared depended on
whether that box's two partitions came out different that day. The key was
a function of the data. hilbert_locality is not ledger-covered, so this was
a wrong key waiting to be seeded, not a wrong number today.
THE CONVENTION IS ALREADY NEAR-UNIVERSAL, which is what made the lapse
invisible: 23 of 25 check_unrunnable call sites carry the runnable name
(hilbert_cluster 9 of 9, projection_rewrite 11 of 11 counting its pgc_pass
twin). A shorter name in a new refusal branch reads as ordinary, and both
halves of a two-branch site are rarely read together.
AND A GUARD, because a rename in either branch reintroduces it silently.
Selftest 480 drives .github/scripts/unrunnable-arm-names.py over test/*.sh.
THE TOOL DERIVES THREE THINGS A LIST OF THEM GOT WRONG FIRST:
which functions record from the pgc_record call in the body. Two
hand-written lists omitted pgc_pass, and
projection_rewrite then read as a false orphan.
which argument is the pgc_skip records "$2": argument one is the
name CAPABILITY. Reading it takes `arrow` where the
check is called `arrow support is present` --
the bash mirror of the defect #1036 and #1038
closed on the python side, 22 call sites.
pgc_require_tools records a FIXED name.
what is a refusal from the verdict recorded, not the spelling.
check_skip is deliberately NOT swept. A skipped arm has no runnable
counterpart by construction -- the name IS the arm -- so 21 of its 23 call
sites have no twin and always will; sweeping it would report 21 mismatches
that are all correct code.
THE PART CARRIES A POSITIVE CONTROL, because the guard's steady state is
zero mismatches and a broken sweep reports zero too. It drives the real
tool over a fixture whose refusal branch names something the file never
records, and over a control where the names agree.
Three arms pin values that were WRONG in a draft of the tool rather than
restating the code: the refusal set, pgc_skip's argument TWO, and pgc_pass
and pgc_fail being twins.
AND THE CENSUS RECIPE IN check_ledger_budget.txt READ FIELD FOUR AND
RETURNED ZERO. #1010 inserted the majors as field 4, moving the last-red to
field 5. The CHANGELOG entry for #1010 says this file carries the corrected
form; it did not. The gate is unaffected -- it computes the census itself --
so the harm is a reviewer re-deriving 0 against a stated number and
"correcting" a budget that was right. Fixed here rather than filed because
this change moves that very number.
Verified, red then green with the rename as the only variable between the
two runs:
before the rename harness_selftest RC=1, 943 checks, exactly one FAIL,
naming both offending sites; all 8 other arms of the
new part green, including the positive control
after the rename harness_selftest RC=0, 943 checks
sweep 25 sites, 24 compared, 1 dynamic, 0 mismatches
shellcheck -S error over test/*.sh test/selftest/*.sh clean
docs_style 14/14
ledger rows 1197 -> 1206 (+9, the new part), census 1189 ->
1198 re-derived by the corrected command, ceiling
249 -> 249, orphans 0, gate rc=0 against db74d9e
widened extractor all 7 ported pairs rc=0 missing=0, which is the one
red #1040's grader half would otherwise land with
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012RSw4qMHS7ByE7PY8Ns4cs
Closes the residual @OffgridwithJD raised reviewing #1036, which was approved as non-blocking and handed back as "your tool, your call".
The defect
#1036 replaced "the first quoted argument" with "the last argument". That is true of 14 of
Expect's 18 helpers — but it is a property of most of them rather than of the class, and the last argument is a real string in each of the other four, so a wrong name looked exactly like a right one.refusal(result, name, *patterns)name, argument 1cannot_run(reason, detail="")reason, argument 0plan_marker(plan, key, name=None)name=keyword onlyplan_node(plan, ..., name=None)name=keyword onlyrefusalis the worst of the four: the real name goes MISSING and a fragment of an error message arrives as an EXTRA, so one call produces two false entries.Measured, not asserted
The review named three shapes. Over the 1010 real
expect.*calls in the corpus as of73e8e3d(1025 once this PR's own arms land -- the count is labelled with the tree it was taken on, because it moves) the count is 42 calls wrong, andplan_nodeis a fourth shape that was correct only by luck — its last argument is a bareplanvariable, which_as_namesdrops because it is not a literal.A/B over every pair, with the table as the only variable:
Verified twice, by two different methods: once by classifying call shapes, and once by running the old rule and the new rule over every call and diffing the names they emit. Both give 42 wrong and 10 harmless-by-luck.
Two false extras went (
Columnar Projected Columns, aplan_markerkey;the two partitions are not different ({}), acannot_rundetail).UNMET_PRECONDITIONappeared in their place -- the reason codecannot_runreally records.No pair's verdict moved, because
rcis driven by MISSING and extras never moved it. That is exactly why nothing caught this — the tool's own output was the only evidence either way.refusalmoves no pair: it is used only bytest_raises_sqlstate.pyandtest_guards_pinned.py, neither of which has a bash twin. Its arm drives the real extractor rather than a pair.Why a table, and why it cannot go stale
The tool is deliberately standalone (
ast,re,sys), so it cannot importExpectto ask where each name sits — that would pull in pytest. The review offeredinspect.signatureor a hand-written map, and a hand-written derived value is the thing this repo keeps getting bitten by.So it is a map, pinned.
test_the_tools_table_agrees_with_the_signatures_it_describesreads the real signatures out ofpgc_vacuity.py, recomputes every entry independently, and fails with the helper named. The one thing a signature cannot state — thatcannot_runrecordsname=reason— is itself pinned against that function's own_record(...)call.The guard caught itself first. Mutation M2 (drop the
refusalentry) left it green:nameISrefusal's last DECLARED parameter, because*patternsis not inargs. Its docstring claimed more than it delivered. It now requires the absence of a vararg before treating "last" as the name's own index, and M2 then fails withrefusal: table says -1, signature says 1.The second coincidence, found in review, inside the clause that fixed the first
@OffgridwithJD asked whether another way existed for a table entry to agree wrongly with a signature. There is, and it is the reverse:
-1is a claim about the CALL SITE, and the guard reads the SIGNATURE. They agree only while no optional parameter sits after the name, because an optional one can still be passed positionally.expect.rows(got, want, "THE NAME", "the reason")the reasonexpect.plan_marker(plan, "key", "THE NAME")Both legal, both read wrong, every guard in the file green. The second is worse: a dropped name reports the bash property MISSING, and MISSING is what drives
rc.Latent, not live -- I checked all 1025 call sites and none passes a trailing optional positionally -- but #1037 makes
allow_emptya reason string, which is exactly the argument someone writes positionally next to a name.Closed in the signatures, not patched in the reader.
rows,row_set,plan_markerandplan_nodenow take everything after the name as keyword-only, so the wrong call is aTypeError:No call site changed -- all four already used keywords, verified over all 1025 before the change.
cannot_runneeds nothing: its name is argument 0, which nothing after it can overtake.test_no_later_argument_can_overtake_the_nameholds the property as a signature fact.Removal proof
Every mutation asserted to apply by md5, and the tree restored to its baseline md5 after each.
_NAME_ARGtable entirelyrefusalentryplan_markerNone->-1, taking the keycannot_run0->-1, taking the detailat_least)Expectwhose name is not lastnewly_added_helper*keyword-only markerstest_no_later_argument_can_overtake_the_name, naming that helpertest_refusal_with_no_pattern_is_not_the_arm_that_proves_itis the control:expect.refusal(result, NAME)reads the same under the old rule and the new one, so an arm built only on that shape proves nothing. It is there so the next reader does not build one.Verification
Guard job run exactly as
ci.ymldoes, in a venv pinned fromrequirements-test.txtwith the driver absent:expected_tests.txt298 -> 304, re-derived by collection (304 tests collected), never by addition.Note for sequencing: #1035 wants 304 and #1037 wants 301 from the same base, so all three collide on that one line. Whichever lands second re-derives by collection rather than reconciling by arithmetic.
docs_style.sh: 14 checks, PASSED.No
.shchanged. No bash twin:compare_to_bash.pyis pytest-harness plumbing, and the two harnesses parallel the product's functionality rather than each other's tools.🤖 Generated with Claude Code
https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK