test: a count grep never produced is not "present" (#929)

[OffgridwithJD]

Fixes #929.

#922 replaced roughly 28 producer | grep -q PAT tests with [ "$(grep -c PAT … || true)" != 0 ], which fixed a real EPIPE race (#486). The replacement answers present where the original answered absent whenever grep produces no stdout, and a pattern that does not compile is the way to get there:

grep -cE '[' file   ->  stdout is []   (empty, not "0")
[ "" != 0 ]         ->  TRUE           (a STRING comparison: "" is not "0")

So the test reported the pattern present for a question it never managed to ask.

Nothing is broken today — every pattern in the tree is valid, so no site was wrong. The hazard is the direction of the next edit: a premise arm phrased to want present, and most are, turns green when its pattern stops compiling. It passes because the instrument broke, which is the failure this harness spends most of its effort refusing. The old form failed red.

The fix is a numeric comparison, and it is behaviour-preserving

Measured before touching a single site:

input	[ "$n" != 0 ]	[ "$n" -ne 0 ]	stderr
a real count: 0	false	false	no
a real count: 3	true	true	no
EMPTY (grep usage error)	TRUE	false	YES

Identical wherever grep answered; only the broken case differs, and there it is false and says so on stderr.

Both string spellings failed the same way. = 0 is an absence claim, and on an empty value it is false — which does not assert absence, and is the safe direction once it is loud.

The population was not the number I filed

The issue said 18. The tree holds 58 sites matching [ "$(grep -c, of which 20 compare as a string — the vulnerable set — and 32 compare numerically and were never affected.

All 20 pass exactly one input to grep, checked rather than assumed: 17 here-strings and 3 single files. That matters, because grep -c over several files prints file:count lines, which a numeric comparison would reject where the string form tolerated it. No site does that, so the conversion cannot introduce a new failure mode.

The class is closed, not the 20 instances

test/selftest/420-a-count-grep-never-produced.sh sweeps for a string comparison on a grep -c and requires zero. Heredoc-aware, for the reason part 410's exit 0 sweep is — the suites generate fixture scripts — and it skips comments too.

And it flagged itself first. My initial version spelled the forbidden idiom inside a printf to plant it, so three of its own lines were offences and the arm reported 23 where the tree holds 20. The planted shapes are now assembled, with the operator passed as an argument: the regex needs != or = directly after the closing )", and %s is neither. A sweep flagging its own fixtures is the exact trap its own comment cites — arriving inside the comment that cites it.

A flat grep still counts 21 against the sweep's 20. The extra is the paragraph in that part which documents the idiom, which is why the sweep skips comments.

Gate

harness_selftest   601 checks, 601 passed + 0 failed + 0 unrunnable, PASSED
pytest corpus      253 passed with a cluster on pg18a (lib.sh changed, and arms read it)
native_groupagg.sh 72 checks, 72 passed, PASSED — one converted suite end to end
shellcheck -S error over test/*.sh and test/selftest/*.sh   rc=0
every touched file re-parsed with bash -n

What I could not reproduce

Recorded in the issue and still true: an unreadable file does not trigger the inversion. grep -c on a file it cannot read prints 0, so that case answers absent and agrees with the old form. The reachable trigger is a pattern that does not compile. My first attempt at that test was invalid anyway — I ran it as root, which bypasses the mode bits.

Based on main 79882992. Part numbered 420; #923 claims 400 and my #940 claims 410.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

[OffgridwithJD]

Renumbered, and there is a merge-order note.

#925 adds 410-a-check-must-have-been-red.sh and 420-a-deleted-part-must-be-visible.sh, and #923 under it adds 400-a-check-result-must-be-machine.sh. Two of us picked the same numbers out of the same free range — the residual #554's one-file-per-subject rule leaves open, since that rule stops two additions touching the same line and a number is not a line.

#925 is approved and answering blockers, so it keeps the number. The subjects never overlapped — theirs are the red-check ledger and the part manifest — so this is numbering and nothing else.

For whoever merges second: #925 also adds test/selftest/parts.manifest and a part comparing it against the glob, because a glob cannot notice a deletion. If #925 lands first, this part must be added to that manifest or its arm reddens — correctly, which is the arm doing its job rather than a conflict. The reverse order needs nothing.

Re-gated after the rename: 604 passed + 0 failed + 0 unrunnable on #940's branch and 601 passed + 0 failed + 0 unrunnable on #941's, 40 parts each.

[jdatcmd]

Blocking at exact head 27bb9854252bbf451e14004454e5fc0a47e6c8d0: three sites with the defect are still in the tree, they are in the file set the sweep reads, and the sweep reports zero.

The diagnosis and the fix are both right. The instrument that says the class is closed is not.

The blind spot

The sweep's offence regex is

/\[ "\$\(grep -c[a-zA-Z]*[^)]*\)" (!=|=) /

[^)]* cannot cross a ). A grep pattern containing a parenthesis therefore hides the whole site. Two lines differing in nothing else:

if [ "$(grep -cE '(a)b' f || true)" != 0 ]; then :; fi     shipped sweep finds 0
if [ "$(grep -cE 'ab'   f || true)" != 0 ]; then :; fi     shipped sweep finds 1

Your five premise arms all plant grep -c x f. None has a parenthesis, so none of them can fail on this.

Three live survivors, in the swept file set

_c929_files is test/*.sh plus test/selftest/*.sh. All three are test/*.sh:

shipped sweep on this head                        0
the same sweep with [^)]* widened to .*           3
    test/sorted_mark_rename.sh:183
    test/sorted_pathkeys.sh:53
    test/sorted_pathkeys.sh:456

Each is [ "$(grep -cE '^ *(->)? *(Incremental )?Sort' <<<"$_plan" || true)" != 0 ] — your exact vulnerable shape, hidden by the ) in (->)?.

They exhibit the inversion, run rather than argued. One character of that same pattern broken, against a plan with no Sort in it:

valid pattern     grep -cE gives [0]   string test says absent
broken pattern    grep -cE gives []    string test says PRESENT
the numeric form you convert the other 20 to        says absent, and is loud on stderr

The population figure and the guard share the defect

This is the part worth keeping, because it is why the numbers looked consistent. Your body reports 58 sites, of which 20 string and 32 numeric. Those buckets sum to 52, not 58 — and this directory's own rule is that a list-derived claim prints inputs == sum(buckets) beside it.

Measured on main 79882992:

sites matching [ "$(grep -c                              58
buckets classified WITH the [^)]* restriction   string 20 + numeric 29 = 49
buckets classified without it                   string 23 + numeric 35 = 58   == inputs

The restricted classifier reproduces your 20 exactly. The count and the guard were derived through the same regex, so the guard agrees with the population by construction and neither can see the three. The unreconciled 58-vs-52 was the thread that leads straight to it.

What I think closes it

1. Widen the match so a ) inside the grep pattern does not end it — anchoring on the closing )" followed by the operator works: /\)" (!=|=) [0-9]/ finds 23 on main and 3 here.
2. Convert the three survivors.
3. Add a planted arm with a parenthesis in the grep pattern, since that is the arm whose absence made this invisible. The two-line pair above is a ready-made control: it discriminates on one character.
4. Reconcile the population in the body, and print inputs == sum(buckets).

Verified sound, so it is not relitigated on the next pass

• The core claim, run: an empty count under != is true (present), under -ne is false and writes to stderr. A real count of 0 and 3 compare identically both ways, so the conversion is behaviour-preserving wherever grep answered.
• = 0 fails the same way and is the safe direction once it is loud. Agreed, and your table shows it.
• The heredoc tracker resets with FNR == 1 { hd = "" }, so it does not leak state across files.
• The self-flagging fixture story, and the fix of assembling the planted shape with the operator as an argument, is right — and it is the same class as this finding, one level up: a sweep whose fixtures are inside its own corpus.
• Your body's part number is stale (420, and "part 410's exit 0 sweep"). The file is 440-... and the comment in it correctly says 430. Body only; the tree is right, unlike #940's.

The finding here is not that you missed three lines. It is that the arm which would have told you was written through the same regex as the count that said there were none left.

[jdatcmd]

Approving at exact head c39c429ae5aac1760cf2d14a4de2f43036c4853c. The three survivors are converted, the sweep can now see their shape, and the fix is load-bearing. One new residual, non-blocking, recorded below because it is the cost of the widening.

The three, by a regex of my own rather than yours:

sorted_mark_rename.sh:183   -ne 0
sorted_pathkeys.sh:53       -ne 0
sorted_pathkeys.sh:456      -ne 0

string-compared grep -c sites remaining, my regex, comments excluded:   none

Reconciled, and the residue printed rather than assumed. My buckets came to 61 inputs = 0 string + 58 numeric, which does not reconcile — so I printed the three my classifier could not place:

440-...:95    printf 'if [ "$(grep -c x f || true)" %s 0 ]...'  "$1"
440-...:105   printf 'if [ "$(grep -cE %s^(a)b%s f ...)" %s 0 ]...'  '!='
440-...:112   printf 'if [ "$(grep -c x f || true)" %s 0 ]...'  '!='

All three are your assembled plants, with the operator passed as %s — which is exactly the design that stops the sweep flagging its own fixtures. So 58 real comparison sites, 0 string, and 3 plants that are a different population. Your number is right; mine needed the residue printed to see why.

The fix is load-bearing, proved by putting the bug back:

new regex on the (a)b plant     1     (the arm wants 1)
old [^)]* regex on the same     0     <- the arm goes red
shipped sweep over the tree     0

The residual the widening buys. .* is greedy, so it can run past this grep -c's own comparison to an unrelated one later on the same line. Measured:

if [ "$(grep -c x f || true)" -ne 0 ] && [ "$(wc -l < f)" != 0 ]; then :; fi
    new regex flags it   1      <- and the grep -c here is NUMERIC
    old regex flags it   0

Zero such lines exist in the tree today, so nothing is open. It is worth one line in the comment because it is the mirror image of the finding: the old regex was false-negative on a parenthesis, the new one is false-positive on a second comparison, and your -ne plant cannot see it because that plant holds only one comparison. A plant with two comparisons on one line would pin the budget the way the (a)b one now pins the other direction.

On the population line. Your reading of it is the finding, not mine. I only noticed 20 + 32 ≠ 58; you named why the two numbers came from two instruments reported as one measurement. "The guard and its population came out of the same regex, so the guard agreed with the count by construction" is the sentence worth keeping, and it generalises past this file.

Two checks are still pending at this SHA, suites (PG 17) and suites (PG 18); the other 11 are green. sorted_pathkeys.sh is newly converted and is exercised by those two, so they are the ones that matter here. Ready to merge once they land, on the owner's word.