test: the piped-loop sweep needs a population, not just a detector (#1033)

CHANGELOG.md

@@ -18,6 +18,30 @@ true until the next version shipped.
 
 ### Added
 
+- The piped-loop sweep reported a clean tree without reading one (#1033).
+
+  `selftest/400` proves its detector FIRES -- a fixture with a check inside a piped
+  loop gives 1 -- and nothing proved it had EXAMINED anything. Without `nullglob` a
+  wrong `$PGC_TESTDIR` leaves both globs LITERAL, `awk` opens no file, `grep -c .`
+  over no input prints `0`, and the arm compares that `0` against `0` and passes:
+
+      PGC_TESTDIR=<a real dir with one offender>   hits=1   detector fires
+      PGC_TESTDIR=/nonexistent                     hits=0   ARM PASSES, nothing read
+
+  The same file already gets this right 260 lines above, where a different sweep
+  carries `[ -e "$_sk_f" ] || continue` and a `premise: the sweep classified a corpus
+  of check-calling files` arm. One sweep was premised and the other was not.
+
+  The population is now counted by what `awk` actually OPENED -- `FNR == 1` fires once
+  per file it reads -- rather than by `ls`, so a file that exists and cannot be read is
+  a miss rather than an invisible one, and the premise uses the same mechanism as the
+  detector so the two cannot drift apart. It is RECONCILED against what the globs
+  offered rather than floored at a constant: a literal glob offers 2 words and reads 0.
+
+  Removal proof: pointed at a nonexistent tree, `no suite calls a check inside a piped
+  loop` still PASSES -- which is the defect -- while both premises redden.
+
+  Closes the third of #1033's three gaps; `ae008c2` closed the other two.
 - `test/pytest/test_projections.py`: the multiple-projections DDL, catalog and read
   path, ported from `test/projections.sh` (#432). All 75 of its check names, one for
   one.
@@ -123,7 +147,6 @@ true until the next version shipped.
   case was forced synthetically instead, and the tool classifies the siblings
   `unprunable` rather than orphans.
 
-- The piped-loop sweep reported a clean tree without reading one (#1033).
 - Userinfo in an object-store ENDPOINT was accepted, and the diagnostic told the
   operator to allow-list it (#995).
 

test/check_ledger.tsv

@@ -842,10 +842,12 @@ harness_selftest	400-a-check-result-must-be-machine	pgc_fail emits one record	15
 harness_selftest	400-a-check-result-must-be-machine	pgc_pass emits one record	15;16;17;18;19	never	-
 harness_selftest	400-a-check-result-must-be-machine	premise: and it would name a file that calls check and echoes a SKIP	15;16;17;18;19	never	-
 harness_selftest	400-a-check-result-must-be-machine	premise: and that is this fragment, not the suite	15;16;17;18;19	never	-
+harness_selftest	400-a-check-result-must-be-machine	premise: and that population is the suite corpus	15;16;17;18;19	never	-
 harness_selftest	400-a-check-result-must-be-machine	premise: it is callable	15;16;17;18;19	never	-
 harness_selftest	400-a-check-result-must-be-machine	premise: it names a check_skip reading an unassigned variable	15;16;17;18;19	never	-
 harness_selftest	400-a-check-result-must-be-machine	premise: lib.sh is where the check helpers live	15;16;17;18;19	never	-
 harness_selftest	400-a-check-result-must-be-machine	premise: the fixtures carry the shapes this sweep is about	15;16;17;18;19	never	-
+harness_selftest	400-a-check-result-must-be-machine	premise: the piped-loop sweep read every file it was offered	15;16;17;18;19	never	-
 harness_selftest	400-a-check-result-must-be-machine	premise: the probe ran every helper shape once	15;16;17;18;19	never	-
 harness_selftest	400-a-check-result-must-be-machine	premise: the sweep classified a corpus of check-calling files	15;16;17;18;19	never	-
 harness_selftest	400-a-check-result-must-be-machine	premise: while a file that calls no check is not its business	15;16;17;18;19	never	-

test/check_ledger_budget.txt

@@ -54,4 +54,8 @@ suites_not_covered 249
 #   the number of attacked checks -- and it overcounts from the first moment this
 #   ledger does the job it exists for. The gate prints both quantities side by side
 #   (`rows=N | never observed red=M`) because they are different questions.
-checks_never_observed_red 1220
+# MERGED: this branch derived 1211 and #1070 derived 1220, each correct for a tree
+# that is not this one. Neither survives. Re-derived by COUNTING on the merged tree,
+# which is the only resolution this number has:
+#     awk -F'\t' '$5=="never"' test/check_ledger.tsv | wc -l
+checks_never_observed_red 1222

test/selftest/400-a-check-result-must-be-machine.sh

@@ -561,6 +561,50 @@ check "and not a piped loop with no check in it" \
 check "and not a check after a one-line piped loop that already closed" \
 	"$(_pipeloop_sites "$_pl_fx/oneline.sh" | grep -c .)" "0"
 
+# THE POPULATION, because the arm below can report a clean tree having read nothing.
+# The detector is proven to FIRE, at :548 against a fixture. Nothing proved it had
+# EXAMINED anything. Without `nullglob` a wrong `$PGC_TESTDIR` leaves both globs
+# LITERAL, `_pipeloop_sites` opens no file, `grep -c .` over no input prints 0, and the
+# arm compares that 0 against 0 and passes. Measured with the identical expression:
+#
+#     PGC_TESTDIR=<a real dir with one offender>   hits=1   detector fires
+#     PGC_TESTDIR=/nonexistent                     hits=0   ARM PASSES, nothing read
+#
+# Counted by what awk actually OPENED -- `FNR == 1` fires once per file it reads -- and
+# not by `ls`, so a file that exists and cannot be read is a miss here rather than an
+# invisible one. That is also the same mechanism the detector uses, so the premise and
+# the thing it premises cannot drift apart.
+#
+# RECONCILED against what the globs offered rather than floored at a number, so there is
+# no constant to maintain: a literal glob offers 2 words and reads 0, which is a
+# mismatch, while the real corpus offers and reads the same 313.
+#
+# `${_pl_read:-0}` IS LOAD-BEARING HERE, NOT BELT-AND-BRACES. On the literal-glob path
+# awk exits 2 WITHOUT reaching END, so `print n+0` never runs and the substitution is
+# EMPTY, not `0`. Measured:
+#
+#     awk against an unopenable file   stdout=[]   rc=2   END never runs
+#     awk against a real EMPTY file    stdout=[0]  rc=0   END runs, FNR==1 does not
+#
+# So `:-0` is what turns that silence into the number the mismatch is computed from.
+# It is the very shape #1033 is about -- a broken probe's silence becoming a value --
+# and it is safe ONLY because the reconciliation against `_pl_offered` is the next
+# line. Do not remove it as redundant; without it `check_num` would refuse the empty
+# string as "not a measurement", which is a correct refusal but a worse message.
+#
+# AND AN EMPTY FILE READS AS UNREAD, because `FNR == 1` never fires for one. A
+# `touch test/new_suite.sh` before it is written gives offered=313 read=312 and reddens
+# this arm with a message pointing nowhere near the cause. Zero empty `.sh` in the tree
+# today, and it fails CLOSED, so it is recorded rather than worked around.
+_pl_offered="$(set -- "$PGC_TESTDIR"/*.sh "$PGC_TESTDIR"/selftest/*.sh; echo $#)"
+_pl_read="$(awk 'FNR == 1 { n++ } END { print n+0 }' \
+	"$PGC_TESTDIR"/*.sh "$PGC_TESTDIR"/selftest/*.sh 2>/dev/null)"
+check_num "premise: the piped-loop sweep read every file it was offered" \
+	"${_pl_read:-0}" "${_pl_offered:-0}"
+# And that the population is the suite corpus rather than a stray directory that
+# happens to reconcile. Only a mass deletion of suites can approach this floor.
+check "premise: and that population is the suite corpus" \
+	"$([ "${_pl_read:-0}" -ge 200 ] && echo yes || echo no)" "yes"
 _pl_hits="$(_pipeloop_sites "$PGC_TESTDIR"/*.sh "$PGC_TESTDIR"/selftest/*.sh 2>/dev/null | grep -c . || true)"
 [ "${_pl_hits:-0}" = 0 ] || _pipeloop_sites "$PGC_TESTDIR"/*.sh "$PGC_TESTDIR"/selftest/*.sh | sed 's/^/    /'
 check "no suite calls a check inside a piped loop" "${_pl_hits:-0}" "0"