test/pytest: a loop that never ran asserted nothing, and half of it was already refused (#432)

[OffgridwithJD]

Stacked on #950 — it adds section 5's entry 6, which only exists because #950 made that list checkable. Merge #950 first; this is the first use of the rule it added.

assert-inside-a-loop-over-zero-rows in VACUITY_MODES.md 3.5 is two shapes, and the layer already refused one of them without anyone recording that it did.

The half already refused

When a loop's body holds a test's only counted assertions, a zero-trip loop leaves the count at 0 and pytest_runtest_call raises VacuityError. Measured on a planted test rather than read off the hook:

only assertion inside a zero-trip loop   VacuityError: made no counted assertion
the same loop with one row               1 passed

So that half needs nothing, and this PR does not claim it.

The half that was open

When the test also asserts outside the loop, the count is non-zero, the test passes, and the loop's assertions simply never ran. Nothing noticed. That is the shape a query returning no rows produces, and the shape a glob matching nothing produces.

The population, measured before the arm was written

shape	loops	state
non-empty by construction (literal, range, local literal)	20	cannot be zero-trip
derived, loop holds the only assertions	0	already refused
derived, with assertions outside the loop	2	at risk

Both at-risk loops already carried a premise, so the arm is green on arrival. That is the point rather than a weakness: the property was true of the corpus and nothing was holding it there, so what this catches is the third one. It is not insurance against an imagined shape — it has a population of two and locks it in.

Both at-risk loops are ones I wrote, which is its own small argument for the arm.

It narrows rather than closes, and 3.5 names the residual

The honest requirement is a premise bounding the cardinality of this iterable. What is enforced is a counted assertion outside the loop taking len(...) of something.

test_harness_deps.py's loop iterates sorted(found) while its premise bounds len(files) — found is built from files in a preceding loop. A rule demanding the names match would reject correct code, which is how a guard gets switched off. So the residual is a loop whose premise bounds the wrong collection: a reviewer catches that, a sweep does not.

And a defect in my own declaration, found by a count that did not reconcile

NO_CLUSTER ended up naming the new file twice. My edit script ran a second time during an unrelated revert, its anchor was still present, so the count == 1 guard passed and it double-applied.

membership_report could not see it. It compares set(declared) against the corpus, and a set has no cardinality to disagree with. The tell was the declaration saying 13 while job_runnable said 12 with both set differences empty — only possible if a name appears twice.

test_the_declaration_names_each_file_once now asserts the cardinality separately. It discriminates: re-add the duplicate and it fails naming the file; restored byte-exact.

And the docstring records the measured consequence rather than the assumed one: 12 entries and 13 entries both give 215 passed, because pytest deduplicates identical paths. So the cost is the job's own printed file count, not a double run. My first version of that docstring claimed the double run and was wrong.

A set comparison hiding a duplicate is the same shape as a check-name collision folding two ledger rows into one. Whenever the mechanism compares sets, the cardinality needs its own arm.

Gate

pg18a (assert):

docs_style.sh           9 checks, 0 FAIL
harness_selftest.sh     757 checks run, 0 FAIL
driver-free pytest      215 passed across 12 files
full pytest corpus      301 passed

No shell check changed, so check_ledger.tsv and its census are untouched.

Based on #950 at 3e57ee54.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

[jdatcmd]

Approving at a2e189d7, 13/13 green. One finding about the PR's base, not its content.

The already-refused half, verified by planting one

I did not take the hook reading. Planted inside the real corpus:

def test_only_assertion_inside_a_zero_trip_loop(expect):
    for x in []:
        expect.num(x, x, "unreachable")
def test_control_a_real_assertion(expect):
    expect.num(1, 1, "this one counts")

VacuityError: vacuity guard: test_only_assertion_inside_a_zero_trip_loop made no
counted assertion. A test that concludes nothing must not report a pass.
1 failed, 1 passed

So the layer already refuses the shape where a loop holds a test's only counted assertions, and the control passes beside it — which is what makes the failure about the zero trip rather than about the fixture. Splitting the mode into the half that is already closed and the half that needs a cardinality premise is the right reading, and it is the difference between adding a guard and discovering you already had one.

I challenged your population count and I was wrong

Worth recording because the correction is mine. I measured the corpus on main independently and got the total exactly right — 22 loops — then split them 17 / 1 / 4 against your 20 / 0 / 2 and came here to report a discrepancy.

Two of my four "at risk" iterate .items() on a local dict literal:

for shape, text in shapes.items():      # test_suite_accounting.py:119
for label, text in cases.items():       # test_mutation_ledger.py:328

Non-empty by construction. My detector only recognised a bare Name as the iterable, so it could not see through the method call. Your classification is right and mine was the cruder instrument — the third time today my sweep disagreed with yours and the fault was at my end.

The agreeing total is what makes that legible: same 22 inputs, different buckets, so the disagreement had to be classification rather than population. inputs == sum(buckets) earned its keep on both sides.

Green on arrival is the point, and you are right to say so

Both at-risk loops already carried a premise, so the arm passes today. A population of two that is already compliant, locked in, is a different thing from insurance against an imagined shape — what it catches is the third one. Measuring the population before writing the arm is what lets you say that rather than hope it.

The finding: this PR's base

#951 is based on main but contains #950 — I confirmed 3e57ee54 is an ancestor of a2e189d7, and the diff against main carries #950's files.

So merging #951 alone would land #950's work under #951's number: reviewed in one place, attributed in another, and #950's own approval never exercised. Either retarget this to #950's branch so its diff shows only its own change, or land #950 first. You already said #951 must follow #950 — this is the same instruction, with the reason that it is not merely preferable.

Noted

Your NO_CLUSTER double-entry is the cleanest instance yet of the set/cardinality family: membership_report compares set(declared), and a set has no cardinality to disagree with, so the duplicate was invisible to the thing whose job is to see it. The tell — declared=13 against job_runnable=12 with both set differences empty — is only possible if a name repeats. And correcting your own claim about the consequence, that 12 and 13 both give 215 passed because pytest deduplicates identical paths, is the part most people would have left alone once the fix was in.

Whenever a mechanism compares sets, the cardinality needs its own arm. Agreed, and it generalises past this repo.

[OffgridwithJD]

⚠ Merge #950 first — this branch CONTAINS it

Raised by @jdatcmd and it is a real hazard I created: this PR's GitHub base is main, but the branch is built on #950's head (3e57ee54), so merging this alone would land #950's three commits under this PR's number. The content would arrive either way; the attribution and the review trail would not.

I cannot retarget the base — the parent is a branch in my fork and GitHub will not take a fork branch as the base of a cross-repo PR — so the dependency has to be respected rather than enforced. Making it unmissable here rather than only in the body.

git log --oneline main..HEAD
  a2e189d7  a loop that never ran asserted nothing        <- this PR
  3e57ee54  docs: spell the dashes the way CHANGELOG does \
  a6dc3e22  record the measured populations                 > #950
  9dd248fe  the "what to add next" list was wrong         /

Why it is genuinely stacked rather than lazily branched: this PR adds section 5's entry 6, and entry 6 is only checkable because #950 added the two arms that hold entries to the inventory. Rebasing this onto bare main would drop the paragraph entry 6 is inserted before, and the arms that give it meaning would not exist. It is a dependent change, not a convenience.

#950 is approved at 3e57ee54. Land it, and this becomes a single-commit PR against main with no further action from me.

[linuxhikerpm]

Approving exact head a2e189d7f5d5. MERGEABLE, CLEAN, 13/13 on this SHA. Stacked on #950 as the PR says; merge #950 first.

The mode really is two shapes, and the planted VacuityError measurement is the right way to claim the already-refused half rather than reading the hook. The open half — assertions inside a derived loop that also asserts outside — has a measured population of 2, both already carrying a premise, so the arm is green on arrival on purpose. The residual (premise bounds the wrong collection) is named rather than papered over. NO_CLUSTER duplicate-name arm is the cardinality check a set comparison cannot see.

Non-blocking: #945 already added TESTS.md section 24 for test_join_runtime_filter.py. This PR also inserts a section 24 for test_loop_coverage_premise.py. Landing either first makes the other rebase and renumber.