Split out of #858, which bundled four independent phases behind one title. This one is done; it is recorded as its own issue so the delivered work is citable rather than buried in a closed parent.
What it asked for
A check whose input is absent — a fixture that did not build, a capability the server lacks — either passes vacuously or fails for a reason unrelated to the property under test, and checks run: N counts it either way. Two states are not enough.
The ask: check_unrunnable NAME REASON_CODE DETAIL, a closed enum of reasons, check-level reconciliation, and an exit code that is not a pass, with failure still outranking it.
Delivered, in both harnesses
bash, in #859 and #874:
test/lib.sh:58 PGC_EXIT_INCOMPLETE=67
test/lib.sh:624 check_unrunnable NAME REASON_CODE DETAIL
test/lib.sh pgc_summary exits 67 when PGC_UNRUN != 0, AFTER the failure arm,
so a real failure still outranks an unrunnable check
pytest, in #897, with the twin the rule requires:
pgc_vacuity.py EXIT_INCOMPLETE, cannot_run(), _UnrunnableCollector
The distinction a reader will get wrong, measured
A pytest test that declares itself unrunnable produces:
UNRUN test_p.py::test_cannot: ABSENT_FIXTURE: no corpus
checks unrunnable: 1
exit code = 67
but pytest's own per-item tally still prints 1 passed. The session exit and the unrunnable count are what carry the refusal; the per-item outcome is unchanged and always was.
That matters because it is exactly what someone greps for. A reader checking passed concludes the refusal does not work — and #902's twin header asserted precisely that for a while, describing a gap that #897 had already closed. A wrong warning outlives a wrong claim, because nobody re-checks a warning.
Closed as completed.
Split out of #858, which bundled four independent phases behind one title. This one is done; it is recorded as its own issue so the delivered work is citable rather than buried in a closed parent.
What it asked for
A check whose input is absent — a fixture that did not build, a capability the server lacks — either passes vacuously or fails for a reason unrelated to the property under test, and
checks run: Ncounts it either way. Two states are not enough.The ask:
check_unrunnable NAME REASON_CODE DETAIL, a closed enum of reasons, check-level reconciliation, and an exit code that is not a pass, with failure still outranking it.Delivered, in both harnesses
bash, in #859 and #874:
pytest, in #897, with the twin the rule requires:
The distinction a reader will get wrong, measured
A pytest test that declares itself unrunnable produces:
but pytest's own per-item tally still prints
1 passed. The session exit and the unrunnable count are what carry the refusal; the per-item outcome is unchanged and always was.That matters because it is exactly what someone greps for. A reader checking
passedconcludes the refusal does not work — and #902's twin header asserted precisely that for a while, describing a gap that #897 had already closed. A wrong warning outlives a wrong claim, because nobody re-checks a warning.Closed as completed.