From 796196e08b8c3fe77c9a794c7df3367fdfbd2618 Mon Sep 17 00:00:00 2001 From: OffgridwithJD Date: Sat, 12 Sep 2026 04:13:56 -0600 Subject: [PATCH] test: the accounting breakdown says which LINE it means (#752) The runner printed: N via lib.sh's accounting; by their own mechanism: audit bench_guards ... and "by their own mechanism" was read as "emits no RESULT records" -- by two different readers, hours apart -- and a bound on how far suites_not_covered can fall was derived from that reading: twelve suites called impossible to seed. IT IS FALSE. Measured by running all twelve and counting RESULT records: audit 31 smoke 9 phase2 42 phase3 32 phase4 38 phase5 36 phase6 43 concurrency 7 unique_conc 31 update_conc 25 bench_guards 0 docs_style 0 TEN OF THE TWELVE EMIT RECORDS. The set is the suites whose log lacks lib.sh's `accounting:` line and carries their own `checks run:` instead -- about the accounting LINE and nothing else. Only bench_guards and docs_style emit none, for the reason pgc_log_shows_any_accounting's own comment already gives: those two never source lib.sh. Defining a private `check` is orthogonal; audit does it and records 31. So the bound is two suites, not twelve, with ~294 records in the other ten. The label now names the line and disclaims the records in the same breath, with the measurement beside it so the next reader does not re-derive it. WHAT IS NOT DONE, stated rather than omitted: the disclaimer is not separately pinned. Pinning it needs either a new check -- a ledger row -- or renaming the existing arm, which would ORPHAN its row, and main has no tool to remove an orphan until #983 lands. So the arm keeps its name and greps the reworded label, the sequencing is in its comment, and widening it belongs to the change that regenerates the ledger anyway. NO LEDGER CHANGE: no check is added, removed or renamed, so the check set does not move. Verified by running harness_selftest on this tree and on main and comparing the counts rather than assuming it. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a --- CHANGELOG.md | 32 +++++++++++++++++++ test/run_all_versions.sh | 11 ++++++- .../390-a-registered-suite-must-account.sh | 15 ++++++++- 3 files changed, 56 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d57dfad3..237673ab 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2055,6 +2055,38 @@ true until the next version shipped. not unwrite it, and rewriting `main` is not something a stray artifact justifies. What this stops is the tree carrying them, and the next `git add -A` re-adding them. +- The matrix runner's accounting breakdown says which LINE it means, because the old + wording produced a wrong planning number twice in one night. + + It printed: + + N via lib.sh's accounting; by their own mechanism: audit bench_guards concurrency ... + + and "by their own mechanism" was read -- by two different readers, hours apart -- as *emits + no RESULT records*, which would make those twelve suites impossible to seed into the + mutation ledger. A bound on how far `suites_not_covered` can fall was derived from that + reading. + + **It is false. Measured by running all twelve and counting:** + + audit 31 smoke 9 phase2 42 phase3 32 phase4 38 phase5 36 + phase6 43 concurrency 7 unique_conc 31 update_conc 25 + bench_guards 0 docs_style 0 + + **Ten of the twelve emit records.** The set is the suites whose log lacks lib.sh's + `accounting:` line and carries their own `checks run:` instead -- a statement about the + accounting LINE and nothing else. Only `bench_guards` and `docs_style` emit no records, for + the reason `pgc_log_shows_any_accounting`'s comment already gives: those two never source + `lib.sh` at all. Defining a private `check` is orthogonal -- `audit` does it and records 31. + + So the bound is **two suites, not twelve**, with about 294 records sitting in the other ten. + + The label now names the line and disclaims the records in the same breath. The selftest arm + that pins it follows the reword; widening that arm to pin the disclaimer separately needs + either a new ledger row or a rename that would orphan an existing one, and `main` has no + tool to remove an orphan until #983 lands -- so the sequencing is written into the arm's + comment rather than quietly skipped. + ## [1.0-alpha3] - 2026-09-02 ### Added diff --git a/test/run_all_versions.sh b/test/run_all_versions.sh index 42bbe63c..cc471633 100755 --- a/test/run_all_versions.sh +++ b/test/run_all_versions.sh @@ -1474,7 +1474,16 @@ pgc_tally_suite() { # pgc_tally_suite NAME VERDICT LOGFILE echo " suites that ran: $suites_ran of ${#SUITES[@]} (skipped: $suites_skipped, incomplete: $suites_incomplete)" echo " of those, $_acc_any accounted for their checks and $((suites_ran - _acc_any)) did not" if [ -n "${_acc_own// /}" ]; then - echo " $_acc_ran via lib.sh's accounting; by their own mechanism:${_acc_own% }" + # SAY WHICH LINE, because "by their own mechanism" was read twice in one night + # as "emits no RESULT records" and produced a wrong planning number from it. + # This set is the suites whose log lacks lib.sh's `accounting:` line and has + # their own `checks run:` instead. That is a statement about the ACCOUNTING + # LINE and nothing else: measured over all twelve of them, ten emit RESULT + # records perfectly well (audit 31, phase4 38, unique_conc 31, ...) and only + # bench_guards and docs_style emit none -- which is the pair the comment on + # pgc_log_shows_any_accounting already names, for the real reason: those two + # never source lib.sh at all. + echo " $_acc_ran printed lib.sh's accounting line; these printed their own \`checks run:\` instead (a different accounting LINE, not a missing RESULT record):${_acc_own% }" fi if [ "$suites_skipped" != 0 ]; then echo " skipped:${skipped_names}" diff --git a/test/selftest/390-a-registered-suite-must-account.sh b/test/selftest/390-a-registered-suite-must-account.sh index c8228077..3467dfeb 100644 --- a/test/selftest/390-a-registered-suite-must-account.sh +++ b/test/selftest/390-a-registered-suite-must-account.sh @@ -695,5 +695,18 @@ check "the breakdown headline counts the wide set, not the narrow one" \ "$(grep -c 'of those, \$_acc_any accounted for their checks' "$_rv")" "1" check "and the population reconciliation counts that same file" \ "$(grep -c '_acc_any="\$(grep -c \. "\$_acc_accounted"' "$_rv")" "1" +# THE LABEL WAS REWORDED AND THIS ARM FOLLOWS IT. It used to grep "by their own +# mechanism", which was read as "emits no RESULT records" twice in one night by two +# different readers, and a planning number came out of the misreading: twelve suites +# called unseedable when ten of them emit records and only two do not. +# +# THE DISCLAIMER IN THAT LABEL IS NOT SEPARATELY PINNED, and the reason is worth +# stating rather than leaving as an omission. Pinning it means either a second check +# or renaming this one to match a wider assertion. A second check costs a ledger row; +# renaming this one ORPHANS its existing row -- and `main` currently has no tool to +# remove an orphan, which is exactly what #983 is about. So the honest sequence is: +# #993 lands the prune, and then this arm can be renamed and widened in the change +# that regenerates the ledger anyway. Until then the reword is protected by this grep +# breaking on any further edit, and by the comment at the echo itself. check "premise: and the narrow count is still printed, as the lib.sh half" \ - "$(grep -c 'via lib.sh.s accounting; by their own mechanism' "$_rv")" "1" + "$(grep -c "printed lib.sh's accounting line" "$_rv")" "1"