One matrix report prints two different answers to "how many suites accounted for their checks", three lines apart, and a reader cannot tell which is the claim.
Measured on a full test/run_all_versions.sh for PG 17 on /usr/local/pg17a, tree clean, under the lock, with ci.yml's environment (PGC_SKIP_TIMING=1 PGC_JOBS=4 PGC_REQUIRE_ISOLATION=1):
accounting reconciliation: inputs=239 | both=239, declared only=0, accounted only=0 | sum=239
population reconciliation: registered=251 | accounted=237, not dispatched=4, known debt=10, unaccounted=0 | sum=251
suites that ran: 242 of 251 (skipped: 9, incomplete: 0)
of those, 235 accounted for their checks and 7 did not
237 and 235. Both describe suites that accounted for their checks, in the same report, and they differ by exactly 2.
Why
The two lines derive from different readers:
- the population reconciliation counts
accounted with the wide reader, pgc_log_shows_any_accounting;
- the breakdown line derives
_acc_ran from _acc_observed, which is built with the narrow reader, pgc_log_shows_accounting.
The narrow reader matches lib.sh's accounting: line. The wide one also recognises a suite that accounts by its own mechanism. The gap is therefore the suites that keep a private tally instead of lib.sh's — bench_guards and docs_style, which is exactly 2, and which I had measured independently from the other direction earlier today: of the twelve registered suites that never call pgc_summary, exactly those two maintain a tally of their own.
So of those, 235 accounted ... and 7 did not overstates the debt by 2. The figure a reader acts on is the second one, because it is the one phrased as a problem.
Why it matters more than a cosmetic mismatch
The breakdown line exists to stop an overcount — it was added because counting a suite as having "run" when it never accounted for its checks was hiding exactly this class of gap. Deriving it from the narrower reader reintroduces a smaller version of the same overcount, one level down, in the line added to close it.
Related, in the same comment
test/run_all_versions.sh, in the comment immediately above _acc_ran, says:
Ten registered suites exit 0 having never called pgc_summary
That conflates two different populations. Measured on this tree: twelve registered suites never call pgc_summary; ten of those keep no tally at all. The two the sentence drops are bench_guards and docs_style — precisely the two that make the numbers above disagree. Both halves of the sentence are true of something; neither is true of what it says.
Suggested shape of a fix
Derive both lines from one reader, or say in the line itself which question it answers. If the breakdown is meant to be "accounted via lib.sh's accounting", the remaining suites are not a debt, they are a different mechanism — and naming them as such would make the 2 visible instead of counted against the total.
Found while running the matrix on #925's branch (2289f10d); it is not that branch's defect. The code came in with #922 and is on main at f0f1f40.
One matrix report prints two different answers to "how many suites accounted for their checks", three lines apart, and a reader cannot tell which is the claim.
Measured on a full
test/run_all_versions.shfor PG 17 on/usr/local/pg17a, tree clean, under the lock, with ci.yml's environment (PGC_SKIP_TIMING=1 PGC_JOBS=4 PGC_REQUIRE_ISOLATION=1):237 and 235. Both describe suites that accounted for their checks, in the same report, and they differ by exactly 2.
Why
The two lines derive from different readers:
accountedwith the wide reader,pgc_log_shows_any_accounting;_acc_ranfrom_acc_observed, which is built with the narrow reader,pgc_log_shows_accounting.The narrow reader matches
lib.sh'saccounting:line. The wide one also recognises a suite that accounts by its own mechanism. The gap is therefore the suites that keep a private tally instead oflib.sh's —bench_guardsanddocs_style, which is exactly 2, and which I had measured independently from the other direction earlier today: of the twelve registered suites that never callpgc_summary, exactly those two maintain a tally of their own.So
of those, 235 accounted ... and 7 did notoverstates the debt by 2. The figure a reader acts on is the second one, because it is the one phrased as a problem.Why it matters more than a cosmetic mismatch
The breakdown line exists to stop an overcount — it was added because counting a suite as having "run" when it never accounted for its checks was hiding exactly this class of gap. Deriving it from the narrower reader reintroduces a smaller version of the same overcount, one level down, in the line added to close it.
Related, in the same comment
test/run_all_versions.sh, in the comment immediately above_acc_ran, says:That conflates two different populations. Measured on this tree: twelve registered suites never call
pgc_summary; ten of those keep no tally at all. The two the sentence drops arebench_guardsanddocs_style— precisely the two that make the numbers above disagree. Both halves of the sentence are true of something; neither is true of what it says.Suggested shape of a fix
Derive both lines from one reader, or say in the line itself which question it answers. If the breakdown is meant to be "accounted via
lib.sh's accounting", the remaining suites are not a debt, they are a different mechanism — and naming them as such would make the 2 visible instead of counted against the total.Found while running the matrix on #925's branch (
2289f10d); it is not that branch's defect. The code came in with #922 and is onmainatf0f1f40.