test: pgc_reconcile_records reports an impossible mismatch on a pytest log

[OffgridwithJD]

pgc_reconcile_records compares a suite log's stated checks run: N against the number of RESULT records it carries. Handed a pytest log it reports a mismatch that cannot be true:

  records=0 but the log states checks run: 126
  126 check(s) reported an outcome the count never saw:
    a check ran in a subshell, so its counter bump died with it while its
    ...

Both harnesses print checks run:. Only the shell one emits RESULT<tab> records. So the function reads a stated total, finds zero records, and concludes the suite lost 126 outcomes.

How it surfaced

Not by looking for it. @jdatcmd observed that a bare bash test/<suite>.sh never reaches this check -- it lives in run_all_versions.sh -- so every green either of us called from a development run today was called without it. I ran it over all 64 suite logs I had produced:

  64 logs reconciled   genuine mismatches: 0   false positives: 1

The one flagged log was a pytest log from an unrelated earlier run.

Why it is latent rather than live

run_all_versions.sh only ever hands this function shell-suite logs, so nothing in the gate can hit it today. It becomes live the moment anyone reconciles a directory of logs rather than a list -- which is the natural thing to do when auditing, and is how it was found.

What is missing

The function already guards two input faults, and its comments name both:

• an absent log -- "no log to reconcile records against"
• a log with no checks run: line at all -- "never reached its summary ... must not read as a clean reconciliation"

It does not guard the third: a perfectly good log of the other harness. One premise closes it -- if the log states a total and carries no RESULT records at all, that is a log this function cannot speak about, and saying so is different from reporting that every check vanished.

A second, smaller thing in the same output

The message prints a cause beneath the count:

a check ran in a subshell, so its counter bump died with it

That is the fault it was written for, and it is a hypothesis, not part of the measurement. On @jdatcmd's counter-clobber failure it would have sent them to subshells rather than to their own assignments; on a pytest log it explains a mismatch that is not real. The count above it is the evidence. Worth separating the two, or hedging the sentence.

Not proposed

Making the pytest harness emit RESULT records, or making this function understand both formats. The two harnesses stay independent, and the fix here is for the function to recognise that it has been handed something it cannot measure.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MpajdQbkVJ9ey1XyYHcikP

[OffgridwithJD]

Taking this one.

@jdatcmd, one heads-up before you re-push #1243. The fix here lives in
pgc_reconcile_records, and the review I just left on #1243 asks for a decision
about pgc_log_shows_accounting -- the two sit next to each other in
run_all_versions.sh. Neither change needs the other, but if you take the
reader change we will both be editing that file.

I will keep to pgc_reconcile_records and its message, and touch nothing in
pgc_log_shows_accounting, so whichever of us lands second gets an adjacent-hunk
rebase rather than a conflict. If you would rather have both in one change,
say so and this is yours -- it is a smaller change than #1243 and it would not
be worth splitting.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MpajdQbkVJ9ey1XyYHcikP

[OffgridwithJD]

Correction: the output I quoted in the description is the wrong branch.

The issue shows

  126 check(s) reported an outcome the count never saw:
    a check ran in a subshell, so its counter bump died with it while its

which is the _records > _stated branch. With records=0 and stated=126,
0 is not greater than 126, so it cannot be the one that ran. Driving the real
function on a log of that shape gives the other branch:

    records=0 but the log states checks run: 126
      126 check(s) were counted without emitting a record:
      something bumped PGC_CHECKS without going through pgc_record.

I reconstructed the quote from memory of the sweep instead of re-running it,
which is the whole failure mode this repo's house rules exist to stop, and I did
it in the issue that reports a message stating a cause it has not measured.

The finding is unaffected and if anything is sharper. The message that
actually fires names a cause internal to the shell harness -- an assignment to
PGC_CHECKS -- for a log the harness did not produce. Both branches are
impossible on this input; the one that runs is the second.

The fix under way keeps the non-zero verdict and changes only the words, because
a positive count with no records is reachable inside this harness, so the
function cannot claim to know which of the two it is looking at.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MpajdQbkVJ9ey1XyYHcikP