Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -1259,6 +1259,29 @@ true until the next version shipped.
now re-records. It names the two cases that remain: a declaration that no longer
resolves, and the implicit base projection, which is not readable by name at all.

- `pgc_ledger.py gate` no longer certifies a census that contradicts its own
ledger (#952).

The gate printed `ledger census: rows=N` and never compared that number to the
`checks_never_observed_red` the budget states, so it returned 0 on a twenty-row
ledger claiming five. Reporting is not enforcing. The comparison existed one layer
out, in a selftest arm, which runs on a pull request and therefore reports the
disagreement after the merge that creates it rather than before.

It creates it because the census is a measurement of the tree, so every merge
invalidates it: two pull requests each re-derive it from the same base, the merged
ledger takes both sets of rows, and the budget keeps whichever side won the
conflict. Three in flight at once set 769, 762 and 800 against a base of 756, and
no two composed. The new refusal is decidable from the two inputs alone, needing no
prior and no `--against`, which is what lets it speak about a merge commit.

It refuses a contradiction in either direction and does not make the census a
ceiling. A ceiling refuses a rise, and bounding this number deadlocks: every added
check enters as `never`, so landing one would mean raising a number the design says
may only fall. A budget that states no census at all is reported rather than
refused, since absence is not a contradiction; what holds the committed budget to
naming both numbers is a separate arm in each harness.

## [1.0-alpha3] - 2026-09-02

### Added
Expand Down
10 changes: 10 additions & 0 deletions test/check_ledger.tsv
Original file line number Diff line number Diff line change
Expand Up @@ -619,7 +619,11 @@ harness_selftest 400-a-check-result-must-be-machine the same timing check, ENABL
harness_selftest 400-a-check-result-must-be-machine the sweep finds a check inside a PIPED loop never -
harness_selftest 410-a-check-must-have-been-red --mutation across two failing checks in one run is refused never -
harness_selftest 410-a-check-must-have-been-red a before-log and an after-log together are refused, not silently empty never -
harness_selftest 410-a-check-must-have-been-red a budget stating no census is not refused, because absence is not a contradiction never -
harness_selftest 410-a-check-must-have-been-red a budget that does not exist at the prior is a note, not a refusal never -
harness_selftest 410-a-check-must-have-been-red a census that matches the ledger passes never -
harness_selftest 410-a-check-must-have-been-red a census that overstates the ledger is refused too, because this is not a ceiling never -
harness_selftest 410-a-check-must-have-been-red a census that understates the ledger is refused never -
harness_selftest 410-a-check-must-have-been-red a check in an UNCOVERED suite is not refused never -
harness_selftest 410-a-check-must-have-been-red a check merely added is not reported as a rename never -
harness_selftest 410-a-check-must-have-been-red a check observed red gains the date it was seen never -
Expand Down Expand Up @@ -675,19 +679,23 @@ harness_selftest 410-a-check-must-have-been-red and one that stayed green keeps
harness_selftest 410-a-check-must-have-been-red and only when there is a base, so a push build does not fail on it never -
harness_selftest 410-a-check-must-have-been-red and records neither as ever having been red never -
harness_selftest 410-a-check-must-have-been-red and the addition in the other part is not called a rename never -
harness_selftest 410-a-check-must-have-been-red and the census line it always printed is still there never -
harness_selftest 410-a-check-must-have-been-red and the comparison passes when the ceiling did not rise never -
harness_selftest 410-a-check-must-have-been-red and the gate prints that it compared them never -
harness_selftest 410-a-check-must-have-been-red and the history it is about to lose travels with it never -
harness_selftest 410-a-check-must-have-been-red and the message says how to fix it, because regenerating is the intended action never -
harness_selftest 410-a-check-must-have-been-red and the refusal names both values never -
harness_selftest 410-a-check-must-have-been-red and the refusal names how many failed, so the author can narrow the run never -
harness_selftest 410-a-check-must-have-been-red and the refusal names the check that reddened never -
harness_selftest 410-a-check-must-have-been-red and the refusal names the raise never -
harness_selftest 410-a-check-must-have-been-red and the refusal quotes the claim beside the measurement never -
harness_selftest 410-a-check-must-have-been-red and the runner names no remote at that call site never -
harness_selftest 410-a-check-must-have-been-red auto refuses when GITHUB_BASE_REF names a ref that is not here never -
harness_selftest 410-a-check-must-have-been-red auto uses the base ref when it resolves, and names it never -
harness_selftest 410-a-check-must-have-been-red auto with no base ref and no upstream is an integrity failure never -
harness_selftest 410-a-check-must-have-been-red both failure arms fail the major never -
harness_selftest 410-a-check-must-have-been-red but that suite is counted as not covered, which is the debt never -
harness_selftest 410-a-check-must-have-been-red but the gate says so, so the skip is visible rather than silent never -
harness_selftest 410-a-check-must-have-been-red control: a well-formed log still merges never -
harness_selftest 410-a-check-must-have-been-red control: an all-PASS log still merges with no flag at all never -
harness_selftest 410-a-check-must-have-been-red control: the same log merges with a different reason, so the refusal above is --mutation-across-two-checks and not the log never -
Expand All @@ -701,9 +709,11 @@ harness_selftest 410-a-check-must-have-been-red once the suite is covered, a new
harness_selftest 410-a-check-must-have-been-red one --mutation cannot be attributed across several runs at once never -
harness_selftest 410-a-check-must-have-been-red premise: and it is a file git has never seen, which is the case that used to fail open never -
harness_selftest 410-a-check-must-have-been-red premise: and the nobudget branch does not, which is the bootstrap shape never -
harness_selftest 410-a-check-must-have-been-red premise: both are never, so this ledger's census is two never -
harness_selftest 410-a-check-must-have-been-red premise: the budget is a tracked file too never -
harness_selftest 410-a-check-must-have-been-red premise: the budget was restored byte-exact never -
harness_selftest 410-a-check-must-have-been-red premise: the check has history before the rename never -
harness_selftest 410-a-check-must-have-been-red premise: the fixture ledger holds two rows never -
harness_selftest 410-a-check-must-have-been-red premise: the fixture log names checks the real ledger already knows never -
harness_selftest 410-a-check-must-have-been-red premise: the gate's status block was found in the runner never -
harness_selftest 410-a-check-must-have-been-red premise: the hasbudget branch carries the budget never -
Expand Down
2 changes: 1 addition & 1 deletion test/check_ledger_budget.txt
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,4 @@ suites_not_covered 250
# Without that it is a hand-maintained count that drifts, which is the failure
# this repository has spent a day proving. It is not a ceiling; it is a
# measurement that must be true.
checks_never_observed_red 819
checks_never_observed_red 829
35 changes: 35 additions & 0 deletions test/pgc_ledger.py
Original file line number Diff line number Diff line change
Expand Up @@ -561,6 +561,41 @@ def cmd_gate(args):
print(f" ledger census: rows={len(rows)} | never observed red={never}, "
f"ever red={len(rows) - never}, new this run={len(unknown)}")

# THE CENSUS IS COMPARED, NOT ONLY PRINTED (#952). Reporting is not enforcing:
# the line above stated the true number while the budget claimed another, and
# rc was 0 on a fifteen-row lie.
#
# DECIDABLE FROM THE TWO INPUTS ALONE. It needs no prior and no `--against`,
# which is the whole point: the case it catches is a MERGE COMMIT, where two
# PRs each re-derived the census from the same base, the ledger then takes both
# sets of rows, and the budget keeps whichever side won the conflict. A check
# that needed the prior could not speak about the commit that creates the
# disagreement.
#
# STILL NOT A CEILING, and this does not make it one. A ceiling refuses a RISE,
# and bounding this number deadlocks -- every added check enters as `never`, so
# landing one would require raising a number the design says may only fall.
# That argument is in check_ledger_budget.txt and nothing here changes it. What
# is refused is a CONTRADICTION, in either direction, which is what "a
# measurement that must be true" means.
stated = budget.get("checks_never_observed_red")
if stated is None:
# Absence is not a contradiction, and a silent skip is not acceptable
# either, so it is said out loud. Measured reason for not refusing: every
# other gate fixture in both harnesses writes a budget stating only
# suites_not_covered. What holds the COMMITTED budget to naming both is a
# separate arm in each harness.
print(" the budget names no checks_never_observed_red, so nothing asserts "
"the census")
elif stated != never:
print(f" the budget states checks_never_observed_red {stated}, the ledger "
f"holds {never}: these describe the same file and disagree")
print(" re-derive it from a run on THIS tree. Arithmetic across merges "
"has been right by accident and is not evidence.")
rc = 1
else:
print(f" census stated {stated}, ledger holds {never}: they agree")

if not args.registered:
raise LedgerError(
"--registered is required: without the registered suite list the coverage "
Expand Down
23 changes: 23 additions & 0 deletions test/pytest/TESTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2159,6 +2159,29 @@ references in `.github/`, zero in the runner.
If they disagree, one was edited by hand. `suites_not_covered` is 250 of 251, so the
gate cannot refuse a new check in 250 suites — a real limit, counted rather than hidden,
which falls as suites are seeded.

### `test_the_gate_refuses_a_census_that_contradicts_its_own_ledger`

The arm above asserts the two committed files agree. This one asserts the **tool
refuses a pair that does not** — because the gate printed `ledger census: rows=N`
and never compared it to the budget's claim, returning 0 on a fifteen-row lie
(#952). Reporting is not enforcing.

It is decidable from the two inputs alone, with no prior and no `--against`, and
that is the point rather than an economy. The disagreement is created by a **merge**:
two PRs each re-derive the census from the same base, the ledger then takes both sets
of rows, and the budget keeps whichever side won the conflict. A check that needed the
prior could not speak about the commit that creates the problem. Three PRs in flight
at once set 769, 762 and 800 from a base of 756, and no two of them composed.

Refused in **both** directions, which is what separates it from a ceiling: a ceiling
refuses a rise, and bounding this number deadlocks, as `check_ledger_budget.txt`
argues. Absence of the field is reported rather than refused, because absence is not a
contradiction — and because every other gate fixture in both harnesses states only
`suites_not_covered`, so refusing there would redden about twenty arms testing
something else. What holds the committed budget to naming both numbers is the arm
above.

## 24. test_loop_coverage_premise.py: a loop that never ran asserted nothing

**Why this file exists.** `assert-inside-a-loop-over-zero-rows` in VACUITY_MODES.md 3.5
Expand Down
51 changes: 51 additions & 0 deletions test/pytest/test_mutation_ledger.py
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,57 @@ def test_the_committed_ledger_and_budget_agree(expect):
"and the ceiling matches the suites with no rows")


def test_the_gate_refuses_a_census_that_contradicts_its_own_ledger(tmp_path, expect):
"""#952. The gate PRINTED the census and never compared it, so rc=0 on a lie.

Reporting is not enforcing. The case this catches is a MERGE, not a PR: two PRs
each rewrote the census from the same base, so the composed tree keeps whichever
side won the conflict while the ledger takes both sets of rows. Decidable from the
two inputs alone -- no prior, no `--against` -- which is exactly why it still
refuses on a merge commit, where the prior is the thing in question.

Refused in BOTH directions. The census is not a ceiling and this does not make it
one: a ceiling refuses a rise, and bounding this number deadlocks, as the budget
file argues. What is refused here is a contradiction.
"""
ledger = _w(tmp_path, "l.tsv", "")
reg = _w(tmp_path, "reg", "demo\n")
log = _w(tmp_path, "g.log", GREEN)
_run("merge", "--ledger", ledger, "--date", "2026-09-10", log)

# THE PREMISE, asserted: the numbers below mean nothing unless the ledger really
# holds two rows and both are `never`.
rows = _rows(ledger)
expect.num(len(rows), 2, "premise: the merged ledger holds two rows")
expect.num(len([r for r in rows if r[3] == "never"]), 2,
"premise: both entered as never, so this ledger's census is 2")

ok = _w(tmp_path, "ok.txt", "suites_not_covered 0\nchecks_never_observed_red 2\n")
out, rc = _run("gate", "--ledger", ledger, "--budget", ok, "--registered", reg, log)
expect.num(rc, 0, "a census that matches the ledger passes")
expect.num(out.count("census stated 2, ledger holds 2"), 1,
"and the agreement is printed, so a reader sees it was compared")

for claim, why in (("1", "understates"), ("3", "overstates")):
b = _w(tmp_path, "b%s.txt" % claim,
"suites_not_covered 0\nchecks_never_observed_red %s\n" % claim)
out, rc = _run("gate", "--ledger", ledger, "--budget", b, "--registered", reg, log)
expect.num(rc, 1, "a census that %s the ledger is refused" % why)
expect.num(out.count("checks_never_observed_red %s, the ledger holds 2" % claim), 1,
"and the refusal quotes the claim and the measurement (%s)" % why)

# ABSENCE IS NOT A MISMATCH, and that is measured rather than preferred: every
# other gate fixture here and in selftest 410 writes a budget stating only
# `suites_not_covered`, so refusing on absence would redden about twenty arms
# testing something else. What stops the COMMITTED budget dropping the field is
# the separate arm asserting it names both numbers, in both harnesses.
none = _w(tmp_path, "none.txt", "suites_not_covered 0\n")
out, rc = _run("gate", "--ledger", ledger, "--budget", none, "--registered", reg, log)
expect.num(rc, 0, "a budget stating no census is not refused")
expect.num(out.count("names no checks_never_observed_red"), 1,
"but the gate says so, so the skip is visible rather than silent")


def test_a_log_that_does_not_parse_is_not_evidence(tmp_path, expect):
"""`len(f) >= 5` accepted four shapes the emitter cannot produce.

Expand Down
53 changes: 53 additions & 0 deletions test/selftest/410-a-check-must-have-been-red.sh
Original file line number Diff line number Diff line change
Expand Up @@ -328,6 +328,59 @@ check "the committed census matches the committed ledger" \
check "the budget names a ceiling and a census, and says which is which" \
"$(grep -cE '^(suites_not_covered|checks_never_observed_red) [0-9]+$' "$_budget")" "2"


# ---- #952: the census is printed AND compared -------------------------------
#
# `gate` printed `ledger census: rows=N` and never compared it to the budget's
# `checks_never_observed_red`. rc=0 on a twenty-row ledger claiming 5 was measured.
# Reporting is not enforcing.
#
# The case it catches is a MERGE, not a PR: two PRs each rewrite the census from the
# same base, so the composed tree keeps whichever side won the conflict while the
# ledger takes both sets of rows. So the comparison must need NO prior, and nothing
# in this block passes --against.
_cw="$PGC_WORKDIR/census952"; mkdir -p "$_cw"
printf 'demo\n' > "$_cw/reg"
: > "$_cw/led"
_led_run merge --ledger "$_cw/led" --date 2026-09-10 "$_lw/green.log" >/dev/null
check "premise: the fixture ledger holds two rows" \
"$(grep -c . "$_cw/led" || true)" "2"
check "premise: both are never, so this ledger's census is two" \
"$(awk -F'\t' '$4=="never"' "$_cw/led" | grep -c . || true)" "2"

printf 'suites_not_covered 0\nchecks_never_observed_red 2\n' > "$_cw/ok.txt"
check "a census that matches the ledger passes" \
"$(_led_rc gate --ledger "$_cw/led" --budget "$_cw/ok.txt" --registered "$_cw/reg" "$_lw/green.log")" "0"
check "and the gate prints that it compared them" \
"$(_led_run gate --ledger "$_cw/led" --budget "$_cw/ok.txt" --registered "$_cw/reg" "$_lw/green.log" \
| grep -c 'census stated 2, ledger holds 2')" "1"
check "and the census line it always printed is still there" \
"$(_led_run gate --ledger "$_cw/led" --budget "$_cw/ok.txt" --registered "$_cw/reg" "$_lw/green.log" \
| grep -c 'ledger census: rows=2')" "1"

printf 'suites_not_covered 0\nchecks_never_observed_red 1\n' > "$_cw/low.txt"
check "a census that understates the ledger is refused" \
"$(_led_rc gate --ledger "$_cw/led" --budget "$_cw/low.txt" --registered "$_cw/reg" "$_lw/green.log")" "1"
check "and the refusal quotes the claim beside the measurement" \
"$(_led_run gate --ledger "$_cw/led" --budget "$_cw/low.txt" --registered "$_cw/reg" "$_lw/green.log" \
| grep -c 'checks_never_observed_red 1, the ledger holds 2')" "1"

# NOT A CEILING: a ceiling refuses a rise, and bounding this number deadlocks. What
# is refused is a contradiction, so overstating is refused too.
printf 'suites_not_covered 0\nchecks_never_observed_red 3\n' > "$_cw/high.txt"
check "a census that overstates the ledger is refused too, because this is not a ceiling" \
"$(_led_rc gate --ledger "$_cw/led" --budget "$_cw/high.txt" --registered "$_cw/reg" "$_lw/green.log")" "1"

# Absence is not a contradiction. Every other gate fixture in this part states only
# suites_not_covered, so refusing here would redden arms testing other things; the
# committed budget is held to naming both by the arm above.
printf 'suites_not_covered 0\n' > "$_cw/absent.txt"
check "a budget stating no census is not refused, because absence is not a contradiction" \
"$(_led_rc gate --ledger "$_cw/led" --budget "$_cw/absent.txt" --registered "$_cw/reg" "$_lw/green.log")" "0"
check "but the gate says so, so the skip is visible rather than silent" \
"$(_led_run gate --ledger "$_cw/led" --budget "$_cw/absent.txt" --registered "$_cw/reg" "$_lw/green.log" \
| grep -c 'names no checks_never_observed_red')" "1"

# ---- the gate cannot refuse a check in a suite it has never seen -------------
#
# The suite restriction is the MEANING of suites_not_covered, not a softening of
Expand Down
Loading