test: pin the projection guard's property, not its caller count (#1077) - #1078
Conversation
`native_fetch_projection.sh` protected one convention -- "every column" is an
explicit flag, never an absent set -- with two arms that counted a string across
`columnar_reader.c` and compared it against a literal `1`. That asserts HOW MANY
honest callers exist, which is a fact about today's tree rather than a property
of the code.
It errs in BOTH directions, and the second is why this is a fix rather than a
widening. Four source states, the same two arms, verified by running them:
state tests guarded flags | OLD arm1 OLD arm2 | NEW
main 1 1 1 | PASS PASS | PASS
honest caller 2 2 2 | RED RED | PASS
unguarded fn 2 1 2 | RED PASS | RED
unguarded INLINE 2 1 1 | PASS PASS | RED
Row 2 is #1077's coalescing read: it takes the flag and tests it exactly as the
convention demands, and was failed for existing.
Row 4 is the defect the arms exist to catch, added inside the existing worker so
no new flag is declared -- and BOTH arms pass it. `arm2` counts the GUARDED form,
which is still 1, so a bare `bms_is_member(c, needed)` beside it is invisible to
the arm named "the column test consults that flag rather than a null set". It is
anti-correlated with its own name: a correct addition reddens it, a wrong one
does not.
Row 3 shows the one red the old arms do produce carries no information either.
`arm1` reddens there for the same reason it reddens on row 2 -- the count moved --
so it cannot distinguish a correct caller from a broken one.
The arms now pin the property against itself: every needed-set membership test
consults the flag, `guarded N of N`. Honest callers move both counts together; an
unguarded one moves only the total. Both numbers are inside the compared strings,
so the arm's message IS the reconciliation. Today the tree reads `guarded 1 of 1`.
Two premises sit beside it because `guarded 0 of 0` also satisfies equality: that
a membership test exists to guard, and that the flag is declared. Against an empty
file the pin passes and both premises redden.
A literal match cannot tell "written differently" from "written wrongly", and
both readings of a mismatch are live -- reversed operands, a renamed variable, a
`pgindent` wrap, or the positive form all redden a correct tree. Failing closed is
right, but the message now names both readings so the next reader does not hunt a
caller that does not exist. It is emitted BEFORE the check and only on mismatch,
so the check's NAME stays the ledger key.
Removal proof, against the real suite rather than extracted expressions:
mutation applied md5 b3cbbc36426a -> 06be33a15eab, counts 1 -> 2
suite run 14 passed + 1 failed + 0 unrunnable + 0 skipped = 15
the one red every needed-set membership test consults that flag rather
than a null set: got [guarded 1 of 2] want [guarded 2 of 2]
both premises stayed green, correctly
restored md5 b3cbbc36426a, counts 1 1, git diff empty
Five majors, own build and own `.so` each, on this exact tree:
PG15 15 PASS PG16 15 PASS PG17 15 PASS PG18 15 PASS PG19 15 PASS
NO LEDGER CHANGE, and that is checked rather than assumed. `native_fetch_projection`
has no rows in the ledger -- it is one of the 249 uncovered suites -- and `cmd_gate`
refuses only "a check the committed ledger has never seen, IN A SUITE THE ledger
covers". Merging the five logs would have SEEDED the suite: +15 rows, 1230 -> 1245,
not the +3 I expected. That is a different change and is not in this one. There are
no orphans to prune either; the two removed names were never in the ledger.
This is the third arm in this file repaired for counting a string across a whole
file. The `deltuples` comment 15 lines above records the first, fixed by scoping;
these two were left as whole-file counts and did the same thing again.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK
OffgridwithJD
left a comment
There was a problem hiding this comment.
APPROVE at ae56f72. 14/14 green, and I verified the thing this PR exists to do rather than the thing it says.
The acceptance test: does it unblock #1077
Ran these arms verbatim against the two real trees, not against extracted expressions:
main tests=1 guarded=1 flags=1 | premise1=yes premise2=yes | pin=PASS
#1077's real tree tests=2 guarded=2 flags=2 | premise1=yes premise2=yes | pin=PASS
So suites (PG 18) clears on #1077 without that PR touching columnar_reader.c, which is the whole point. And main anchors at guarded 1 of 1, so the false-positive budget in the body is a measurement.
The removal proof, on a real tree rather than a synthetic one
Planted the row-4 defect — a bare bms_is_member(c, needed) inside the existing worker, no new flag declared — into #1077's actual source:
#1077 unmodified t=2 g=2 f=2 pin=PASS
+ unguarded INLINE test (the defect) t=3 g=2 f=2 pin=RED
mutation asserted applied: +1 membership test, no new flag declared
And the blindness it replaces, on main, where the old arms are not already saturated:
main + the same defect t=2 g=1 f=1 OLD1=PASS OLD2=PASS pin=RED
Both old arms pass the defect on a clean tree. That is the finding, and it is why this is a fix and not a widening.
I reproduced the four-state table independently
Cell for cell, from main's source, before reading your version — including the vacuity case your premises exist for:
tests=0 guarded=0 flags=0 -> pin PASS (equality satisfied by nothing), premises RED
guarded 0 of 0 is the case that would have made this a decorative arm, and both premises catch it.
The note
Emitted before the check and only on mismatch, so the check label stays the ledger key — that is the detail that made it safe to reuse the earlier logs, and diffing the names against the file to prove it is the check I would have asked for.
It also says the thing I care about most: a literal match cannot distinguish written differently from written wrongly. I measured five correct rewrites that redden exactly as the defect does — reversed operands, extra spacing, a wrapped line, the positive form, a renamed variable — so both readings are live, and pgindent wraps on width. Without the note the next reader spends an afternoon hunting a caller that does not exist.
The ledger: correctly nothing
native_fetch_projection has zero rows in main, so it is one of the 249 uncovered suites and the gate cannot refuse its checks. Removing two check names orphans nothing, because neither was ever in the ledger. suites_not_covered stays 249.
You caught that by merging five logs, expecting +3 rows and getting +15, then restoring. That is worth more than the fix: it is the same trap five other PRs are sitting in, from the opposite side, and it is what produced the finding that seeding is opt-in and a partial seed is the only way to lose.
One thing for later, not for this PR
This is the third arm in this file repaired for counting a string across a whole file, and the deltuples comment 15 lines above records the first. Two were left after one was fixed. Worth a sweep of the other suites for grep -c ... "1" against a literal — the shape is mechanical to find and it fails in both directions wherever it appears.
Not a condition of this approval, and I would not hold a 19-line fix for it.
|
A sweep for the same shape elsewhere, since this PR's body says "the third arm in this file". On the numbers it is the third, fourth and fifth — and the file is not the only one. Not a condition of the approval above; #1078 is right as it stands. Recording it so the class is countable rather than anecdotal. Filtering Two of the twelve are this PR. Two more are in the same fileBoth are call-site counts rather than definitions, so both fail in the same two directions this PR documents: a second honest caller reddens them, and a call written with different arguments is invisible to them. They are the arms named "index deletion asks whether the row is live" and "the reconstruct caller asks only for uncovered columns" — and neither asserts that, it asserts there is exactly one such call today. The The rest of the class
Not all twelve are wrongA pinned count is right where the count is the property — exactly one definition, one entry point, one Happy to take these as a separate change if nobody else wants them. |
My own new arms tripped selftest/080: five sites piped a captured string into `grep -q`, which exits on its first match and closes the pipe under its writer. Caught by @jdatcmd on commandprompt#1081's CI, with the sites rather than a count. The answer was already in the file family I was working in -- native_fetch_projection.sh:155 uses a `case` over a captured variable. No subprocess, no pipe, and it reads better than the `&& echo yes || echo no` tail. Patterns are quoted so the brackets in valOffset[c][present] stay literal rather than becoming glob character classes. AND THE DISCARD ARMS GAINED THE PREMISE THEY WERE MISSING. As written they could not tell "this function does not discard the cache" from "the awk range matched nothing" -- the same both-readings problem I raised against commandprompt#1078's literal match, in my own arm one day later. Each function's extraction is now captured, asserted non-empty, and then matched. Re-verified rather than assumed, because the arms changed: selftest/080 no suite pipes a captured string into an early-exit reader PASS suites native_fetch_cache 26 passed, native_fetch_position 8 passed matrix identical verdicts to the pre-fix run, every restore md5-checked five majors PG15 PG16 PG17 PG18 PG19, both suites, PASSED The pre-existing `| sort -n | head -1` sites in the same file are correctly NOT flagged: `sort` reads all its input before writing, so `head` closing early reaches `sort` rather than the producer. The detector discriminates rather than banning pipes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012RSw4qMHS7ByE7PY8Ns4cs
#1078 repaired two arms in native_fetch_projection.sh that compared a whole-file `grep -c` against a literal. A sweep of test/ found TWELVE sites of that shape. These are six of the remaining ten. THE REPAIR DIFFERS PER ARM BECAUSE THE FAILURE DIRECTION DOES, which is the part worth reading: the entry key both directions -> self-referential, keyed N of N the cid reject noise only -> scoped to the function that must hold it the geometry blind to 3 of 4 -> membership over all four compared fields the discard proxy for "where" -> the two functions named rank, valOffset noise only -> scoped to pgcolumnar_fetch_row TWO OF THE EIGHT I WAS ASSIGNED ARE CORRECT AND ARE LEFT ALONE. A pinned count is right where the count IS the property: `^#define COLUMNAR_DECODE_INTERRUPT(i)` appearing twice would be a redefinition, and native_saop_pushdown's premise is load-bearing for an `awk` range that would silently concatenate two expressions into one `guard` string. Classifying them took longer than fixing the six. MEASURED. Every mutation compiles, so the suite rebuilds and runs end to end -- the harness refuses a source/.so mismatch, correctly, and there is no shortcut: case OLD arms NEW arms unkeyed group lookup key=1 PASS RED keyed 1 of 2 second keyed lookup key=2 RED 24 passed rowCount dropped geom=1 PASS RED rowCount executor-end discard gone discard=1 RED RED names the function third discard call discard=3 RED 24 passed rank replaced by a walk rank=0 RED RED rank prefix The two OLD-PASS rows are the case for the change: an unkeyed lookup and a dropped geometry field both leave the old arms green. The two OLD-RED-NEW-PASS rows are what fired on #1077 and cost a correct PR a red. MY FIRST MUTATION MATRIX WAS WRONG AND I ALMOST SHIPPED IT. It reported the second-keyed-lookup case reddening an unrelated `natts` arm. Run alone that case is 24/24. The harness asserted each mutation APPLIED and never asserted it was RESTORED, so one case was measuring two mutations. Re-run with an md5 restore assertion per case, and the contamination is gone. A mutation harness that does not check its own restore produces exactly the false finding I would have filed. FIVE MAJORS, both suites, own `make clean` each: PG15 PG16 PG17 PG18 PG19 native_fetch_cache PASSED, native_fetch_position PASSED No ledger change. `native_fetch_cache` and `native_fetch_position` have zero rows, so they are two of the 249 uncovered suites and no check name here is a ledger key. Checked, not inherited from #1078. Four of the twelve remain, in native_fetch_cache (0 left), decode_interrupts (1, correct), native_saop_pushdown (1, correct) and the two #1078 already fixed -- leaving native_fetch_cache's siblings done and nothing outstanding in these two files. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012RSw4qMHS7ByE7PY8Ns4cs
…-not-the-caller-count Pin six more guards to their property, not their caller count (#1078)
CHANGELOG.md only. #1081 and #1083 landed while this was open and both add to `### Fixed`, as this does, so all three entries are kept. Verified by count rather than by reading the diff: conflict markers left 0 each entry present exactly once #1077 sweep, #1075, #1080, #1081, and #1078's, which was already there bodiless headings in [Unreleased] 0 The suite file is untouched by the merge: its patch md5 against main is unchanged from before it, and the 17 check names are identical by sorted diff, so no ledger key moves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK
native_fetch_projection.shprotected one convention -- "every column" is anexplicit flag, never an absent set -- with two arms that counted a string across
columnar_reader.cand compared it against a literal1. That asserts HOW MANYhonest callers exist, which is a fact about today's tree rather than a property of
the code.
It errs in both directions, and the second one is why this is a fix rather than
a widening. Four source states, the same two arms:
Row 2 is #1077's coalescing read. It takes the flag and tests it exactly as the
convention demands, and was failed for existing.
Row 4 is the finding. Add a bare
bms_is_member(c, needed)inside the existingworker, so no new
bool allColumnsis declared, and BOTH arms pass.arm2countsthe GUARDED form, which is still 1, so an unguarded test beside it is invisible to
the arm named "the column test consults that flag rather than a null set". The
arm is anti-correlated with its own name: a correct addition reddens it, a wrong
one does not.
Row 3 shows the one red the old arms do produce carries no information either.
arm1reddens there for the same reason it reddens on row 2 -- the count moved --so it cannot tell a correct caller from a broken one.
The change
The arms pin the property against itself: every needed-set membership test
consults the flag,
guarded N of N. Honest callers move both counts together; anunguarded one moves only the total. Both numbers sit inside the compared strings,
so the arm's message is the reconciliation rather than a bare verdict.
Anchored, not asserted: on this tree the pin reads
guarded 1 of 1.Two premises sit beside it because
guarded 0 of 0also satisfies equality.Against an empty file:
A literal match cannot tell "written differently" from "written wrongly", and both
readings are live: reversed operands, a renamed variable, a
pgindentwrap, or thepositive form all redden a correct tree. Failing closed is the right direction, so
the message now names both readings instead of one. It is emitted BEFORE the check
and only on mismatch, so the check's NAME stays the ledger key.
Removal proof, against the real suite
The note printed above the red, and printed nothing on the clean tree.
Five majors, this exact tree
Own
make clean, own build and own.soper major:Each log carries exactly its own major. The suite's check names diffed against the
file: 14 of 15 identical, the one difference being
the wide fixture has $((NCOLS + 1)) columnsexpanding to 41, which is the log doing its job.No ledger change, and that is checked rather than assumed
native_fetch_projectionhas no rows in the ledger -- it is one of the 249uncovered suites -- and
cmd_gaterefuses only "a check the committed ledger hasnever seen, IN A SUITE THE ledger covers".
I found that by merging the five logs expecting +3 rows and getting +15, 1230
to 1245, then restoring. Merging would have SEEDED the suite, which is a different
change and not this one. There are no orphans to prune either: the two removed
names were never in the ledger.
Not in this PR
Seeding
native_fetch_projectionfrom the five logs would dropsuites_not_covered249 to 248. It is a clean standalone change; putting it herewould make a 19-line test fix carry a 15-row ledger diff a reviewer has to check
separately. The logs are kept.
Unblocks #1077, whose change is correct and was failed by this guard.
🤖 Generated with Claude Code
https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK