fix: do not store a validity bitmap for a chunk that holds no null (#1130)

[jdatcmd]

Closes #1130.

A column chunk's page was always [validity bitmap][encoded values]. The bitmap is one bit per row and is written RAW, ahead of the block codec, which therefore never saw it — so a NOT NULL column, or one that simply holds no nulls, still stored ceil(rows / 8) bytes of 0xFF for ever.

The writer now omits it for a chunk that holds no null, and records that in the encoding descriptor's new flags byte.

Measured

ClickBench hits_0.parquet — 1,000,000 rows, 105 columns, no null in any of them — through pgcolumnar.import_parquet on PG17. Both arms loaded into the same cluster on the same day, rather than one of them being a number from an earlier session:

	before (21aa465)	after
stored chunk bytes	78,109,810	64,984,810	−16.80%
validity bytes	13,125,000	0
relation size	78,381,056	65,216,512
chunks with no bitmap	0 of 735	735 of 735

The saving is exactly the bitmap: −13,125,000 bytes = 105 columns × 125,000. Nothing else moved.

Decided from the rows written, not from the attribute's NOT NULL flag. A nullable column whose rows happen to be complete gets the saving too, and a constraint added later cannot make an already-written chunk lie.

The format moves: descriptor version 2 → 3

Version 3 spends the byte version 2 wrote as a zero reserved byte. Every field keeps its offset, so a version-2 descriptor is a version-3 one whose flags are clear: readers accept 2 and 3, writers emit 3, and tables written by an older build keep reading with no conversion.

Downgrading a binary below the one that wrote the table is not supported, and the shape of that failure is measured rather than asserted. An alpha4 binary reading a table this build wrote:

path	result
sequential scan	ERROR: unrecognized native encoding descriptor, every time
index fetch, chunk narrower than the bitmap	ERROR: validity bitmap longer than the chunk
index fetch, chunk wider than the bitmap	40 single-row fetches: 24 errored, 16 returned NULL for a row that holds a value

The last row is why #1137 exists: the old reader tests a bit in bytes that are not a bitmap before it reaches any version check, and pgcolumnar.storage.format_version — the stamp that runs early on both paths and could refuse the table outright — is checked for equality, so bumping it would also make this build refuse every table alpha4 wrote. That is a decision about the versioning model rather than about this change, so it is filed rather than smuggled in here.

Three readers, not one, and they are correlated

The first implementation taught only the sequential scan that the bitmap's size is now a property of the CHUNK rather than of the row group. The PG17 matrix went red in 33 suites, with native_index's point lookup returning NO ROW for a row that is there.

The two paths are not independent: pgcolumnar_fetch_coalesce_read skips any chunk whose page_length is below the group's bitmap size, and eliding the bitmap is exactly what takes a well-encoded chunk below it. So the chunks this change helps most are the ones that fall to the per-column fetch path — a fixture built to exercise the elision is systematically the fixture that lands on the reader most likely to have been left behind. All three readers now call one helper, which is also what keeps the coalesced read and the per-column loop agreeing about a pointer and a length that one of them computed.

A hardening gap the change's own comments forced into the open

The scan's fast path for fixed-width by-value types (columnar_reader.c, the #289 inline) read a value without checking it against the end of the stream — it trusted the validity bitmap to stop first. A synthesized all-ones bitmap is a new way to reach past it, so that path now carries the bound the general path has always had. Two guards refuse the catalog that would get there: a descriptor claiming NO_VALIDITY while accounting for fewer values than the group has rows, and a row count whose bitmap would not fit in memory.

The bound costs nothing measurable. Backend instructions for sum over a 1,000,000-row bigint column, cpu_core/instructions/ pinned to the P-cores, three repetitions per backend, two backends per arm:

arm	run 1	run 2
with the bound	6,819,185,307	6,811,086,223
without it	7,408,953,743	7,394,084,968

The bounded build is 7.9% lower, reproduced across two build directories with the arms interleaved. It is not claimed as a saving, and the direction is unexplained. Everything with a mechanism has since been excluded by measurement, not by argument (see the comments below):

hypothesis	status
the check costs instructions	excluded — the bounded build is lower
a different plan	excluded — same node, same actual rows=1000000, same chunk groups and vectors decoded
a different fixture	excluded — same rows, same checksum
more iterations somewhere	excluded — branches equal within 0.2% (1,679,418,138 against 1,675,963,100)
a bigger or smaller compiled function	excluded — 97 instructions with the bound, 88 without

Equal branches with 590M more instructions retired means equal control flow and a different instruction mix in straight-line code. What the measurement supports, and all it supports, is that the bound is not a cost.

One attractive bound is wrong, and this branch is the proof. Bounding the synthesized bitmap by the row group's own byteLength — a stored bitmap cannot exceed the bytes the group occupies — was implemented, and it refused a correct table: ve_full's group is 360 bytes on disk and needs 12,500 bytes of synthesized bits. That gap is the saving, not a defect. Reverted, and the reason is recorded where the next person will look for it.

Tests

test/validity_elision.sh and test/pytest/test_validity_elision.py, 25 checks each, green on PG15/16/17/18/19. compare_to_bash.py grades the pair missing: 0.

The 25th arrived from @OffgridwithJD's review and is the one worth reading first: the per-chunk arm expects 0, and 0 is also what the residual returns when it sums nothing. See the review-fix comment below for the mutation that proves it, and for what a shared variable does and does not buy over a shared literal.

Five of the 24 are premises, and each answers a way the headline arm reads 0 without measuring anything: coalesce(sum(...), 0) returns 0 over an empty set (so the chunk count is asserted beside it); ceil(rows/8) is one group's bitmap (so the group count is asserted); and the subtraction is only the bitmap while the block codec is off (so the setting is read back rather than assumed).

One arm exists because both size arms are satisfied by a per-ROW-GROUP decision — one fixture's group holds no null anywhere, the other's holds some. The null-bearing fixture carries a null-free key column in the same row group, so only a per-chunk decision elides one and keeps the other.

Three mutations, each asserted to have applied (the .so md5 moved) and to have been restored (the control run reproduced the original md5 byte for byte):

mutation	arms reddened
writer: presentCount == rowCount forced false	1 — the size arm
reader: PgColumnarEncdescOmitsValidity returns false	10
fetch path: group-wide size instead of the chunk's own	exactly the 3 fetch arms

Two arrangements in those suites are load-bearing, and both were found by an arm that failed rather than by reasoning:

• the table carries a KEY column beside the measured one, because an index on the only column is answered by an Index Only Scan that never calls the table AM at all — the first version of the fetch arms passed against a build whose fetch path was provably broken;
• pgcolumnar.enable_custom_scan = off, because enable_seqscan does not govern the columnar custom scan and the plan was otherwise Custom Scan (PgColumnarScan), a scan wearing a fetch's name.

The premise arm asserts the PLAN NODE rather than the row, for the reason @OffgridwithJD put well in review of the mechanism: a row that comes back says nothing about which reader produced it.

Derived artefacts, each re-derived on the rebased tree

• check_ledger.tsv: 14 rows, seeded from five real runs merged in ONE call, so each carries 15;16;17;18;19 and the short-major warning stayed silent. Eleven of the fourteen carry an observed red from the mutations above.
• check_ledger_budget.txt: checks_never_observed_red 1407 → 1410, re-derived by counting. suites_not_covered does NOT move — registering the suite takes registered 260 → 261 and seeding it takes covered 11 → 12, which is why the seeding is in this change rather than after it.
• expected_tests.txt: cluster_tests 423 → 426 by collection; guard_tests re-derived in the same run and did not move (380).

Neighbours that read the descriptor

encode_post_codec (both halves) subtracted ceil(rows/8) from page_length unconditionally to get the value stream. With the bitmap gone that removes bytes that were never written, so both now subtract only where the flag says there is one. native_encdesc_golden pins the version byte at 3 and adds a per-column arm on the flags byte — per column, because the flag is a property of the chunk. Four other descriptor readers had comments calling byte 1 "a reserved byte"; they are corrected. Entry field offsets do not move, which is why native_dict_underfill's byte-offset reads at 6 and 11..14 are untouched and passed throughout.

The one open decision, filed rather than smuggled in

#1137: pgcolumnar.storage.format_version is the stamp that runs early on both read paths and could refuse a future descriptor outright, which is what would have made the downgrade case loud everywhere. It is checked for equality, so bumping it would also make this build refuse every table alpha4 wrote — the fix is a range check plus a bump, and it changes the on-disk format's identity (PGCN v1) across the spec and the user docs. That is a decision about the versioning model, not about eliding a bitmap, so it is filed with the measurement rather than taken here.

#1139: alpha5's named item, cascading, still has no measurement that favours it. Filed with the gate the plan itself states, what has changed under it since (the post-codec arbiter from #1132, and descriptor v3 being spent here so a chain entry is v4), and the measurement still owed.

---

Verification, so a reviewer can start from what is already done rather than repeat it.

• Full PG17 matrix on this exact commit: ALL VERSIONS PASSED, 256 of 261 suites ran, 5 skipped, 0 incomplete. The ledger gate in the same run: census stated 1420, ledger holds 1420: they agree and coverage: registered=261 | covered=12, not covered=249, ceiling=249.
• The suite ran green on PG15, PG16, PG17, PG18 and PG19 — 25 checks each, five separate runs, which is also where the ledger rows came from.
• Three consecutive green matrices, one per review round: 5b5e590, 3351586, 73cc36a.
• Both harnesses run clean under shellcheck -S error and compare_to_bash.py (missing: 0).

🤖 Generated with Claude Code

https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK

[OffgridwithJD]

Reviewed against your three questions. The C is careful and the format work is documented
properly; I went after the tests, since that is what you pointed at.

One blocker, and it is in the arm your own comment calls the only one that proves the
headline property. It is one line per harness.

The per-chunk arm passes on a population that does not exist

a null-free column elides its bitmap beside a null-bearing one in the same row group is,
as the comment above it says, the only arm that distinguishes a per-CHUNK decision from a
per-ROW-GROUP one. It compares nulls_key_res against 0 — and 0 is also what
coalesce(sum(...), 0) returns over an empty set.

You anticipated exactly this and wrote the premise for it:

check "premise: the residual was summed over chunks that exist"    <- guards full_res

but full_res is the one residual that needed it least, because a bug there tends to
produce a large number. The two residuals with no population premise are nulls_res,
which is compared against 12500 and so protects itself, and nulls_key_res, which is
compared against 0 and does not.

Measured, not argued. I pointed that one residual at a column index that cannot exist and
changed nothing else:

CONTROL   validity_residual ve_nulls 0    key_residual=0   PASS   24 passed + 0 failed
MUTATED   validity_residual ve_nulls 99   key_residual=0   PASS   24 passed + 0 failed

The suite is green against a column index that does not exist. The arm cannot tell "the
bitmap was elided" from "I measured no chunks at all".

The pytest twin has it identically, which is the part I would not have predicted —
the helper states the contract and the call sites do not keep it:

    THE COUNT IS RETURNED BESIDE THE SUM because a sum over nothing is 0, which
    is exactly what the headline arm wants to see. A residual of 0 is evidence
    only together with the number of chunks it was summed over.

full_res, full_chunks = _residual(c, "vb_full")
nulls_res,  _         = _residual(c, "vb_nulls")
key_res,    _         = _residual(c, "vb_nulls", KEY_COLUMN)   # <- the 0 comparison

Same mutation, KEY_COLUMN = 0 -> 99:

CONTROL   24 checks, 4 passed
MUTATED   24 checks, 4 passed

The fix is the premise you already wrote, applied to the third call: chunks_measured ve_nulls 0 in the shell suite, and keeping the count instead of _ in the twin. I would
put it beside the arm rather than with the other premises, so the two move together.

Your question 2: I did not find a variant of the reverted mistake

The reverted idea was bounding a SYNTHESIZED all-ones bitmap by the row group's stored
byteLength, which refuses a correct table precisely because elision makes the group
smaller than its bitmap. Every surviving bound I can find compares a stored bitmap
against pageLength, which is the right bound and cannot have that failure:

columnar_reader.c   cc->pageLength < (uint64) cvb                  (coalesced fetch)
                    cc->pageLength - cvb > PG_UINT32_MAX
                    cc->pageLength >= (uint64) cvb

and for an elided chunk cvb is 0, so those are vacuously satisfied rather than
wrongly restrictive. The synthesized path is bounded by MaxAllocSize and by the
accounted != rg->rowCount equality — neither of which is a stored size. Your comment
at the need > MaxAllocSize site records the reasoning; I would keep that comment
exactly as it is, because it is the thing that stops the next person re-deriving the
same wrong bound.

Your question 3: instruction count rules out the usual suspect

The usual explanation for a few percent moving under a trivial code change on this box is
code layout — alignment shifting a hot loop across a boundary, link order, even the size
of the environment block. That explanation is not available to you, and that is the
useful part: instructions retired is insensitive to layout. Alignment changes cycles,
not the number of instructions executed. A 7.9% move in instruction count means the
machine really is executing fewer instructions, so the difference is semantic and should
be visible rather than mysterious.

Two places it can come from, and they are distinguishable:

1. Codegen. An explicit bound can let the optimiser prove a range and drop later
   redundant checks, or unlock a vectorised form of the loop. This is static: objdump -d
   the one function in both builds and diff the instruction counts of the hot loop. If the
   loop body shrank, you have your mechanism and it is a real saving.
2. A different dynamic path. The check short-circuits work that the unchecked build
   performs. This shows as a different call count, not a different loop body: perf stat
   the two arms for branches and branch-misses alongside instructions, and count calls
   to the functions below it.

If (1), "the check is not a cost" understates it and you can say why. If neither, I would
keep your current wording — reporting the direction without a mechanism is the honest
form, and you were right not to claim a saving.

To answer what you actually asked: I have seen unexplained swings of this size on this
box, but on the CLOCK, not on instruction count — a byte-identical dsm round-trip that
measured 26% slower. I have not seen instruction count move without a mechanism, and I
would not expect to.

Everything else I checked and found sound

sqlstate()        pins XX001 (ERRCODE_DATA_CORRUPTED) rather than grepping ERROR,
                  so a FATAL, a 42501 or a missing function cannot satisfy it
guard 1 fixture   ve_nulls' v column holds 90% of rows, so setting NO_VALIDITY on it
                  is the exact lie the guard names
guard 2           poisons row_count rather than the descriptor, so it exercises the
                  refuse-before-allocate ordering the comment claims
premise: codec    read back from SHOW rather than assumed from ALTER DATABASE
premise: groups   asserted, so ceil(rows/8) really is the whole expected bitmap

One small thing, no action needed unless it is free: groups="$(row_groups ve_full)$(row_groups ve_nulls)"
compared against "11" concatenates two counts with no separator, so 1+1 and
11+"" are the same string. It cannot bite at this fixture size, but "$a|$b" against
"1|1" costs nothing and says what it counted.

CI is green on 12 of 14 with the two suites jobs still running. Happy to re-review as
soon as the population premise is on that third residual — and I will run the same
mutation against it, which is the check I would want against my own fix.

[jdatcmd]

Review fix, with the mutation that proves it

@OffgridwithJD found the one arm in this branch that could not fail, and it was the most important one: a null-free column elides its bitmap beside a null-bearing one in the same row group is the only arm that distinguishes a per-CHUNK decision from a per-ROW-GROUP one, and it compared a residual against 0 — which is also what coalesce(sum(...), 0) returns when it sums nothing. Pointing that residual at column_index 99 left all 24 checks green in both harnesses.

Fixed by giving that arm its own premise, beside it rather than with the other five so the two move together:

check "premise: the per-chunk arm's residual was summed over chunks that exist"

The removal proof is the reviewer's own mutation, run here. validity_residual ve_nulls 99 + chunks_measured ve_nulls 99 in the shell, KEY_COLUMN = 99 in the twin:

harness	under the mutation	restored
validity_elision.sh	24 passed + 1 failed — the new premise, and nothing else	25 passed + 0 failed
test_validity_elision.py	23 pass + 1 fail	25 pass + 0 fail

The reusable distinction, which is now in TESTS.md: _residual returns the chunk count and two of its three call sites still discard it — deliberately, because those two residuals are ones a bug makes LARGE. A premise is owed wherever the failure mode and the pass look the same.

Ledger re-seeded from five fresh runs merged in one call, so all 25 rows carry 15;16;17;18;19. Census 1420 → 1421, re-derived by counting. cluster_tests 426 → 427 by collection.

The instruction-count paragraph is weakened, not defended

The objdump answer is in: pgcolumnar_row_read_column compiles to 97 instructions with the bound and 88 without, and nothing else in the object moved. So the bounded build's code is bigger, and a gap of 197 instructions per row is larger than the entire function — a local codegen effect cannot produce it.

My own instrument is now the first suspect: it pins the .so by md5 per arm and never asserted the plan or the row count. The CHANGELOG paragraph has been rewritten to say the number bounds the check's cost from above and nothing else. The settling run — EXPLAIN (ANALYZE, VERBOSE) per arm, branches/branch-misses beside instructions, and the actual rows and a checksum per arm — will be posted here either way. If it turns out the arms processed different work, the paragraph comes out.

Composing with #1136, measured rather than predicted

I composed this branch with pull/1136/head in a worktree. The loud half is safe and the quiet half is not:

	files
CONFLICT (a human must look)	test/pytest/TESTS.md, test/pytest/expected_tests.txt, test/pytest/test_compare_to_bash.py
auto-merges SILENTLY	test/check_ledger.tsv, test/check_ledger_budget.txt, test/run_all_versions.sh, CHANGELOG.md

Derived on the composed tree so whoever merges second does not have to:

cluster_tests    430      (this branch says 427, #1136 says 426; neither is right after the merge)
guard_tests      380      unchanged
ledger rows     1443
never observed  1421      -- unchanged from this branch: #1136 adds no ledger rows
registered       261 | covered 12 | not covered 249

Both PRs also claim TESTS.md section 57, which is one of the conflicts. The second one to merge takes 58.

[jdatcmd]

Full PG17 matrix on 5b5e590, the commit carrying the review fix: ALL VERSIONS PASSED, 256 of 261 suites ran, 5 skipped, 0 incomplete, MATRIX_RC=0. The ledger gate in the same run: census stated 1421, ledger holds 1421: they agree and coverage: registered=261 | covered=12, not covered=249, ceiling=249.

[jdatcmd]

Correction to my compose comment above. I listed check_ledger.tsv, check_ledger_budget.txt and run_all_versions.sh under "auto-merges silently" in a way that reads as live exposure between this branch and #1136. It is not. @OffgridwithJD checked what #1136 actually touches, and I verified it rather than taking it:

$ git diff --name-only 17b2c4d pr1136-check
CHANGELOG.md
test/pytest/TESTS.md
test/pytest/expected_tests.txt
test/pytest/test_compare_to_bash.py
test/pytest/test_index_fetch_penalty_width.py
test/pytest/test_native_index_fetch_stripe_cost.py
test/pytest/test_scan_decode_cost.py

#1136 is pytest-only. Those three files merge quietly because only one side writes them, which is not the failure mode — that needs two sides each writing a plausible value. The census key is this branch's, uncontested.

What survives, and is live: cluster_tests on the composed tree is 430, derived by collection, while this branch states 427 and #1136 states 426. It sits behind a conflict, so nobody can merge it blind; the number is here so whoever merges second does not have to derive it under time pressure. Both PRs also claim TESTS.md section 57.

Also pushed, 3351586: the per-chunk premise and the arm it guards now take their column from one variable rather than from the same literal written twice. @OffgridwithJD measured the third cell I had not run — move the residual to a column that does not exist and leave the premise pointing at column 0, and the suite goes green over nothing again. Three cells on the pushed tree:

cell	result
A control	25 passed + 0 failed
B VE_KEY_COL=99, both follow it	24 + 1 failed, the premise, nothing else
C residual-only move	not expressible — the column appears once, so any edit that moves the residual moves the premise with it

The pytest twin never had cell C: one call there returns the residual and its chunk count together. The shell now says the same thing.

[jdatcmd]

The instruction gap: three hypotheses excluded, mechanism still unknown

Per-arm capture, both arms on the same tree and the same build directory, differing only in the five-line bound:

                      bound                         nobound
plan node             Custom Scan (PgColumnarScan)  Custom Scan (PgColumnarScan)
actual rows           1000000                       1000000
chunk groups read     7                             7
vectors decoded       100                           100
vector decodes        100                           100
checksum              6916307057775100222333653     6916307057775100222333653
instructions          6,807,146,061                 7,393,275,259
                      6,799,740,237                 7,393,734,871

So:

• The plan hypothesis is dead. Same node, same counters. My own instrument's missing premise turned out not to be hiding anything, though it was still right to name it before running.
• The fixture hypothesis is dead (@OffgridwithJD's third candidate, which I would not have thought to exclude). Both arms processed 1,000,000 rows and returned the same 25-digit checksum, so it is the same work on the same bytes.
• The codegen hypothesis is dead. pgcolumnar_row_read_column is 97 instructions with the bound and 88 without — bigger with it — and no other symbol in the object changes shape. A gap of 197 instructions per row is larger than the whole function.

What remains is a reproducible 7.9% difference in instructions retired between two builds that execute the same plan over the same rows and produce the same answer, with the bounded build lower. I have no mechanism for it.

The CHANGELOG says exactly that — the number bounds the check's cost from above and nothing else. Branch counts beside instructions are running now; if they do not separate the arms either, the honest end state is a measured fact with no explanation, recorded as such rather than dressed up as a saving.

[jdatcmd]

Correction to my own claim about cell C, two comments up. I wrote that after VE_KEY_COL the residual-only move is "not expressible". That is too strong, and @OffgridwithJD demonstrated it rather than argued it — they replaced the variable with a literal in one of the two calls:

nulls_key_res="$(validity_residual ve_nulls 99)"
nulls_key_chunks="$(chunks_measured ve_nulls $VE_KEY_COL)"

25 passed + 0 failed. Green over a population that does not exist.

What the variable actually buys is narrower and still worth having: the desync is no longer reachable by changing the column, which is the edit someone would really make. A deliberate replacement of the variable with a literal still reaches it, and nothing structural in shell closes that.

The pytest twin does close it, and for a reason worth naming because it is the transferable part: key_res, key_chunks = _residual(c, "vb_nulls", KEY_COLUMN) — one call cannot return two populations. That is a property of returning the measurement and its population together, not of naming a variable. The shell suite gets as close as shell gets; the twin gets it by construction.

The file will be read as precedent for the next suite that needs a population premise, so the claim in it should be the one the change earns.

[jdatcmd]

Q3, one level further: same control flow, different instruction mix

@OffgridwithJD asked for the branch count beside the instructions, on the grounds that a 197-instruction-per-row gap with equal branches would be the odd case. It is the odd case.

                 bound            nobound          delta
branches         1,679,418,138    1,675,963,100     +0.2%
instructions     6,817,514,459    7,409,086,680     +8.7%
cache-refs           3,145,126        3,121,518     +0.8%

Equal branches means equal control flow: the unbounded build is not executing more iterations of anything. It retires 590 million more instructions over the same number of branches, on the same plan, the same 1,000,000 rows, the same 7 chunk groups and 100 decoded vectors, returning the same checksum.

So the gap is the instruction mix in straight-line code, which is a compiler codegen difference rather than a difference in work. Every hypothesis with a mechanism is now excluded:

hypothesis	status
the check costs instructions	excluded - the bounded build is lower
a different plan	excluded - same node, same counters
a different fixture	excluded - same rows, same checksum
more iterations somewhere	excluded - branches equal within 0.2%
a bigger/smaller compiled function	excluded - 97 instructions with the bound, 88 without

I am stopping the investigation here and recording the result as what it is. The CHANGELOG (pushed, 73cc36a) states the branch figures and says the mechanism is unidentified. What the measurement supports, and all it supports, is that the bound is not a cost - which is the only question the change needed answered.

[jdatcmd]

Full PG17 matrix on 73cc36a, the current head: ALL VERSIONS PASSED, 256 of 261 suites ran, 5 skipped, 0 incomplete, MATRIX_RC=0, and the ledger gate in the same run reports census stated 1421, ledger holds 1421: they agree. That is the third consecutive green matrix on this branch, one per review round: 5b5e590 (the per-chunk premise), 3351586 (one variable for the column), 73cc36a (the branch-count figures in the CHANGELOG).

[OffgridwithJD]

Approving. The blocker is closed and I verified the fix by running the mutation myself
rather than reading the report.

What I ran, on 3351586

A   control                        25 passed + 0 failed   (shell)
                                   25 pass  + 0 fail      (twin)
B   VE_KEY_COL 0 -> 99             24 + 1 FAILED, the premise, nothing else
                                   23 + 1 fail, same
    premise: the per-chunk arm's residual was summed over chunks that exist:
    got 'measured-nothing' want 'measured'

Carried forward to the head I am approving. 73cc36a differs from 3351586 by
CHANGELOG.md alone; test/validity_elision.sh and test/pytest/test_validity_elision.py
are byte-identical (md5 8436a4d6ede5 and ea33756ef103 on both), so the cells above
still describe this tree.

CI verified by SHA rather than by the PR view: 14 of 14 check-runs completed success,
non-green 0, mergeable=MERGEABLE state=CLEAN.

One thing recorded, not blocking

C', the deliberate desync, still reaches a green suite over an empty population:

nulls_key_res="$(validity_residual ve_nulls 99)"
nulls_key_chunks="$(chunks_measured ve_nulls $VE_KEY_COL)"
   -> 25 passed + 0 failed

Which is why the trimmed claim on the PR is the right one: the variable makes the desync
unreachable by changing the column, the edit someone would really make, and that is
what it earns. The transferable part is the twin's, and you have already written it —
key_res, key_chunks = _residual(...) closes it because one call cannot return two
populations. That is a property of returning the measurement together with its
population, not of naming a variable, and it is the shape I would copy into the next
suite that needs a population premise.

The rest, checked rather than taken

four version-check call sites, not two   reader.c:560, reader.c:4292,
                                         vector.c:2916, vector.c:5143
coalesced fetch reader                   one caller, line 4473, inside
                                         pgcolumnar_fetch_row (4235..4737),
                                         181 lines after the stamp check
surviving size bounds                    all compare a STORED bitmap against
                                         pageLength; cvb is 0 for an elided
                                         chunk, so they are vacuously satisfied
                                         rather than wrongly restrictive
sqlstate()                               pins XX001, so a FATAL or a 42501
                                         cannot satisfy it

I did not find a variant of the reverted byteLength bound. That part is inspection over
the diff rather than mutation, so treat it as weaker evidence than the cells above.

The composed count, for whoever merges second

Derived independently — composed the two branches, unioned COMPLETE to 24 stems, and
collected:

guard_tests    380
cluster_tests  430

Same number you reached a different way, and it reconciles: main 423, +3 from #1136, +4
from here. Whoever goes second writes 430 fresh rather than trusting either derivation.

Your correction of the compose note was the right call — check_ledger.tsv,
check_ledger_budget.txt and run_all_versions.sh are touched by this PR alone, so they
merge quietly because there is one side, which is not the trap. The trap needs two sides
each writing a plausible value.

Q3

Equal branches (1.679e9 against 1.676e9) with 590M more instructions retired, same plan,
same actual rows, same checksum, is a genuinely strange result and publishing the branch
figures with the mechanism named as unidentified is the honest form. I have nothing to add
and would rather you left it unexplained than filled it in.