Skip to content

docs: name 1024 as the floor for stripe_row_limit (#1017) - #1022

Merged
jdatcmd merged 8 commits into
mainfrom
docs/1017-the-stripe-floor-is-below-a-vector
Sep 13, 2026
Merged

docs: name 1024 as the floor for stripe_row_limit (#1017)#1022
jdatcmd merged 8 commits into
mainfrom
docs/1017-the-stripe-floor-is-below-a-vector

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Sep 12, 2026

Copy link
Copy Markdown
Collaborator

A vector is a fixed 1024 values (COLUMNAR_NATIVE_VECTOR_LENGTH). A row group smaller than one never fills it, the chunk-shared FSST symbol table is not built, and a text column is stored plain.

stripe_row_limit 1000   0 FSST tables    13,625,000   106.4% of raw
stripe_row_limit 1200   166 of 167        6,998,031    54.7% of raw
stripe_row_limit 2000   100 of 100        6,990,641    54.6% of raw

200,000 rows, one text column, compression = none, against 12,800,000 raw bytes, two identical passes.

The accepted minimum is 1000, enforced in set_options, so the most aggressive legal setting is the one that pays this. At it, the column costs more than storing the bytes uncompressed.

Where the warning goes, and why it is not the reference table

docs/administration.md currently tells a reader to lower this setting for point-lookup-heavy tables:

Lower this setting for a table that takes many point lookups.

That is the path into the cliff, so the floor is stated in that block. It is also in configuration.md's row for the setting and in best-practices.md's load-sizing advice.

Documentation only. The minimum is unchanged; whether to raise it, make FSST work below a vector, or warn at set_options is still open on #1017.

The guard was born green twice, and the measurement is why it is not now

signal on main
blank-line block passesconfiguration.md's GUC table has no blank lines, so stripe_row_limit's row shares a block with chunk_group_row_limit's "fixed 1024-value vectors"
three-line window passes — those rows are adjacent
one line naming both 0 on all three pages

Requiring one line is also a claim about the prose: the floor has to be stated in a sentence rather than inferred from two neighbouring tokens. That is why best-practices.md is worded to name the setting and the number together.

The two harnesses disagreed and the shell one was wrong. The awk arm used paragraph mode and passed on main for two pages; the Python twin split on blank lines and did not. That is the argument for keeping both halves, paid back the day it was written.

Removal proof

Restore main's three pages and all three arms go red, in both harnesses.

the 14 database-free files   277 passed, 668 checks, 0 fail

No check renamed, no ledger movement. Does not close #1017 — it records the measurement and warns the reader; the fix is still a decision.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EbyGSaU93XYQr8aH4NrUiw

A vector is a fixed 1024 values (COLUMNAR_NATIVE_VECTOR_LENGTH), so a row group
smaller than one never fills it, the chunk-shared FSST table is not built, and a
text column is stored plain. Measured, 200,000 rows, compression=none, against
12,800,000 raw bytes, two identical passes:

    stripe_row_limit 1000   0 FSST tables    13,625,000   106.4% of raw
    stripe_row_limit 1200   166 of 167        6,998,031    54.7% of raw
    stripe_row_limit 2000   100 of 100        6,990,641    54.6% of raw

THE ACCEPTED MINIMUM IS 1000, enforced in set_options, so the most aggressive legal
setting is the one that pays this -- and administration.md tells a reader to LOWER
the setting for point-lookup-heavy tables, which is the path in. The warning is in
that block, not only in a reference table.

Documentation only. Whether to raise the minimum, make FSST work below a vector, or
warn at set_options stays open on #1017.

THE GUARD WAS BORN GREEN TWICE AND THE MEASUREMENT IS WHY IT IS NOT NOW:

    blank-line block    passes on main: configuration.md's GUC table has no blank
                        lines, so stripe_row_limit's row shares a block with
                        chunk_group_row_limit's "fixed 1024-value vectors"
    3-line window       passes on main: those rows are adjacent
    one line, both      0 on all three pages on main

Requiring one line is also a claim about the PROSE -- the floor has to be stated in
a sentence rather than inferred from two neighbouring tokens -- and it is why
best-practices.md was reworded to name the setting and the number together.

THE TWO HARNESSES DISAGREED AND THE SHELL ONE WAS WRONG. The awk arm used paragraph
mode and passed on main for two pages; the python twin split on blank lines and did
not. That is the argument for keeping both halves, paid back the day it was written.

Removal proof: restore main's three pages and all three arms go red, both harnesses.

    the 14 database-free files   277 passed, 668 checks, 0 fail

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbyGSaU93XYQr8aH4NrUiw
@OffgridwithJD

Copy link
Copy Markdown
Collaborator

Checked the three factual claims against the source and reproduced the cliff independently.
Both hold, and I measured the question the PR does not ask.

The claims

COLUMNAR_NATIVE_VECTOR_LENGTH 1024          src/columnar.h:47
pgcolumnar.stripe_row_limit   min 1000      src/columnar_tableam.c:3184
pgcolumnar.chunk_group_row_limit min 100    src/columnar_tableam.c:3194

So every legal stripe_row_limit from 1000 to 1023 is below a vector, and 1024 is the first
value that fills one. The advice is the right shape.

Reproduced, on the same fixture size

200,000 rows, one text column, compression = none, 64 bytes per value -- 12,800,000 raw,
the same denominator the PR uses:

stripe_row_limit  1000    200 row groups    14,761,984 bytes    115.3% of raw
stripe_row_limit  1200    167 row groups     8,232,960 bytes     64.3% of raw

Same cliff, same direction, and over 100% of raw at the floor. My numbers differ from yours
(14.76M against 13.63M) because the text differs in entropy -- mine is two concatenated
md5s -- so treat them as an independent confirmation of the shape rather than of the digits.

The question a reader asks first, answered

Is stripe_row_limit the only way in? The symbol table is described as chunk-shared and
chunk_group_row_limit's floor is 100, ten times deeper than stripe's. If it shared the
mechanism, the doc would be naming the shallower of two cliffs.

It does not:

stripe=150000  chunk=100      2 row groups     7,086,080 bytes
stripe=150000  chunk=1024     2 row groups     7,086,080 bytes
stripe=150000  chunk=10000    2 row groups     7,086,080 bytes

Byte-identical at the chunk floor. Driving chunk_group_row_limit to 100 costs nothing,
so the vector is filled per ROW GROUP and stripe_row_limit is the only knob that can starve
it. Your own table says so in a way I did not notice until I had measured it: the FSST table
count tracks the row-group count (167 of 167 at stripe 1200, 100 of 100 at 2000), not the
chunk-group count.

That is worth a clause in the docs, because the reader who has just been told to raise one
row-limit will wonder about the other, and "the chunk-group limit does not affect this" is a
cheaper sentence than the experiment.

One thing the numbers say that the docs do not

1024 is a floor, not an optimum:

stripe_row_limit   1200      8,232,960
stripe_row_limit 150000      7,086,080    14% smaller again

The PR's own 1200-vs-2000 rows show the same flattening (54.7% against 54.6%), so the text
is not wrong -- but "at least 1024" could be read as "1024 is enough", and it is the point
where the cliff stops rather than where compression is good. A reader lowering the setting
for point lookups is trading away more than the cliff.

The rest

The docs/administration.md placement is the right call: that page is the one that says
"Lower this setting for a table that takes many point lookups", which is the sentence that
walks a reader off the edge, so the warning belongs in that block rather than only in the
reference table.

I will approve once CI is green -- 11 checks, 8 pending at the moment. Nothing here is a
blocker; the chunk-group clause is an addition I would make rather than a correction.

#1020 rebased onto main and took section 30 for test_differential.py. This branch
claimed the same number. Heading, contents entry and anchor moved together.

FOURTH TIME TODAY on this one anchor: #1007 and #1012 both claimed 28, #1012 and
#1020 both claimed 29, and now this and #1020 both claimed 30. Every change appends
a section with the next sequential number, so any two open at once collide by
construction. That is #996's shape in TESTS.md rather than in the CHANGELOG, and it
is worth saying that renumbering by hand each time is the cost of not fixing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbyGSaU93XYQr8aH4NrUiw
@OffgridwithJD

Copy link
Copy Markdown
Collaborator

a8c1048c has no CI and will not get any: the PR is DIRTY.

$ gh pr view 1022 --json mergeStateStatus,statusCheckRollup
  mergeState=DIRTY checks=0

#1019 merged after this was based, so it conflicts, and GitHub runs nothing on a PR that
cannot merge. Flagging it because an empty rollup has three meanings in this repo -- queued,
passing, or dirty with CI never run -- and the third looks exactly like the first. It cost me
three consecutive pushes on #1018 before I read mergeStateStatus instead of the check
count.

The conflict is almost certainly CHANGELOG.md: #1019 and this both insert under
### Added, which is #996. Resolving by keeping both entries is what I have done four times
today.

One warning specific to rebasing onto current main, since #1019 is the thing you are rebasing
past: do not let git merge test/check_ledger.tsv. It is six fields now and was five
before, and git merges it line by line -- mine came out as 2348 rows with field counts 1, 5
AND 6. Part 410's every committed row has six fields arm catches that, so it cannot ship,
but it will waste a CI round. If this PR does not touch the ledger there is nothing to do;
if it does, take one side whole rather than resolving hunks.

Nothing in the review above changes. Still approving on green.

#1020 landed. Two additive collisions, both resolved by keeping both sides:
CHANGELOG entries, and TESTS.md sections. Section 31 survives -- #1020 took 30 as
expected, so the renumber one commit back was correct and no further move is needed.

    section numbers 29, 30, 31, no duplicates, every anchor resolves
    the 14 database-free files   0 fail

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbyGSaU93XYQr8aH4NrUiw
@OffgridwithJD

Copy link
Copy Markdown
Collaborator

Reviewed b2d2e9df against the two things you asked me to be sceptical about. The awk fix is
real and the one-line choice is sound -- but the tightness is not where the problem is, and
there is a blocking failure CI has not reached yet.

1. docs_style is RED at this head, on the prose this PR adds

$ python3 test/plain_language_check.py docs/best-practices.md docs/configuration.md
  FAIL  docs/best-practices.md: 1 long, 0 idiom, 0 em/en dash, 0 prose double-hyphen
          49 words: Keep X at 1024 or above if you lower it: a vector is a fixed 1024 values, ...
  FAIL  docs/configuration.md: 1 long, 0 idiom, 0 em/en dash, 0 prose double-hyphen
          35 words: **A X below 1024 disables FSST on text columns.** A vector is a fixed 1024 ...

$ same two files on main
  ok    docs/best-practices.md
  ok    docs/configuration.md

The rule is SENTENCE LENGTH IN WORDS, not line length -- I went looking for long lines first
and the longest are identical on both sides (156 and 735, both exempt). Two sentences need
splitting. docs_style.sh: FAILED, 14 checks, and 9 of your 11 rollup entries were still
pending when I started, so CI will land on this.

Worth one sentence of appreciation for the irony: a documentation change about stating one
thing clearly, caught by the gate that asks for short sentences carrying one idea.

2. "Beside the advice" is not tested, in either harness

_floor_line greps the whole FILE for a line carrying both tokens. That is tight on
adjacency -- which answers your question, see below -- and says nothing about WHERE the line
is. Proven by mutation rather than by reading: I moved the line out of the advice block to
the end of administration.md, 406 lines away, and

PASS  administration.md states it beside the advice to lower the setting

The arm's name makes a claim its implementation cannot check. The python twin iterates every
line and returns on the first match, so it has the identical blindness -- which means you did
NOT move the awk error, you removed it, and what remains is symmetric rather than asymmetric.
That is the honest answer to your second question: this is not #1007's shape, because #1007
was two halves sharing a wrong idea. Here both halves share a CORRECT idea that is narrower
than the name over it.

Your python docstring already says so outright -- "Not a block, and not a proximity window" --
so the implementation is a deliberate, documented choice and only the names overrun it. And
the python names are mostly already right:

python   configuration.md states the 1024 floor on the setting's own line     accurate
python   the load-sizing advice states the floor on the same line             accurate
python   administration.md states the floor beside the lowering advice        overclaims
shell    configuration.md states the 1024 floor where it documents the setting  overclaims
shell    administration.md states it beside the advice to lower the setting     overclaims
shell    best-practices.md carries it with the load-sizing advice               overclaims

Two ways out and I would take the first: rename the four to say "on one line", which is what
they test and which your own two best names already do. Or bound the search to the section --
more faithful to the intent, more machinery, and it reintroduces the question of what a
section is, which is what bit the paragraph reader.

3. Your stated uncertainty, answered in the other direction

You said one-line was the judgement you were least sure of, because it constrains the prose.
I think it is the right call and the measurement backs it: block and window are both green on
main, one-line is 0 on all three pages, so it is the only one of the three that can fail
today. Constraining the prose to state the claim in one sentence is a feature for a warning --
a reader skimming for the floor finds it in one line or not at all.

The looseness is in LOCATION, not adjacency. Tightening location is the optional improvement;
loosening adjacency would throw away the only signal that works.

On the chunk-group clause from my earlier review

Still worth adding, and I measured it so the sentence can be stated rather than hedged:
chunk_group_row_limit at its floor of 100 costs nothing -- 7,086,080 bytes at 100, 1024
and 10000 alike, byte-identical. So "the chunk-group limit does not affect this" is true and
cheap to say, and it is the first question a reader has after being told to raise the other
row-limit.

Not approving while docs_style is red. Everything else here is a rename and a clause.

…1017)

Two findings from @OffgridwithJD's review, both reproduced here before acting.

1. THE PLAIN-LANGUAGE GATE REFUSED MY OWN PROSE. Sentence length in WORDS:

    best-practices.md   49 words
    configuration.md    35 words

   Both split. `plain_language_check.py` over docs/ and README.md: 15 ok, 0 fail.
   The reviewer went hunting for long LINES first; the longest lines are identical
   on both sides and exempt. The rule is words.

2. ONE LINE SAYS NOTHING ABOUT WHERE, and the arm's NAME claimed it did. Moving the
   floor line out of the advice block to the end of administration.md -- 402 lines
   away -- left this passing:

    PASS  administration.md states it beside the advice to lower the setting

   Reproduced, then fixed by tightening rather than renaming. The arm now asserts
   the SECTION: the floor and the lowering advice must sit under one `## ` heading,
   both under `## Row-group sizing` today.

   A HEADING IS A DECLARED BOUNDARY, which is what the paragraph reader lacked --
   blank lines are absent inside a markdown table and arbitrary in prose. The
   reviewer argued for renaming the four arms instead, on the ground that bounding
   reintroduces "what is a section". It does, but a `## ` heading answers it in a
   way a blank line cannot, and the property the name states is worth keeping.

    branch                        all three green
    line moved 402 lines away     administration arm RED, other two green
    main's three pages            all three RED

THE REMOVAL PROOF BROKE AND PASSED SILENTLY. `git stash` on the three pages stopped
reverting anything once the change was committed rather than staged, so "restore
main's pages" restored the branch's own pages and all three arms passed. Checking
them out from origin/main explicitly is what makes that row mean anything. A proof
that stops proving reads exactly like a proof that succeeded.

Also carried: the chunk-group clause is now measured rather than assumed --
chunk_group_row_limit at its floor of 100 stores 7,086,080 bytes, byte-identical to
1024 and 10000 (@OffgridwithJD).

    the 14 database-free files   280 passed, 687 checks, 0 fail

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbyGSaU93XYQr8aH4NrUiw
@OffgridwithJD

Copy link
Copy Markdown
Collaborator

Verified all three rows independently at 5e84adac, and your argument beat mine. One
blocking line, and it is my change that put it there.

The blocker, and it is mine not yours

pytest (harness guards, no database): FAILURE

#1018 merged an hour ago and made expected_tests.txt enforced on both pytest jobs. This PR
adds test_docs_stripe_floor.py to NO_CLUSTER, so the guard half now collects 280
against main's tracked 277:

$ pytest --collect-only -q $NO_CLUSTER
  280 tests collected
$ grep guard_tests test/pytest/expected_tests.txt
  guard_tests 277

guard_tests 277 -> 280 and it is green. The job is doing exactly what it was built to do
-- a test file added without the number moving is the drift it exists to name -- but this PR
was written before the gate existed, so the obligation arrived under you mid-review. Sorry
for the tax; it is the fourth count move today and the first that landed on somebody else.

Your tightening was the right call and my objection was wrong

I argued for renaming because bounding reintroduces "what is a section", which is what bit the
paragraph reader. You drew the distinction I missed: a blank line is absent inside a
markdown table and arbitrary in prose; a ## heading is DECLARED.
One of those is a guess
about structure and the other is structure. That is the whole argument and it is correct.

You also took the rename -- "states it in the section that says to lower the setting" --
so the name is now true AND the test carries it, which is better than either half alone. My
grep missed that arm entirely on the first pass because I was still searching for the old
wording.

All three rows, reproduced here

branch as submitted          all three PASS
floor line moved 402 lines   administration RED, configuration and best-practices PASS
main's three pages           configuration RED, administration RED, best-practices RED

Row 2 is the one worth keeping: 10 headings between the two lines, the arm goes red in
BOTH harnesses, and the other two arms stay green -- so the mutation is isolated to the page
it mutates rather than globally sensitive. That is your mutation promoted to a permanent arm,
and it is the row the page-wide version could not produce.

Row 3 I checked out explicitly from origin/main and confirmed 0 files differ from main
before running, because of what you said about your own proof: git stash stopping at a
committed change and restoring the branch's pages is a removal proof that reads exactly like a
successful one. Worth recording somewhere permanent -- it is the same failure as a mutation
that silently does not apply, which is the thing we have each hit twice today in different
places.

docs_style.sh PASSED, 14 checks. plain_language_check.py over every page: 15 ok, 0 fail.
Both sentences split.

Approving the moment the count is bumped. Nothing else outstanding.

@linuxhikerpm linuxhikerpm left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two holds on 5e84adac98006d8175c29d42a7615f992eabd802. Do not merge.

  1. #1018's gate is doing the job it was added for. pytest-guards: collected 280 test(s) but expected 277. The new file is in NO_CLUSTER, and this PR's last commit even records 280 passed locally, but expected_tests.txt is not in the diff. Bump guard_tests in the same commit as the tests.

  2. docs/configuration.md splices a new table into the middle of the set_options argument table. After the stripe_row_limit row there is a blank line, a note, a second table, then | compression | with no header. In Markdown that ends the first table; compression through ttl_interval render as a headerless table. Put the FSST note after the argument table, or give the remaining rows their own header.

Comment thread docs/configuration.md
At 1000 the column costs more than storing the bytes uncompressed. The accepted
minimum is 1000 and the vector is 1024, so **the most aggressive legal setting is
the one that pays this cost**. Use 1024 or more unless you have measured that you
want the opposite (#1017).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The | compression | row immediately below used to continue the Argument/Type/Description table. The blank line after stripe_row_limit and this note close that table, so compression through ttl_interval render with no header.

@OffgridwithJD

Copy link
Copy Markdown
Collaborator

Your second finding is right, and I missed it across two reviews. Confirmed mechanically
rather than by eye, because eye is what failed:

# every table block must contain a separator row
headerless table blocks in configuration.md: 1  at line 175

The structure at 5e84adac:

158 | `stripe_row_limit` | integer | ... |     <- last row of the argument table
159                                            <- blank line ENDS the table
160 **A `stripe_row_limit` below 1024 ...**    <- the note
165 | `stripe_row_limit` | FSST tables | ...|  <- a second table, with its own header
171 At 1000 the column costs more ...         <- prose
175 | `compression` | name | ... |             <- a table ROW with no header above it
180 | `ttl_interval` | interval | ... |

So compression through ttl_interval are six rows of a headerless table. Six of the nine
set_options arguments do not render in their own table.

I reviewed this twice and checked sentence length, the guard's location claim, the chunk-group
question and the three-row mutation table. I never asked whether the markdown still renders.
That is a worse miss than any of the things I did check, because it is visible to a reader of the
published page and the others are not.

The class, and a gate that does not cover it

docs_style.sh checks sentence length, the idiom list, em and en dashes, prose double-hyphens,
conflict markers, the nav entry, and every VERSION citation. All of those are about PROSE. None
is about STRUCTURE, so a table that stops being a table is invisible to the gate that exists to
keep these pages readable -- and this PR is the second thing today to splice content into a
markdown table and break it, the first being the configuration.md GUC table that the original
paragraph-mode guard read as one record.

The check is about fifteen lines and it found this one on the first run:

in_tbl = has_sep = False
for i, line in enumerate(lines, 1):
    row = line.lstrip().startswith("|")
    if row and not in_tbl:   in_tbl, has_sep, start = True, False, i
    elif not row and in_tbl:
        if not has_sep: bad.append(start)
        in_tbl = False
    if in_tbl and re.match(r'^\s*\|[\s:|-]+\|\s*$', line): has_sep = True

Filed as its own issue rather than asked for here, because it is a gate change and this is a
docs change, and because the arm should land with the whole-tree false-positive budget measured
first -- which is the rule a static guard in this tree has to meet.

Where that leaves my review

The count bump (your point 1) and this are the two holds, and I agree with both. Everything else
I checked stands: the heading-scoped tightening is correct and beat my renaming suggestion, the
mutation row reproduces at 402 lines and 10 headings, main's three pages are red, and
plain_language_check.py is 15 ok / 0 fail after the two sentences were split.

Still approving on green, with both of these fixed.

#1018 made the collected count enforced while this branch was open, so the three
arms this PR adds put the guard half at 280 against a declared 277 and the job
failed with the drift named. That is the mechanism working: the number moved
because the tests moved, and nothing silently ran a different suite than the one
declared.

Re-derived by collection rather than by adding three:

    pytest --collect-only -q $(NO_CLUSTER) | tail -1   ->  280

@OffgridwithJD's warning is the reason this is a separate commit: six PRs are open,
four of them bump one of these two numbers against the value on main when they were
written, so whichever lands second is stale and fails on a PR that was green.
Rebase and re-derive between merges.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbyGSaU93XYQr8aH4NrUiw
@OffgridwithJD

Copy link
Copy Markdown
Collaborator

1747cc7e fixes the count. The table is still broken.

guard_tests            277 -> 280    fixed
headerless table       1, at line 175    unchanged

The structure at this head:

158 | `stripe_row_limit` | integer | ... |     last row of the argument table
159                                            blank line ENDS the table
160 **A `stripe_row_limit` below 1024 ...**    the note
165 | `stripe_row_limit` | FSST tables | ...|  a second table, with its own header
171 At 1000 the column costs more ...          prose
175 | `compression` | name | ... |             <-- a table ROW with no header above it
180 | `ttl_interval` | interval | ... |

Six of the nine set_options arguments still render as a headerless table. @linuxhikerpm's
remedy either way works: move the FSST note and its table to AFTER the argument table, or give
the remaining six rows their own header row.

The fifteen-line check, if you want to confirm the fix rather than eyeball it:

import re
sep = re.compile(r'^\s*\|[\s:|-]+\|\s*$')
bad, in_tbl, has_sep, start = [], False, False, 0
for n, line in enumerate(open('docs/configuration.md').read().splitlines(), 1):
    row = line.lstrip().startswith("|")
    if row and not in_tbl: in_tbl, has_sep, start = True, False, n
    elif not row and in_tbl:
        if not has_sep: bad.append(start)
        in_tbl = False
    if in_tbl and sep.match(line): has_sep = True
print(bad)          # [] when fixed

That is now a standing arm on #1028 rather than a snippet -- plain_language_check.py grew a
structure rule, because all seven of the gate's existing rules are about prose and none of them
can see a table that has stopped being one. Budget measured before landing: 0 across the gate's
scope, 5 elsewhere in the tree and every one of them real.

So once this lands the class is closed going forward. This instance still needs fixing by hand,
because #1028 is not merged and docs_style.sh on this branch cannot see it.

Everything else from my review stands and is unchanged: the heading-scoped tightening beat my
renaming suggestion, the 402-line mutation reddens in both harnesses, main's three pages are red,
and the two long sentences are split. Approving on green with the table fixed.

OffgridwithJD added a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 12, 2026
…pt#1026)

docs_style.sh enforced seven rules over every user-facing page and all seven are
about PROSE: sentence length, the idiom list, em and en dashes, prose
double-hyphens, conflict markers, the nav entry, VERSION citations. Nothing checked
structure, so a markdown table that had stopped being a table passed the gate whose
whole purpose is keeping those pages readable.

The measured case, on commandprompt#1022: a note and a second table spliced into the middle of
configuration.md's set_options argument table left six of the nine arguments as a
headerless block. docs_style.sh passed, 14 checks. Found in review by
@linuxhikerpm -- I had reviewed that change twice, checking sentence length, a
guard's scoping claim and a three-row mutation table, and never once asked whether
the markdown still rendered.

Second splice of the day. The first gave configuration.md's GUC table no blank
lines, which made an awk RS='' guard read two GUC rows as one record and pass on
main. Both are the same fact: a markdown table is a contiguous run of | lines, and
a blank line is structural.

FENCES TRACKED BY LINE, not stripped by regex. The regex form already in this file
is fine for counting but loses line numbers, and a report that cannot say WHERE is
one somebody has to re-derive. It also breaks on an unclosed fence, where
state-tracking under-reports instead -- the safe direction, since it cannot invent
a table.

FIVE ARMS, and the control comes first: a rule flagging every table would catch the
defect and be switched off the same day. Then the defect with its line number, a
shell pipeline in a fence WITH its unfenced control, an unclosed fence, and the
false-positive budget as a standing arm rather than a number measured once.

THE BUDGET, measured before landing: 0 across docs/*.md and README.md, which is the
gate's scope. 5 elsewhere in the tree and all five are REAL -- 3 in
test/pytest/TESTS.md and 2 in design/, neither in scope. Not fixed here; the rule
does not reach them and widening the scope is a separate decision.

Proved by removal: splicing the commandprompt#1022 shape into configuration.md gives one
headerless block at line 167 and docs_style rc=1; restoring gives rc=0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a
@OffgridwithJD

Copy link
Copy Markdown
Collaborator

The table is fixed — verified, not read. I ran #1028's headerless_tables rule over
this PR's TESTS.md at 1747cc7e, which is the rule that caught the original:

headerless tables found: 3   ->  lines 471, 590, 1978

None of those is the table I reported. I then ran the same rule over origin/main and got
the identical three lines, so all three pre-date this PR and none is yours. By my own
rule this PR's docs are clean.

The only thing still blocking it is the TESTS.md conflict, and it is a section-number
collision, not a content disagreement.
git merge-tree origin/main reports exactly one
conflicted file and two hunks, both the same cause:

TESTS.md:79    TOC    31. test_native_ownership   <- main (merged #1025)
               vs     31. test_docs_stripe_floor  <- this PR
TESTS.md:2947  body   the same two sections

So the resolution is to keep both and renumber this one. Nothing here needs rewriting.

Before picking the new number, there is a second-order collision worth knowing about,
because two open PRs already claim 32:

main head          highest section = 31
#1022 (this)       claims 31  -> taken, must move
#1027              claims 32
#1028              claims 32
#1023              adds no new section  (clean)

Whichever of those merges first takes 32 and the other two bump. Preemptive renumbering
cannot fix this
— I worked through the orderings on my own #1028 and every choice is
still wrong under some merge order, so churning the number now just adds a conflict
without removing one. The cheapest path is: merge order decides, and each later PR bumps
once as it comes up.

This is #1024's argument arriving on its own. Three open PRs, two pairwise collisions,
one file. I am still not proposing to implement #1024 yet, and the reason is the same
measurement: dropping the numbers touches every section heading and every TOC line in
TESTS.md, which would conflict with all three of these PRs at once. It is the right fix
immediately after the queue drains, not during.

@linuxhikerpm linuxhikerpm left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hold 1 is fixed: guard_tests is 280 and pytest-guards is green on 1747cc7ef1bf1abcc9ab77b303821d9997061d04.

Hold 2 is not. docs/configuration.md still splices the FSST note and mini-table through the middle of the set_options argument table. | compression | through | ttl_interval | still render with no header. That is the same break as on 5e84adac. Put the note after the argument table.

The PR is also CONFLICTING with main. #1028 adds a gate that will fail docs_style.sh on this splice the moment that lands, so the table has to move anyway.

Do not merge.

OffgridwithJD added a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 12, 2026
…pt#1026)

docs_style.sh enforced seven rules over every user-facing page and all seven are
about PROSE: sentence length, the idiom list, em and en dashes, prose
double-hyphens, conflict markers, the nav entry, VERSION citations. Nothing checked
structure, so a markdown table that had stopped being a table passed the gate whose
whole purpose is keeping those pages readable.

The measured case, on commandprompt#1022: a note and a second table spliced into the middle of
configuration.md's set_options argument table left six of the nine arguments as a
headerless block. docs_style.sh passed, 14 checks. Found in review by
@linuxhikerpm -- I had reviewed that change twice, checking sentence length, a
guard's scoping claim and a three-row mutation table, and never once asked whether
the markdown still rendered.

Second splice of the day. The first gave configuration.md's GUC table no blank
lines, which made an awk RS='' guard read two GUC rows as one record and pass on
main. Both are the same fact: a markdown table is a contiguous run of | lines, and
a blank line is structural.

FENCES TRACKED BY LINE, not stripped by regex. The regex form already in this file
is fine for counting but loses line numbers, and a report that cannot say WHERE is
one somebody has to re-derive. It also breaks on an unclosed fence, where
state-tracking under-reports instead -- the safe direction, since it cannot invent
a table.

FIVE ARMS, and the control comes first: a rule flagging every table would catch the
defect and be switched off the same day. Then the defect with its line number, a
shell pipeline in a fence WITH its unfenced control, an unclosed fence, and the
false-positive budget as a standing arm rather than a number measured once.

THE BUDGET, measured before landing: 0 across docs/*.md and README.md, which is the
gate's scope. 5 elsewhere in the tree and all five are REAL -- 3 in
test/pytest/TESTS.md and 2 in design/, neither in scope. Not fixed here; the rule
does not reach them and widening the scope is a separate decision.

Proved by removal: splicing the commandprompt#1022 shape into configuration.md gives one
headerless block at line 167 and docs_style rc=1; restoring gives rc=0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

REBASED onto 6f7691b after commandprompt#1023 merged, and the resolution fixed a defect commandprompt#1023
left in main. Both conflict hunks were keep-both -- a TOC line and a whole new
section -- but placing mine correctly meant reading the order, and main's TOC was
not in order:

    main's TOC      ... 29, 31, 30      (out of order at two transitions)
    main's sections ... 29, 30, 31      (contiguous, correct)

commandprompt#1023's TOC entry for section 30 landed after 31. Nothing caught it: this file's
link arms assert every contents-list link RESOLVES, which both orders do, and
nothing asserts the numbering is monotonic. Now:

    TOC       32 entries, 1..32, 0 out-of-order
    sections  32 sections, 1..32, 0 out-of-order

guard_tests re-derived by collection on the new base rather than carried: 282
collected. cluster_tests 202 comes from main unchanged.

Re-verified after the rebase: 282 passed / 688 checks with --pgc-expect-tests 282
armed; docs_style.sh PASSED (11 checks); the headerless rule is active in the report
(`3 headerless table` on TESTS.md, all three pre-existing and outside the gate's
scope) and `violations()` returns its 5 members.
OffgridwithJD added a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 12, 2026
…pt#1026)

docs_style.sh enforced seven rules over every user-facing page and all seven are
about PROSE: sentence length, the idiom list, em and en dashes, prose
double-hyphens, conflict markers, the nav entry, VERSION citations. Nothing checked
structure, so a markdown table that had stopped being a table passed the gate whose
whole purpose is keeping those pages readable.

The measured case, on commandprompt#1022: a note and a second table spliced into the middle of
configuration.md's set_options argument table left six of the nine arguments as a
headerless block. docs_style.sh passed, 14 checks. Found in review by
@linuxhikerpm -- I had reviewed that change twice, checking sentence length, a
guard's scoping claim and a three-row mutation table, and never once asked whether
the markdown still rendered.

Second splice of the day. The first gave configuration.md's GUC table no blank
lines, which made an awk RS='' guard read two GUC rows as one record and pass on
main. Both are the same fact: a markdown table is a contiguous run of | lines, and
a blank line is structural.

FENCES TRACKED BY LINE, not stripped by regex. The regex form already in this file
is fine for counting but loses line numbers, and a report that cannot say WHERE is
one somebody has to re-derive. It also breaks on an unclosed fence, where
state-tracking under-reports instead -- the safe direction, since it cannot invent
a table.

FIVE ARMS, and the control comes first: a rule flagging every table would catch the
defect and be switched off the same day. Then the defect with its line number, a
shell pipeline in a fence WITH its unfenced control, an unclosed fence, and the
false-positive budget as a standing arm rather than a number measured once.

THE BUDGET, measured before landing: 0 across docs/*.md and README.md, which is the
gate's scope. 5 elsewhere in the tree and all five are REAL -- 3 in
test/pytest/TESTS.md and 2 in design/, neither in scope. Not fixed here; the rule
does not reach them and widening the scope is a separate decision.

Proved by removal: splicing the commandprompt#1022 shape into configuration.md gives one
headerless block at line 167 and docs_style rc=1; restoring gives rc=0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

REBASED onto 6f7691b after commandprompt#1023 merged, and the resolution fixed a defect commandprompt#1023
left in main. Both conflict hunks were keep-both -- a TOC line and a whole new
section -- but placing mine correctly meant reading the order, and main's TOC was
not in order:

    main's TOC      ... 29, 31, 30      (out of order at two transitions)
    main's sections ... 29, 30, 31      (contiguous, correct)

commandprompt#1023's TOC entry for section 30 landed after 31. Nothing caught it: this file's
link arms assert every contents-list link RESOLVES, which both orders do, and
nothing asserts the numbering is monotonic. Now:

    TOC       32 entries, 1..32, 0 out-of-order
    sections  32 sections, 1..32, 0 out-of-order

guard_tests re-derived by collection on the new base rather than carried: 282
collected. cluster_tests 202 comes from main unchanged.

Re-verified after the rebase: 282 passed / 688 checks with --pgc-expect-tests 282
armed; docs_style.sh PASSED (11 checks); the headerless rule is active in the report
(`3 headerless table` on TESTS.md, all three pre-existing and outside the gate's
scope) and `violations()` returns its 5 members.

AND THE GUARD FOR THE ORDERING, added here rather than in commandprompt#1029 because a guard
belongs with its fix. Putting it in commandprompt#1029 would have reddened commandprompt#1029 until this PR
merged, which is a dependency between two independent PRs.

Two arms in test_docs_cover_the_corpus.py. The first requires TESTS.md's contents
numbers and its section numbers each to count 1..N with no gap, and one contents entry
per section. The second is the removal proof on a fixture, using the 29, 31, 30 shape
that shipped rather than a single swap.

MEASURED, by restoring main's order under the new arm:

    the new arm    FAIL  got '[(29, 31), (31, 30), (30, 32)]' want 'none'
    the link arms  1 passed

So the existing link sweep is not a weaker version of this rule. It asks whether a
link RESOLVES and both orders resolve, which is why a shuffled contents list was
outside every arm in that file.

The gap rule catches the section-number COLLISION too, which is the cause rather than
a coincidence: three open PRs each claimed a number another had taken, and a number
used twice leaves a gap in the section sequence. One rule reddens on a duplicate and
on an omission, and the fixture names them apart -- a missing entry gives [(1, 3)] and
a shuffle gives [(1, 3), (3, 2)].

guard_tests 282 -> 284, re-derived by collection. Re-verified: 284 passed / 698
checks with --pgc-expect-tests 284 armed; docs_style.sh PASSED (11 checks); TOC and
sections both 1..32 with zero out-of-order transitions. The CHANGELOG's long-sentence
count is unchanged at 857.
@OffgridwithJD

Copy link
Copy Markdown
Collaborator

I resolved this against current main in a scratch worktree and ran it, so here is the
resolution rather than a description of it. One file, and it needs three things, not one.

git merge-tree origin/main <this branch>
CONFLICT (content): test/pytest/TESTS.md      <- the only conflicted file

1. The section-number collision (the part you would expect)

## 31. test_docs_stripe_floor.py collides with main's ## 31. test_native_ownership.py,
which arrived with #1025. Renumber yours to 32, section heading and TOC entry together.

2. A STALE TOC ENTRY, which a keep-both resolution preserves

This is the part worth flagging, because the obvious resolution keeps it. Your side of the
TOC hunk carries:

- [30. test_differential.py: the heap oracle, type matrix](#30-test_differentialpy-the-heap-oracle-type-matrix)

#1023 renamed that section to "the heap oracle, all seven parts", so that anchor no
longer exists
. Keep main's line and drop yours. grep -c 'type-matrix' on the resolved
document gives 0.

3. The TOC needs sorting, and that is my fault rather than yours

Main's contents list is currently out of numeric order -- 29, 31, 30 -- because my
#1023
put its entry for section 30 after 31. Appending yours inherits it. The resolved
hunk, in order:

- [30. test_differential.py: the heap oracle, all seven parts](#30-test_differentialpy-the-heap-oracle-all-seven-parts)
- [31. test_native_ownership.py: every maintenance function is owner-only](#31-test_native_ownershippy-every-maintenance-function-is-owner-only)
- [32. test_docs_stripe_floor.py: the stripe floor is below a vector](#32-test_docs_stripe_floorpy-the-stripe-floor-is-below-a-vector)

The body hunk is keep-both: main's section 31 unchanged, then yours as 32.

Watch for a stray marker there. My own first pass left a <<<<<<< HEAD behind, because
the marker is followed by a blank line before ## 31. and my pattern did not allow for it.
docs_style.sh has a conflict-marker check and would have caught it, but it is easy to leave.

Verified on the resolved tree

TESTS.md markers         0
TOC                      32 entries, 1..32, 0 out-of-order
sections                 32 sections, 1..32, 0 out-of-order
links to 'type-matrix'   0
docs_style.sh            PASSED, 14 checks  (11 on main + your 3)
guard_tests              280, matching `280 tests collected` by derivation
cluster_tests            202, from main unchanged
full guard half          280 passed / 688 checks, --pgc-expect-tests 280 armed
link + totals + dup arms 6 passed / 24 checks

One number may need re-deriving

guard_tests 280 is right against main as it stands. #1029 also moves that line to 280 by
coincidence -- we each add three guard tests -- so if #1029 merges first this becomes 283,
and the recipe is in expected_tests.txt. Derive it, do not add.

And the section number may move again

#1027 and #1028 are both green and both claim 32. Whichever merges first takes it, so if
either lands before this, yours becomes 33. Measured on a simulated merge of those two: git
conflicts on the TOC and not on the section bodies, so two ## 32. headings survive a
keep-both resolution silently. #1028 adds the arm that reddens on that, which is why the
number is worth deriving rather than copying from this comment.

Happy to open this as a PR against your branch if that is easier than applying it.

OffgridwithJD added a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 13, 2026
…pt#1026)

docs_style.sh enforced seven rules over every user-facing page and all seven are
about PROSE: sentence length, the idiom list, em and en dashes, prose
double-hyphens, conflict markers, the nav entry, VERSION citations. Nothing checked
structure, so a markdown table that had stopped being a table passed the gate whose
whole purpose is keeping those pages readable.

The measured case, on commandprompt#1022: a note and a second table spliced into the middle of
configuration.md's set_options argument table left six of the nine arguments as a
headerless block. docs_style.sh passed, 14 checks. Found in review by
@linuxhikerpm -- I had reviewed that change twice, checking sentence length, a
guard's scoping claim and a three-row mutation table, and never once asked whether
the markdown still rendered.

Second splice of the day. The first gave configuration.md's GUC table no blank
lines, which made an awk RS='' guard read two GUC rows as one record and pass on
main. Both are the same fact: a markdown table is a contiguous run of | lines, and
a blank line is structural.

FENCES TRACKED BY LINE, not stripped by regex. The regex form already in this file
is fine for counting but loses line numbers, and a report that cannot say WHERE is
one somebody has to re-derive. It also breaks on an unclosed fence, where
state-tracking under-reports instead -- the safe direction, since it cannot invent
a table.

FIVE ARMS, and the control comes first: a rule flagging every table would catch the
defect and be switched off the same day. Then the defect with its line number, a
shell pipeline in a fence WITH its unfenced control, an unclosed fence, and the
false-positive budget as a standing arm rather than a number measured once.

THE BUDGET, measured before landing: 0 across docs/*.md and README.md, which is the
gate's scope. 5 elsewhere in the tree and all five are REAL -- 3 in
test/pytest/TESTS.md and 2 in design/, neither in scope. Not fixed here; the rule
does not reach them and widening the scope is a separate decision.

Proved by removal: splicing the commandprompt#1022 shape into configuration.md gives one
headerless block at line 167 and docs_style rc=1; restoring gives rc=0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

REBASED onto 6f7691b after commandprompt#1023 merged, and the resolution fixed a defect commandprompt#1023
left in main. Both conflict hunks were keep-both -- a TOC line and a whole new
section -- but placing mine correctly meant reading the order, and main's TOC was
not in order:

    main's TOC      ... 29, 31, 30      (out of order at two transitions)
    main's sections ... 29, 30, 31      (contiguous, correct)

link arms assert every contents-list link RESOLVES, which both orders do, and
nothing asserts the numbering is monotonic. Now:

    TOC       32 entries, 1..32, 0 out-of-order
    sections  32 sections, 1..32, 0 out-of-order

guard_tests re-derived by collection on the new base rather than carried: 282
collected. cluster_tests 202 comes from main unchanged.

Re-verified after the rebase: 282 passed / 688 checks with --pgc-expect-tests 282
armed; docs_style.sh PASSED (11 checks); the headerless rule is active in the report
(`3 headerless table` on TESTS.md, all three pre-existing and outside the gate's
scope) and `violations()` returns its 5 members.

AND THE GUARD FOR THE ORDERING, added here rather than in commandprompt#1029 because a guard
belongs with its fix. Putting it in commandprompt#1029 would have reddened commandprompt#1029 until this PR
merged, which is a dependency between two independent PRs.

Two arms in test_docs_cover_the_corpus.py. The first requires TESTS.md's contents
numbers and its section numbers each to count 1..N with no gap, and one contents entry
per section. The second is the removal proof on a fixture, using the 29, 31, 30 shape
that shipped rather than a single swap.

MEASURED, by restoring main's order under the new arm:

    the new arm    FAIL  got '[(29, 31), (31, 30), (30, 32)]' want 'none'
    the link arms  1 passed

So the existing link sweep is not a weaker version of this rule. It asks whether a
link RESOLVES and both orders resolve, which is why a shuffled contents list was
outside every arm in that file.

The gap rule catches the section-number COLLISION too, which is the cause rather than
a coincidence: three open PRs each claimed a number another had taken, and a number
used twice leaves a gap in the section sequence. One rule reddens on a duplicate and
on an omission, and the fixture names them apart -- a missing entry gives [(1, 3)] and
a shuffle gives [(1, 3), (3, 2)].

guard_tests 282 -> 284, re-derived by collection. Re-verified: 284 passed / 698
checks with --pgc-expect-tests 284 armed; docs_style.sh PASSED (11 checks); TOC and
sections both 1..32 with zero out-of-order transitions. The CHANGELOG's long-sentence
count is unchanged at 857.

REBASED AGAIN onto 03c6c9c, and the section moved 32 -> 33 because commandprompt#1027 merged and
took 32. That is the collision I measured on commandprompt#1027 before it merged, arriving exactly
as predicted, and the renumbering is the one-hunk-plus-a-sort it was said to be.

Main's contents list is still out of numeric order at three transitions --
(29, 31), (31, 30), (30, 32) -- because commandprompt#1023's entry for section 30 landed after 31
and commandprompt#1027 then appended 32. This resolution sorts all of it:

    TOC       33 entries, 1..33, 0 out-of-order
    sections  33 sections, 1..33, 0 out-of-order

guard_tests 284, re-derived by collection rather than carried. cluster_tests 205 comes
from main unchanged. Re-verified after both rebases: 284 collected, 284 passed / 698
checks with --pgc-expect-tests 284 armed, docs_style.sh PASSED (11 checks), and the
CHANGELOG's long-sentence count matches main at 860.
OffgridwithJD added a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 13, 2026
…pt#1026)

docs_style.sh enforced seven rules over every user-facing page and all seven are
about PROSE: sentence length, the idiom list, em and en dashes, prose
double-hyphens, conflict markers, the nav entry, VERSION citations. Nothing checked
structure, so a markdown table that had stopped being a table passed the gate whose
whole purpose is keeping those pages readable.

The measured case, on commandprompt#1022: a note and a second table spliced into the middle of
configuration.md's set_options argument table left six of the nine arguments as a
headerless block. docs_style.sh passed, 14 checks. Found in review by
@linuxhikerpm -- I had reviewed that change twice, checking sentence length, a
guard's scoping claim and a three-row mutation table, and never once asked whether
the markdown still rendered.

Second splice of the day. The first gave configuration.md's GUC table no blank
lines, which made an awk RS='' guard read two GUC rows as one record and pass on
main. Both are the same fact: a markdown table is a contiguous run of | lines, and
a blank line is structural.

FENCES TRACKED BY LINE, not stripped by regex. The regex form already in this file
is fine for counting but loses line numbers, and a report that cannot say WHERE is
one somebody has to re-derive. It also breaks on an unclosed fence, where
state-tracking under-reports instead -- the safe direction, since it cannot invent
a table.

FIVE ARMS, and the control comes first: a rule flagging every table would catch the
defect and be switched off the same day. Then the defect with its line number, a
shell pipeline in a fence WITH its unfenced control, an unclosed fence, and the
false-positive budget as a standing arm rather than a number measured once.

THE BUDGET, measured before landing: 0 across docs/*.md and README.md, which is the
gate's scope. 5 elsewhere in the tree and all five are REAL -- 3 in
test/pytest/TESTS.md and 2 in design/, neither in scope. Not fixed here; the rule
does not reach them and widening the scope is a separate decision.

Proved by removal: splicing the commandprompt#1022 shape into configuration.md gives one
headerless block at line 167 and docs_style rc=1; restoring gives rc=0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

REBASED onto 6f7691b after commandprompt#1023 merged, and the resolution fixed a defect commandprompt#1023
left in main. Both conflict hunks were keep-both -- a TOC line and a whole new
section -- but placing mine correctly meant reading the order, and main's TOC was
not in order:

    main's TOC      ... 29, 31, 30      (out of order at two transitions)
    main's sections ... 29, 30, 31      (contiguous, correct)

link arms assert every contents-list link RESOLVES, which both orders do, and
nothing asserts the numbering is monotonic. Now:

    TOC       32 entries, 1..32, 0 out-of-order
    sections  32 sections, 1..32, 0 out-of-order

guard_tests re-derived by collection on the new base rather than carried: 282
collected. cluster_tests 202 comes from main unchanged.

Re-verified after the rebase: 282 passed / 688 checks with --pgc-expect-tests 282
armed; docs_style.sh PASSED (11 checks); the headerless rule is active in the report
(`3 headerless table` on TESTS.md, all three pre-existing and outside the gate's
scope) and `violations()` returns its 5 members.

AND THE GUARD FOR THE ORDERING, added here rather than in commandprompt#1029 because a guard
belongs with its fix. Putting it in commandprompt#1029 would have reddened commandprompt#1029 until this PR
merged, which is a dependency between two independent PRs.

Two arms in test_docs_cover_the_corpus.py. The first requires TESTS.md's contents
numbers and its section numbers each to count 1..N with no gap, and one contents entry
per section. The second is the removal proof on a fixture, using the 29, 31, 30 shape
that shipped rather than a single swap.

MEASURED, by restoring main's order under the new arm:

    the new arm    FAIL  got '[(29, 31), (31, 30), (30, 32)]' want 'none'
    the link arms  1 passed

So the existing link sweep is not a weaker version of this rule. It asks whether a
link RESOLVES and both orders resolve, which is why a shuffled contents list was
outside every arm in that file.

The gap rule catches the section-number COLLISION too, which is the cause rather than
a coincidence: three open PRs each claimed a number another had taken, and a number
used twice leaves a gap in the section sequence. One rule reddens on a duplicate and
on an omission, and the fixture names them apart -- a missing entry gives [(1, 3)] and
a shuffle gives [(1, 3), (3, 2)].

guard_tests 282 -> 284, re-derived by collection. Re-verified: 284 passed / 698
checks with --pgc-expect-tests 284 armed; docs_style.sh PASSED (11 checks); TOC and
sections both 1..32 with zero out-of-order transitions. The CHANGELOG's long-sentence
count is unchanged at 857.

REBASED AGAIN onto 03c6c9c, and the section moved 32 -> 33 because commandprompt#1027 merged and
took 32. That is the collision I measured on commandprompt#1027 before it merged, arriving exactly
as predicted, and the renumbering is the one-hunk-plus-a-sort it was said to be.

Main's contents list is still out of numeric order at three transitions --
(29, 31), (31, 30), (30, 32) -- because commandprompt#1023's entry for section 30 landed after 31
and commandprompt#1027 then appended 32. This resolution sorts all of it:

    TOC       33 entries, 1..33, 0 out-of-order
    sections  33 sections, 1..33, 0 out-of-order

guard_tests 284, re-derived by collection rather than carried. cluster_tests 205 comes
from main unchanged. Re-verified after both rebases: 284 collected, 284 passed / 698
checks with --pgc-expect-tests 284 armed, docs_style.sh PASSED (11 checks), and the
CHANGELOG's long-sentence count matches main at 860.

REBASED onto 14c9dd4 after commandprompt#1029 merged, and the resolution is the one this PR's
sibling work is about. `expected_tests.txt` conflicted and keep-both produced TWO
guard_tests lines:

    guard_tests 280     <- main, after commandprompt#1029
    guard_tests 284     <- this branch, from the old base

Keep-both is right for the CHANGELOG hunk in the same merge and wrong for a key-value
file. One line, re-derived by collection rather than by adding: 287 collected.

Re-verified: 287 passed / 707 checks with --pgc-expect-tests 287 armed; docs_style.sh
PASSED (11 checks); TOC and sections both 1..33 with zero out-of-order; both CHANGELOG
entries survived the merge.
OffgridwithJD added a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 13, 2026
…pt#1026)

docs_style.sh enforced seven rules over every user-facing page and all seven are
about PROSE: sentence length, the idiom list, em and en dashes, prose
double-hyphens, conflict markers, the nav entry, VERSION citations. Nothing checked
structure, so a markdown table that had stopped being a table passed the gate whose
whole purpose is keeping those pages readable.

The measured case, on commandprompt#1022: a note and a second table spliced into the middle of
configuration.md's set_options argument table left six of the nine arguments as a
headerless block. docs_style.sh passed, 14 checks. Found in review by
@linuxhikerpm -- I had reviewed that change twice, checking sentence length, a
guard's scoping claim and a three-row mutation table, and never once asked whether
the markdown still rendered.

Second splice of the day. The first gave configuration.md's GUC table no blank
lines, which made an awk RS='' guard read two GUC rows as one record and pass on
main. Both are the same fact: a markdown table is a contiguous run of | lines, and
a blank line is structural.

FENCES TRACKED BY LINE, not stripped by regex. The regex form already in this file
is fine for counting but loses line numbers, and a report that cannot say WHERE is
one somebody has to re-derive. It also breaks on an unclosed fence, where
state-tracking under-reports instead -- the safe direction, since it cannot invent
a table.

FIVE ARMS, and the control comes first: a rule flagging every table would catch the
defect and be switched off the same day. Then the defect with its line number, a
shell pipeline in a fence WITH its unfenced control, an unclosed fence, and the
false-positive budget as a standing arm rather than a number measured once.

THE BUDGET, measured before landing: 0 across docs/*.md and README.md, which is the
gate's scope. 5 elsewhere in the tree and all five are REAL -- 3 in
test/pytest/TESTS.md and 2 in design/, neither in scope. Not fixed here; the rule
does not reach them and widening the scope is a separate decision.

Proved by removal: splicing the commandprompt#1022 shape into configuration.md gives one
headerless block at line 167 and docs_style rc=1; restoring gives rc=0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

REBASED onto 6f7691b after commandprompt#1023 merged, and the resolution fixed a defect commandprompt#1023
left in main. Both conflict hunks were keep-both -- a TOC line and a whole new
section -- but placing mine correctly meant reading the order, and main's TOC was
not in order:

    main's TOC      ... 29, 31, 30      (out of order at two transitions)
    main's sections ... 29, 30, 31      (contiguous, correct)

link arms assert every contents-list link RESOLVES, which both orders do, and
nothing asserts the numbering is monotonic. Now:

    TOC       32 entries, 1..32, 0 out-of-order
    sections  32 sections, 1..32, 0 out-of-order

guard_tests re-derived by collection on the new base rather than carried: 282
collected. cluster_tests 202 comes from main unchanged.

Re-verified after the rebase: 282 passed / 688 checks with --pgc-expect-tests 282
armed; docs_style.sh PASSED (11 checks); the headerless rule is active in the report
(`3 headerless table` on TESTS.md, all three pre-existing and outside the gate's
scope) and `violations()` returns its 5 members.

AND THE GUARD FOR THE ORDERING, added here rather than in commandprompt#1029 because a guard
belongs with its fix. Putting it in commandprompt#1029 would have reddened commandprompt#1029 until this PR
merged, which is a dependency between two independent PRs.

Two arms in test_docs_cover_the_corpus.py. The first requires TESTS.md's contents
numbers and its section numbers each to count 1..N with no gap, and one contents entry
per section. The second is the removal proof on a fixture, using the 29, 31, 30 shape
that shipped rather than a single swap.

MEASURED, by restoring main's order under the new arm:

    the new arm    FAIL  got '[(29, 31), (31, 30), (30, 32)]' want 'none'
    the link arms  1 passed

So the existing link sweep is not a weaker version of this rule. It asks whether a
link RESOLVES and both orders resolve, which is why a shuffled contents list was
outside every arm in that file.

The gap rule catches the section-number COLLISION too, which is the cause rather than
a coincidence: three open PRs each claimed a number another had taken, and a number
used twice leaves a gap in the section sequence. One rule reddens on a duplicate and
on an omission, and the fixture names them apart -- a missing entry gives [(1, 3)] and
a shuffle gives [(1, 3), (3, 2)].

guard_tests 282 -> 284, re-derived by collection. Re-verified: 284 passed / 698
checks with --pgc-expect-tests 284 armed; docs_style.sh PASSED (11 checks); TOC and
sections both 1..32 with zero out-of-order transitions. The CHANGELOG's long-sentence
count is unchanged at 857.

REBASED AGAIN onto 03c6c9c, and the section moved 32 -> 33 because commandprompt#1027 merged and
took 32. That is the collision I measured on commandprompt#1027 before it merged, arriving exactly
as predicted, and the renumbering is the one-hunk-plus-a-sort it was said to be.

Main's contents list is still out of numeric order at three transitions --
(29, 31), (31, 30), (30, 32) -- because commandprompt#1023's entry for section 30 landed after 31
and commandprompt#1027 then appended 32. This resolution sorts all of it:

    TOC       33 entries, 1..33, 0 out-of-order
    sections  33 sections, 1..33, 0 out-of-order

guard_tests 284, re-derived by collection rather than carried. cluster_tests 205 comes
from main unchanged. Re-verified after both rebases: 284 collected, 284 passed / 698
checks with --pgc-expect-tests 284 armed, docs_style.sh PASSED (11 checks), and the
CHANGELOG's long-sentence count matches main at 860.

REBASED onto 14c9dd4 after commandprompt#1029 merged, and the resolution is the one this PR's
sibling work is about. `expected_tests.txt` conflicted and keep-both produced TWO
guard_tests lines:

    guard_tests 280     <- main, after commandprompt#1029
    guard_tests 284     <- this branch, from the old base

Keep-both is right for the CHANGELOG hunk in the same merge and wrong for a key-value
file. One line, re-derived by collection rather than by adding: 287 collected.

Re-verified: 287 passed / 707 checks with --pgc-expect-tests 287 armed; docs_style.sh
PASSED (11 checks); TOC and sections both 1..33 with zero out-of-order; both CHANGELOG
entries survived the merge.
jdatcmd and others added 2 commits September 12, 2026 20:53
main gained test_native_ownership.py, test_stats_privilege.py and
test_docs_cover_the_corpus.py's duplicate-entry arms while this was open.
Three conflicts, resolved by file kind:

- CHANGELOG.md: additive, both sides kept.
- test/pytest/TESTS.md: additive in the body, NOT in the table of contents.
  Both sides numbered a section 31, so test_docs_stripe_floor.py moves to 33.
  One index entry pointed at "#30-...-type-matrix", an anchor main's rename
  to "all seven parts" had already removed; a union keeps it and ships a
  dangling link. Removed.
- test/pytest/expected_tests.txt: a key-value file has no union. Both sides
  had derived guard_tests 280 against a tree holding only their own three
  arms, so keeping either ships 280 against a tree that collects 283.
  Re-derived by collection on the merged tree: 283 and 205.

Verified: the guard half runs green at 283 (697 checks, 0 unrun), and
--pgc-expect-tests=280 refuses it with rc=4, so the number is load-bearing.
cluster_tests 205 is from collection, which is what the flag compares against.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbyGSaU93XYQr8aH4NrUiw
I put the note and its measurement table between `stripe_row_limit` and
`compression` in the set_options argument table. A blank line ends a markdown
table, so the six arguments after it -- compression, compression_level,
encode_effort, sort_by, ttl_column, ttl_interval -- rendered as a headerless
block or as literal pipes.

docs_style.sh passed this page with 14 checks, and passes the fixed page with
14 checks, because all seven of its rules are about prose. That is what #1028
adds a structure rule for; @OffgridwithJD found this reviewing #1022.

The note now follows the table it was splitting, still inside the section that
documents the setting, and the `stripe_row_limit` row points at it the way the
GUC table's row already did.

Verified: #1028's headerless_tables() flags docs/configuration.md at line 175
on the previous commit and flags nothing on this one; main was clean, so this
branch introduced it. Both #1017 guards still pass in both harnesses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbyGSaU93XYQr8aH4NrUiw
#1028 landed while this was open. Three conflicts, resolved by file kind:

- test/pytest/test_harness_deps.py: a python LIST is additive, both entries kept.
- test/pytest/TESTS.md: both sides numbered a section 33, so this branch's
  test_docs_stripe_floor.py becomes 34 and follows #1028's section. #1028's own
  new rule now enforces that ordering, and it reports toc=34 sec=34 with no gap,
  no inversion and no dangling anchor.
- test/pytest/expected_tests.txt: re-derived, not resolved. Three branches had
  each added three arms to 277 and each derived 280, every one correct against a
  tree holding only its own three. Keeping either side ships a value no tree
  collects, and so does adding the deltas up. Collection on the merged tree says
  290 and 205.

Also dropped the note #1028 left under its value, which described a move to 284
beside a value of 287; I said in review I would take that rather than send the
author round again.

Verified: guard half green at 290 (715 checks, 0 unrun), docs_style.sh 14 checks
PASSED.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbyGSaU93XYQr8aH4NrUiw
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 13, 2026
…one (commandprompt#982)

A ledger row is keyed on (suite, part, name). Two checks with the same name in one
part share a row, and two ROWS with the same key collapse into one. Neither is
mis-recorded while everything passes. Both lose a red observation the moment one
appears.

--- THE GATE COULD NOT SEE TWO CHECKS IN ONE RUN -------------------------------

`merge` has printed "duplicate check name in one run" since commandprompt#982 was filed and
returns 0, which is how three of them sat in one part of selftest/400 for a day. The
instance was fixed by c3b13ae; this is the mechanism the issue called the more
valuable half.

`cmd_gate` builds its records as `sorted({(s, p, n, m) for ...})`. A set collapses the
duplicate before any arm can count it, so the same canonicalisation that makes the
rest of that function correct made this one class unreachable. The count now comes
from the raw records through `_by_run`, which already existed for the
two-runs-versus-one-duplicate distinction.

EVERY SUITE, DELIBERATELY UNLIKE THE NEW-CHECK REFUSAL. That one is restricted to
covered suites because it cannot know which of an uncovered suite's checks are new.
This one needs no history: two records, one key, one log is decidable from the log
alone. The ledger covers four suites of 253, so copying the restriction would close
the class in four places only, and the next collision would sit in one of the other
249 until that suite was seeded.

--- AND THE LEDGER COULD NOT SEE TWO ROWS IN ONE FILE --------------------------

`read_ledger` did `rows[key] = [...]`, so a duplicated key in the tracked file
collapsed silently and the LAST line won. Measured on a two-line fixture, both
orders:

    never first, then 2026-09-01   survivor last_red='2026-09-01'
    2026-09-01 first, then never   survivor last_red='never'      the red is GONE

Line order decided whether a recorded red observation survived. A merge that keeps
both sides of a changed row turns `ever red` back into `never`.

NOTHING ELSE COULD CATCH IT, and bounding the census cannot. check_ledger_budget.txt
says `checks_never_observed_red` is a CENSUS and must not become a ceiling, because
every new check enters as `never` and bounding it deadlocks. The gate compares the
budget's number with the ledger's, and both come from the same dict, so they agree
either way. Measured: with the budget regenerated alongside, an erased red passes the
gate at rc=0. It is now refused as an integrity failure (rc=2), beside the other
inputs that do not parse.

So the two halves are the same shape at two levels. A SET hid two checks in one run;
a DICT hid two rows in one file. The question that found both is what the input
canonicalises before the guard sees it.

--- MEASURED BEFORE WIDENING IT -----------------------------------------------

A gate that reddens 250 unmeasured suites is a gate somebody turns off, so the
refusal was not widened on reasoning. A full PG 18 matrix ran with it armed for
every suite:

    247 suites ran, 6 skipped, 0 incomplete | ALL VERSIONS PASSED, RC=0 | 0 shared keys

Check names are static, so one major's matrix measures this class completely rather
than sampling it.

THE LOAD-BEARING EVIDENCE IS THAT RUN, not a log snapshot. An earlier draft of this
message cited "244 logs, 6651 records, 6651 distinct keys, 0 collisions" from
per-suite logs copied out of the build directory while the matrix was still running.
That figure is withdrawn: the copies were taken mid-write, and measuring the same way
again showed 150 of 253 had no `checks run:` line yet. Zero collisions across
truncated logs is not evidence of zero across complete ones. The full-matrix figure
stands because the gate reads the logs after each suite has finished.

It does still answer what commandprompt#982 said nobody had measured -- the gap between "checks
that ran" and "rows the ledger can hold" across the other 250 suites -- because the
gate saw every suite's complete log and refused nothing.

--- VERIFICATION --------------------------------------------------------------

The committed ledger still loads at 1197 rows. The gate against the COMMITTED ledger
and budget over a real 934-record harness_selftest log: rc=0, no shared-key line,
census agrees at 1189. A planted duplicate check: rc=1, naming the key. A planted
duplicate row: rc=2, naming the line and what it would lose.

The refusal arm also asserts no Traceback in the output, because an unhandled
exception exits 1 too and would satisfy an rc check on its own. Measured, it did: an
earlier draft referenced `covered_suites` before it was defined, which compiles and
fails at runtime.

guard_tests 277 -> 283, re-derived by collection. 283 passed / 698 checks with
--pgc-expect-tests 283 armed. docs_style.sh PASSED (11 checks). Long-sentence counts
unchanged: CHANGELOG 860, TESTS.md 319.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

--- AND A THIRD INSTANCE, IN THE ARM THAT POLICES THE OTHER TRACKED FILE -------

test_harness_deps.py read expected_tests.txt with `nums[f[0]] = int(f[1])`, so a
duplicated key collapsed and the last line won -- exactly as read_ledger did, one file
over. Measured on one fixture read both ways:

    the line form   names guard_tests as duplicated
    the dict form   sees two keys and keeps 280, the LAST line

NOT HYPOTHETICAL. Three PRs were open at once, each moving guard_tests, and resolving
all three keep-both produced three of those lines. ci.yml reads the value with
`awk '$1=="guard_tests"{print $2}'`, which prints one line per match. So WANT becomes
multi-line, `test -n "$WANT"` still passes, and the flag refuses it:

    pytest: error: argument --pgc-expect-tests: invalid int value: '284\n280\n283'
    exit 4

It FAILS CLOSED, so these two arms are about legibility rather than a hole. What exit 4
does not say is that a line is duplicated. The removal proof is on the real file: with a
duplicate planted, the new arm reddens and the pre-existing one stays green.

Keep-both is right for a changelog and wrong for a key-value file, and nothing in the
tree said so.

cluster_tests 205 -> 207, NOT guard_tests: test_harness_deps.py DEFINES NO_CLUSTER and
is not in it. I bumped the wrong number first and --pgc-expect-tests caught it --
`collected 283 test(s) but expected 285` -- which is the argument for the mechanism.
Both halves re-derived by collection: guard 283, cluster 207.

REBASED onto 14c9dd4 after commandprompt#1029 merged, and the conflict was the exact shape the third
arm here is about. `expected_tests.txt` conflicted on `guard_tests`:

    guard_tests 280     <- main, after commandprompt#1029
    guard_tests 283     <- this branch, from the old base

Keep-both would have produced two lines and, through ci.yml's
`awk '$1=="guard_tests"{print $2}'`, a multi-line value the flag refuses at exit 4. One
line, and both halves re-derived by collection rather than by arithmetic:

    guard_tests 286, cluster_tests 207

Re-verified: 286 passed / 707 checks with --pgc-expect-tests 286 armed; docs_style.sh
PASSED (11 checks); the gate against the committed ledger and budget over the real
934-record log still returns 0. Long-sentence counts unchanged against the new base:
CHANGELOG 860, TESTS.md 319.

--- FIXTURE CORRECTED AFTER REVIEW (@jdatcmd) -----------------------------------

test_the_gate_refuses_two_checks_sharing_one_ledger_key asserted rc == 1 on a
fixture where the gate returned 1 for TWO independent reasons, so the arm did not
discriminate. Found by mutation, and I reproduced it before changing anything:

    with `rc = 1` deleted from the shared-key block, the OLD fixture's arm PASSED
    with the same mutation, the corrected fixture's arm FAILS: got 0 want 1

The second route was the pre-existing new-check refusal. The ledger named only
`some other check` in that part, which made `demo` a covered suite whose log carried
a check the ledger had never seen:

    not in the ledger: demo   part1   shared name   (on major 18)

The ledger now NAMES `shared name`, so that refusal has nothing to say and only the
shared-key refusal can move rc.

This is the same defect class the PR is about, in my own arm: an assertion that
passes for a reason other than the one it is named for. My docstring claimed I had
measured this and closed it -- I had closed the COVERAGE-CEILING route and not the
new-check route, and the note said "routes" as though it were all of them. The
comment now enumerates all three, including the unhandled-exception route that also
exits 1.

It also described the refusal as "restricted to suites a row exists for", which was
the design I abandoned two pushes earlier when the matrix measurement said refusing
everywhere was safe. That sentence is gone.

--- REBASED onto 2ea7366 ------------------------------------------------------

guard_tests 293 and cluster_tests 207, both re-derived by COLLECTION on this base.

And a sharper point than the one I had, measured by @jdatcmd on commandprompt#1022: two branches
each derived `guard_tests 280` correctly, each against a tree holding only its own
three arms, and the merged value was 283. Both numbers were right for the tree they
were taken on and both were wrong for the merge. So "derive rather than add" is
necessary and not sufficient -- it has to be re-derived after every rebase, which
expected_tests.txt now says.

Re-verified: 293 passed / 725 checks with --pgc-expect-tests 293 armed; docs_style.sh
PASSED (11 checks); the gate against the committed ledger and budget over the real
934-record log returns 0; long-sentence counts unchanged at CHANGELOG 860 and
TESTS.md 319.

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at 5e07e392. 14 checks, 0 failures, CLEAN. I verified the merged tree rather
than the diff.

The headerless table is fixed, checked with the rule that found it rather than by reading
the diff:

#1022 at 1747cc7e   headerless tables in docs/configuration.md: [175]
main                headerless tables: []          (it was never on main)
#1022 at 5e07e392   headerless tables: []          fixed
across all of docs/ on the merged tree: 0

5928438a moved the 1024 note out of the middle of the set_options argument table, so the
six arguments that had been orphaned are under the header again. That is #1026's motivating
case closed in the file it came from.

The conflict resolution is right on all three counts I raised.

section number    31 -> 33, correct against main's 32 (test_stats_privilege)
stale TOC link    [30. ... type matrix](#30-...-type-matrix) removed, main's
                  "all seven parts" entry kept in its place
TOC order         inherited from main, and main is now in order since #1028 merged

I nearly reported the second one as unfixed. grep 'type matrix' still returns 2 hits, and
I read that as the stale entry surviving. They are prose — "the type matrix, the boundary
conditions, the lightweight encodings"
and "Part 1, the type matrix, is twenty columns"
present identically on main at the same lines, from #1023's rework. The stale thing was the
LINK, and grep -E '^- \[.*\]\(#.*type-matrix' returns nothing on your tree. I checked before
saying anything, which is the only reason this is a note and not a wrong finding.

And your tree is now the first to be graded by #1028's arms, which merged after you pushed:

TOC        34 entries, 1..34, 0 out-of-order
sections   34 sections, 1..34, 0 out-of-order
the numbering + shuffle + headerless arms    7 passed / 18 checks

So the renumbering is not just consistent by inspection, it is gated.

Counts derived, not added. guard_tests 290 on one line, matching 290 tests collected
by derivation on the merged base. cluster_tests 205 from main unchanged.

full guard half   290 passed / 715 checks, --pgc-expect-tests 290 armed
docs_style.sh     PASSED, 14 checks (11 from main + your 3)

Your guard_tests 280 observation on #1035 is the sharper version of the rule I had, and I
have folded it into expected_tests.txt there: two branches each derived 280 correctly against
a tree holding only their own arms, and 283 was the merged truth. Derivation is necessary and
not sufficient — it has to be re-derived after every rebase, which is exactly what you did here.

@OffgridwithJD

Copy link
Copy Markdown
Collaborator

@linuxhikerpm both of your holds are resolved at 5e07e392, and I verified the second one with
the rule rather than by reading the diff.

Hold 1 (guard_tests not bumped) — you confirmed this yourself at 1747cc7e. On the
merged tree it is now 290, one line, matching 290 tests collected by derivation, and the
guard half runs 290 passed / 715 checks with --pgc-expect-tests 290 armed.

Hold 2 (the FSST note splicing the set_options table) — fixed by 5928438a, which moved
the note out of the middle of the table:

#1022 at 1747cc7e   docs/configuration.md headerless tables: [175]
#1022 at 5e07e392   headerless tables: []
across all of docs/ on the merged tree: 0

Line 175 is the block you named: | compression | through | ttl_interval |, the six
arguments that were rendering without a header.

And your prediction about #1028 is confirmed, which is worth recording because it is the
gate's first real case.
You wrote:

#1028 adds a gate that will fail docs_style.sh on this splice the moment that lands, so the
table has to move anyway.

#1028 landed in 2ea73661. Measured by putting the spliced file back on top of it:

main + the new gate, own docs          PASSED, 11 checks
same gate, spliced configuration.md    FAILED, exit 1
                                       "headerless table block starting at line 175"
                                       "every user-facing document meets the measurable
                                        plain-language rules: got [1] want [0]"

So the rule catches the exact defect, in the exact file, at the exact line you identified, and
fails the gate rather than printing a note. You found this by reading twice, after I had walked
past it twice; that is what #1026 and the rule exist for.

One correction to something I nearly said. grep 'type matrix' still returns 2 hits in
TESTS.md and I briefly read that as the stale contents entry surviving the merge. They are
prose from #1023's rework, present identically on main at the same lines. The stale thing was
the LINK, and grep -E '^- \[.*\]\(#.*type-matrix' returns nothing on this tree.

I have approved at 5e07e392. Your CHANGES_REQUESTED is still recorded against the older
heads, so the PR needs your re-review to clear it — I am not in a position to dismiss it and
would not want to be.

@jdatcmd
jdatcmd merged commit 1d76ccf into main Sep 13, 2026
14 checks passed
@jdatcmd
jdatcmd deleted the docs/1017-the-stripe-floor-is-below-a-vector branch September 13, 2026 03:24
jdatcmd added a commit that referenced this pull request Sep 13, 2026
#1022 landed while this was open. Two conflicts:

- test/pytest/TESTS.md: main's test_docs_stripe_floor.py took 34, so this
  branch's two sections become 35 and 36. #1028's numbering rule reports
  toc=36 sec=36, no gap, no inversion, no dangling anchor.
- test/pytest/expected_tests.txt: re-derived, not resolved. Both sides had a
  guard number measured against a tree holding only its own arms. Collection on
  the merged tree says 298 and 217.

Verified: guard half green at 298 (737 checks, 0 unrun).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbyGSaU93XYQr8aH4NrUiw
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 13, 2026
…one (commandprompt#982)

A ledger row is keyed on (suite, part, name). Two checks with the same name in one
part share a row, and two ROWS with the same key collapse into one. Neither is
mis-recorded while everything passes. Both lose a red observation the moment one
appears.

--- THE GATE COULD NOT SEE TWO CHECKS IN ONE RUN -------------------------------

`merge` has printed "duplicate check name in one run" since commandprompt#982 was filed and
returns 0, which is how three of them sat in one part of selftest/400 for a day. The
instance was fixed by c3b13ae; this is the mechanism the issue called the more
valuable half.

`cmd_gate` builds its records as `sorted({(s, p, n, m) for ...})`. A set collapses the
duplicate before any arm can count it, so the same canonicalisation that makes the
rest of that function correct made this one class unreachable. The count now comes
from the raw records through `_by_run`, which already existed for the
two-runs-versus-one-duplicate distinction.

EVERY SUITE, DELIBERATELY UNLIKE THE NEW-CHECK REFUSAL. That one is restricted to
covered suites because it cannot know which of an uncovered suite's checks are new.
This one needs no history: two records, one key, one log is decidable from the log
alone. The ledger covers four suites of 253, so copying the restriction would close
the class in four places only, and the next collision would sit in one of the other
249 until that suite was seeded.

--- AND THE LEDGER COULD NOT SEE TWO ROWS IN ONE FILE --------------------------

`read_ledger` did `rows[key] = [...]`, so a duplicated key in the tracked file
collapsed silently and the LAST line won. Measured on a two-line fixture, both
orders:

    never first, then 2026-09-01   survivor last_red='2026-09-01'
    2026-09-01 first, then never   survivor last_red='never'      the red is GONE

Line order decided whether a recorded red observation survived. A merge that keeps
both sides of a changed row turns `ever red` back into `never`.

NOTHING ELSE COULD CATCH IT, and bounding the census cannot. check_ledger_budget.txt
says `checks_never_observed_red` is a CENSUS and must not become a ceiling, because
every new check enters as `never` and bounding it deadlocks. The gate compares the
budget's number with the ledger's, and both come from the same dict, so they agree
either way. Measured: with the budget regenerated alongside, an erased red passes the
gate at rc=0. It is now refused as an integrity failure (rc=2), beside the other
inputs that do not parse.

So the two halves are the same shape at two levels. A SET hid two checks in one run;
a DICT hid two rows in one file. The question that found both is what the input
canonicalises before the guard sees it.

--- MEASURED BEFORE WIDENING IT -----------------------------------------------

A gate that reddens 250 unmeasured suites is a gate somebody turns off, so the
refusal was not widened on reasoning. A full PG 18 matrix ran with it armed for
every suite:

    247 suites ran, 6 skipped, 0 incomplete | ALL VERSIONS PASSED, RC=0 | 0 shared keys

Check names are static, so one major's matrix measures this class completely rather
than sampling it.

THE LOAD-BEARING EVIDENCE IS THAT RUN, not a log snapshot. An earlier draft of this
message cited "244 logs, 6651 records, 6651 distinct keys, 0 collisions" from
per-suite logs copied out of the build directory while the matrix was still running.
That figure is withdrawn: the copies were taken mid-write, and measuring the same way
again showed 150 of 253 had no `checks run:` line yet. Zero collisions across
truncated logs is not evidence of zero across complete ones. The full-matrix figure
stands because the gate reads the logs after each suite has finished.

It does still answer what commandprompt#982 said nobody had measured -- the gap between "checks
that ran" and "rows the ledger can hold" across the other 250 suites -- because the
gate saw every suite's complete log and refused nothing.

--- VERIFICATION --------------------------------------------------------------

The committed ledger still loads at 1197 rows. The gate against the COMMITTED ledger
and budget over a real 934-record harness_selftest log: rc=0, no shared-key line,
census agrees at 1189. A planted duplicate check: rc=1, naming the key. A planted
duplicate row: rc=2, naming the line and what it would lose.

The refusal arm also asserts no Traceback in the output, because an unhandled
exception exits 1 too and would satisfy an rc check on its own. Measured, it did: an
earlier draft referenced `covered_suites` before it was defined, which compiles and
fails at runtime.

guard_tests 277 -> 283, re-derived by collection. 283 passed / 698 checks with
--pgc-expect-tests 283 armed. docs_style.sh PASSED (11 checks). Long-sentence counts
unchanged: CHANGELOG 860, TESTS.md 319.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

--- AND A THIRD INSTANCE, IN THE ARM THAT POLICES THE OTHER TRACKED FILE -------

test_harness_deps.py read expected_tests.txt with `nums[f[0]] = int(f[1])`, so a
duplicated key collapsed and the last line won -- exactly as read_ledger did, one file
over. Measured on one fixture read both ways:

    the line form   names guard_tests as duplicated
    the dict form   sees two keys and keeps 280, the LAST line

NOT HYPOTHETICAL. Three PRs were open at once, each moving guard_tests, and resolving
all three keep-both produced three of those lines. ci.yml reads the value with
`awk '$1=="guard_tests"{print $2}'`, which prints one line per match. So WANT becomes
multi-line, `test -n "$WANT"` still passes, and the flag refuses it:

    pytest: error: argument --pgc-expect-tests: invalid int value: '284\n280\n283'
    exit 4

It FAILS CLOSED, so these two arms are about legibility rather than a hole. What exit 4
does not say is that a line is duplicated. The removal proof is on the real file: with a
duplicate planted, the new arm reddens and the pre-existing one stays green.

Keep-both is right for a changelog and wrong for a key-value file, and nothing in the
tree said so.

cluster_tests 205 -> 207, NOT guard_tests: test_harness_deps.py DEFINES NO_CLUSTER and
is not in it. I bumped the wrong number first and --pgc-expect-tests caught it --
`collected 283 test(s) but expected 285` -- which is the argument for the mechanism.
Both halves re-derived by collection: guard 283, cluster 207.

REBASED onto 14c9dd4 after commandprompt#1029 merged, and the conflict was the exact shape the third
arm here is about. `expected_tests.txt` conflicted on `guard_tests`:

    guard_tests 280     <- main, after commandprompt#1029
    guard_tests 283     <- this branch, from the old base

Keep-both would have produced two lines and, through ci.yml's
`awk '$1=="guard_tests"{print $2}'`, a multi-line value the flag refuses at exit 4. One
line, and both halves re-derived by collection rather than by arithmetic:

    guard_tests 286, cluster_tests 207

Re-verified: 286 passed / 707 checks with --pgc-expect-tests 286 armed; docs_style.sh
PASSED (11 checks); the gate against the committed ledger and budget over the real
934-record log still returns 0. Long-sentence counts unchanged against the new base:
CHANGELOG 860, TESTS.md 319.

--- FIXTURE CORRECTED AFTER REVIEW (@jdatcmd) -----------------------------------

test_the_gate_refuses_two_checks_sharing_one_ledger_key asserted rc == 1 on a
fixture where the gate returned 1 for TWO independent reasons, so the arm did not
discriminate. Found by mutation, and I reproduced it before changing anything:

    with `rc = 1` deleted from the shared-key block, the OLD fixture's arm PASSED
    with the same mutation, the corrected fixture's arm FAILS: got 0 want 1

The second route was the pre-existing new-check refusal. The ledger named only
`some other check` in that part, which made `demo` a covered suite whose log carried
a check the ledger had never seen:

    not in the ledger: demo   part1   shared name   (on major 18)

The ledger now NAMES `shared name`, so that refusal has nothing to say and only the
shared-key refusal can move rc.

This is the same defect class the PR is about, in my own arm: an assertion that
passes for a reason other than the one it is named for. My docstring claimed I had
measured this and closed it -- I had closed the COVERAGE-CEILING route and not the
new-check route, and the note said "routes" as though it were all of them. The
comment now enumerates all three, including the unhandled-exception route that also
exits 1.

It also described the refusal as "restricted to suites a row exists for", which was
the design I abandoned two pushes earlier when the matrix measurement said refusing
everywhere was safe. That sentence is gone.

--- REBASED onto 2ea7366 ------------------------------------------------------

guard_tests 293 and cluster_tests 207, both re-derived by COLLECTION on this base.

And a sharper point than the one I had, measured by @jdatcmd on commandprompt#1022: two branches
each derived `guard_tests 280` correctly, each against a tree holding only its own
three arms, and the merged value was 283. Both numbers were right for the tree they
were taken on and both were wrong for the merge. So "derive rather than add" is
necessary and not sufficient -- it has to be re-derived after every rebase, which
expected_tests.txt now says.

Re-verified: 293 passed / 725 checks with --pgc-expect-tests 293 armed; docs_style.sh
PASSED (11 checks); the gate against the committed ledger and budget over the real
934-record log returns 0; long-sentence counts unchanged at CHANGELOG 860 and
TESTS.md 319.

REBASED again onto 1d76ccf after commandprompt#1022 merged. Two hunks in expected_tests.txt, and the
second one is worth a note: my side carried the new `cluster_tests 207` comment PLUS two
stale fragments left by earlier keep-both resolutions of the same file ("177 -> 180",
"166 -> 182"), describing values the file no longer holds. Resolved to ONE coherent block
rather than the union, which is the same lesson the arms here are about: keep-both is
right for a changelog and wrong for a file where each key appears once.

guard_tests 296 and cluster_tests 207, both re-derived by collection on this base.
Re-verified: 296 passed / 733 checks armed; docs_style.sh PASSED; the gate over the real
934-record log returns 0.
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 13, 2026
…one (commandprompt#982)

A ledger row is keyed on (suite, part, name). Two checks with the same name in one
part share a row, and two ROWS with the same key collapse into one. Neither is
mis-recorded while everything passes. Both lose a red observation the moment one
appears.

--- THE GATE COULD NOT SEE TWO CHECKS IN ONE RUN -------------------------------

`merge` has printed "duplicate check name in one run" since commandprompt#982 was filed and
returns 0, which is how three of them sat in one part of selftest/400 for a day. The
instance was fixed by c3b13ae; this is the mechanism the issue called the more
valuable half.

`cmd_gate` builds its records as `sorted({(s, p, n, m) for ...})`. A set collapses the
duplicate before any arm can count it, so the same canonicalisation that makes the
rest of that function correct made this one class unreachable. The count now comes
from the raw records through `_by_run`, which already existed for the
two-runs-versus-one-duplicate distinction.

EVERY SUITE, DELIBERATELY UNLIKE THE NEW-CHECK REFUSAL. That one is restricted to
covered suites because it cannot know which of an uncovered suite's checks are new.
This one needs no history: two records, one key, one log is decidable from the log
alone. The ledger covers four suites of 253, so copying the restriction would close
the class in four places only, and the next collision would sit in one of the other
249 until that suite was seeded.

--- AND THE LEDGER COULD NOT SEE TWO ROWS IN ONE FILE --------------------------

`read_ledger` did `rows[key] = [...]`, so a duplicated key in the tracked file
collapsed silently and the LAST line won. Measured on a two-line fixture, both
orders:

    never first, then 2026-09-01   survivor last_red='2026-09-01'
    2026-09-01 first, then never   survivor last_red='never'      the red is GONE

Line order decided whether a recorded red observation survived. A merge that keeps
both sides of a changed row turns `ever red` back into `never`.

NOTHING ELSE COULD CATCH IT, and bounding the census cannot. check_ledger_budget.txt
says `checks_never_observed_red` is a CENSUS and must not become a ceiling, because
every new check enters as `never` and bounding it deadlocks. The gate compares the
budget's number with the ledger's, and both come from the same dict, so they agree
either way. Measured: with the budget regenerated alongside, an erased red passes the
gate at rc=0. It is now refused as an integrity failure (rc=2), beside the other
inputs that do not parse.

So the two halves are the same shape at two levels. A SET hid two checks in one run;
a DICT hid two rows in one file. The question that found both is what the input
canonicalises before the guard sees it.

--- MEASURED BEFORE WIDENING IT -----------------------------------------------

A gate that reddens 250 unmeasured suites is a gate somebody turns off, so the
refusal was not widened on reasoning. A full PG 18 matrix ran with it armed for
every suite:

    247 suites ran, 6 skipped, 0 incomplete | ALL VERSIONS PASSED, RC=0 | 0 shared keys

Check names are static, so one major's matrix measures this class completely rather
than sampling it.

THE LOAD-BEARING EVIDENCE IS THAT RUN, not a log snapshot. An earlier draft of this
message cited "244 logs, 6651 records, 6651 distinct keys, 0 collisions" from
per-suite logs copied out of the build directory while the matrix was still running.
That figure is withdrawn: the copies were taken mid-write, and measuring the same way
again showed 150 of 253 had no `checks run:` line yet. Zero collisions across
truncated logs is not evidence of zero across complete ones. The full-matrix figure
stands because the gate reads the logs after each suite has finished.

It does still answer what commandprompt#982 said nobody had measured -- the gap between "checks
that ran" and "rows the ledger can hold" across the other 250 suites -- because the
gate saw every suite's complete log and refused nothing.

--- VERIFICATION --------------------------------------------------------------

The committed ledger still loads at 1197 rows. The gate against the COMMITTED ledger
and budget over a real 934-record harness_selftest log: rc=0, no shared-key line,
census agrees at 1189. A planted duplicate check: rc=1, naming the key. A planted
duplicate row: rc=2, naming the line and what it would lose.

The refusal arm also asserts no Traceback in the output, because an unhandled
exception exits 1 too and would satisfy an rc check on its own. Measured, it did: an
earlier draft referenced `covered_suites` before it was defined, which compiles and
fails at runtime.

guard_tests 277 -> 283, re-derived by collection. 283 passed / 698 checks with
--pgc-expect-tests 283 armed. docs_style.sh PASSED (11 checks). Long-sentence counts
unchanged: CHANGELOG 860, TESTS.md 319.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

--- AND A THIRD INSTANCE, IN THE ARM THAT POLICES THE OTHER TRACKED FILE -------

test_harness_deps.py read expected_tests.txt with `nums[f[0]] = int(f[1])`, so a
duplicated key collapsed and the last line won -- exactly as read_ledger did, one file
over. Measured on one fixture read both ways:

    the line form   names guard_tests as duplicated
    the dict form   sees two keys and keeps 280, the LAST line

NOT HYPOTHETICAL. Three PRs were open at once, each moving guard_tests, and resolving
all three keep-both produced three of those lines. ci.yml reads the value with
`awk '$1=="guard_tests"{print $2}'`, which prints one line per match. So WANT becomes
multi-line, `test -n "$WANT"` still passes, and the flag refuses it:

    pytest: error: argument --pgc-expect-tests: invalid int value: '284\n280\n283'
    exit 4

It FAILS CLOSED, so these two arms are about legibility rather than a hole. What exit 4
does not say is that a line is duplicated. The removal proof is on the real file: with a
duplicate planted, the new arm reddens and the pre-existing one stays green.

Keep-both is right for a changelog and wrong for a key-value file, and nothing in the
tree said so.

cluster_tests 205 -> 207, NOT guard_tests: test_harness_deps.py DEFINES NO_CLUSTER and
is not in it. I bumped the wrong number first and --pgc-expect-tests caught it --
`collected 283 test(s) but expected 285` -- which is the argument for the mechanism.
Both halves re-derived by collection: guard 283, cluster 207.

REBASED onto 14c9dd4 after commandprompt#1029 merged, and the conflict was the exact shape the third
arm here is about. `expected_tests.txt` conflicted on `guard_tests`:

    guard_tests 280     <- main, after commandprompt#1029
    guard_tests 283     <- this branch, from the old base

Keep-both would have produced two lines and, through ci.yml's
`awk '$1=="guard_tests"{print $2}'`, a multi-line value the flag refuses at exit 4. One
line, and both halves re-derived by collection rather than by arithmetic:

    guard_tests 286, cluster_tests 207

Re-verified: 286 passed / 707 checks with --pgc-expect-tests 286 armed; docs_style.sh
PASSED (11 checks); the gate against the committed ledger and budget over the real
934-record log still returns 0. Long-sentence counts unchanged against the new base:
CHANGELOG 860, TESTS.md 319.

--- FIXTURE CORRECTED AFTER REVIEW (@jdatcmd) -----------------------------------

test_the_gate_refuses_two_checks_sharing_one_ledger_key asserted rc == 1 on a
fixture where the gate returned 1 for TWO independent reasons, so the arm did not
discriminate. Found by mutation, and I reproduced it before changing anything:

    with `rc = 1` deleted from the shared-key block, the OLD fixture's arm PASSED
    with the same mutation, the corrected fixture's arm FAILS: got 0 want 1

The second route was the pre-existing new-check refusal. The ledger named only
`some other check` in that part, which made `demo` a covered suite whose log carried
a check the ledger had never seen:

    not in the ledger: demo   part1   shared name   (on major 18)

The ledger now NAMES `shared name`, so that refusal has nothing to say and only the
shared-key refusal can move rc.

This is the same defect class the PR is about, in my own arm: an assertion that
passes for a reason other than the one it is named for. My docstring claimed I had
measured this and closed it -- I had closed the COVERAGE-CEILING route and not the
new-check route, and the note said "routes" as though it were all of them. The
comment now enumerates all three, including the unhandled-exception route that also
exits 1.

It also described the refusal as "restricted to suites a row exists for", which was
the design I abandoned two pushes earlier when the matrix measurement said refusing
everywhere was safe. That sentence is gone.

--- REBASED onto 2ea7366 ------------------------------------------------------

guard_tests 293 and cluster_tests 207, both re-derived by COLLECTION on this base.

And a sharper point than the one I had, measured by @jdatcmd on commandprompt#1022: two branches
each derived `guard_tests 280` correctly, each against a tree holding only its own
three arms, and the merged value was 283. Both numbers were right for the tree they
were taken on and both were wrong for the merge. So "derive rather than add" is
necessary and not sufficient -- it has to be re-derived after every rebase, which
expected_tests.txt now says.

Re-verified: 293 passed / 725 checks with --pgc-expect-tests 293 armed; docs_style.sh
PASSED (11 checks); the gate against the committed ledger and budget over the real
934-record log returns 0; long-sentence counts unchanged at CHANGELOG 860 and
TESTS.md 319.

REBASED again onto 1d76ccf after commandprompt#1022 merged. Two hunks in expected_tests.txt, and the
second one is worth a note: my side carried the new `cluster_tests 207` comment PLUS two
stale fragments left by earlier keep-both resolutions of the same file ("177 -> 180",
"166 -> 182"), describing values the file no longer holds. Resolved to ONE coherent block
rather than the union, which is the same lesson the arms here are about: keep-both is
right for a changelog and wrong for a file where each key appears once.

guard_tests 296 and cluster_tests 207, both re-derived by collection on this base.
Re-verified: 296 passed / 733 checks armed; docs_style.sh PASSED; the gate over the real
934-record log returns 0.
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 13, 2026
…ndprompt#1031)

`Expect.rows` documents the argument as taking "a REASON, not a flag", and gives the
rationale: the escape hatch should cost more to type than the honest assertion. One
line below sat a truthiness test:

    if _empty(got) and _empty(want) and not allow_empty:

So `allow_empty=True` satisfied it and carried nothing, and the hatch cost LESS than
the assertion rather than more. Measured before the refusal: `allow_empty=True` and
`allow_empty=1` both passed, 3 passed. `row_set` forwards the argument, so it
inherited the hole and has its own arm asserting it does not route around the refusal.

THE CHECK FIRES WHENEVER THE ARGUMENT IS GIVEN, not only when both sides turn out to
be empty. Otherwise a flag form in a test whose sides happen to be non-empty passes
today and refuses on the day the data changes, which is the worst moment to learn it.

TWO LIVE SITES USED THE FLAG FORM AND BOTH WERE SUBSTANTIVELY CORRECT. Each had its
population premise on the line above, and test_check_records.py even stated the
argument in a comment: "an empty offender list is the answer to both, and only one of
them is good news." So nothing was hiding behind the hatch. The cost fell on the next
reader: the hatch exists so every empty-on-both-sides comparison carries its
justification where an audit of `allow_empty=` can read it, and half of them carried
none. Both now do, using the reason that was already in the comment.

Pinned in test_guards_pinned.py, which exists so a refusal is asserted by its message
rather than by "something failed" -- built after a census found 12 of 17 guards
deletable with the suite still green. Three arms: the refusal, the accepted form as
its control, and row_set inheriting it.

guard_tests 290 -> 293, derived by collection. The comment above the number now says
to RE-DERIVE AFTER EVERY REBASE rather than once: two branches can each derive
correctly against a tree holding only their own arms and both be wrong for the merge,
which @jdatcmd measured on commandprompt#1022 where two independent derivations of 280 had 283 as
the merged truth.

Verified: 293 passed / 718 checks with --pgc-expect-tests 293 armed; the layer's own
files 89 passed / 133 checks; docs_style.sh PASSED (14 checks); long-sentence counts
unchanged at CHANGELOG 868 and TESTS.md 324.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 13, 2026
…one (commandprompt#982)

A ledger row is keyed on (suite, part, name). Two checks with the same name in one
part share a row, and two ROWS with the same key collapse into one. Neither is
mis-recorded while everything passes. Both lose a red observation the moment one
appears.

--- THE GATE COULD NOT SEE TWO CHECKS IN ONE RUN -------------------------------

`merge` has printed "duplicate check name in one run" since commandprompt#982 was filed and
returns 0, which is how three of them sat in one part of selftest/400 for a day. The
instance was fixed by c3b13ae; this is the mechanism the issue called the more
valuable half.

`cmd_gate` builds its records as `sorted({(s, p, n, m) for ...})`. A set collapses the
duplicate before any arm can count it, so the same canonicalisation that makes the
rest of that function correct made this one class unreachable. The count now comes
from the raw records through `_by_run`, which already existed for the
two-runs-versus-one-duplicate distinction.

EVERY SUITE, DELIBERATELY UNLIKE THE NEW-CHECK REFUSAL. That one is restricted to
covered suites because it cannot know which of an uncovered suite's checks are new.
This one needs no history: two records, one key, one log is decidable from the log
alone. The ledger covers four suites of 253, so copying the restriction would close
the class in four places only, and the next collision would sit in one of the other
249 until that suite was seeded.

--- AND THE LEDGER COULD NOT SEE TWO ROWS IN ONE FILE --------------------------

`read_ledger` did `rows[key] = [...]`, so a duplicated key in the tracked file
collapsed silently and the LAST line won. Measured on a two-line fixture, both
orders:

    never first, then 2026-09-01   survivor last_red='2026-09-01'
    2026-09-01 first, then never   survivor last_red='never'      the red is GONE

Line order decided whether a recorded red observation survived. A merge that keeps
both sides of a changed row turns `ever red` back into `never`.

NOTHING ELSE COULD CATCH IT, and bounding the census cannot. check_ledger_budget.txt
says `checks_never_observed_red` is a CENSUS and must not become a ceiling, because
every new check enters as `never` and bounding it deadlocks. The gate compares the
budget's number with the ledger's, and both come from the same dict, so they agree
either way. Measured: with the budget regenerated alongside, an erased red passes the
gate at rc=0. It is now refused as an integrity failure (rc=2), beside the other
inputs that do not parse.

So the two halves are the same shape at two levels. A SET hid two checks in one run;
a DICT hid two rows in one file. The question that found both is what the input
canonicalises before the guard sees it.

--- MEASURED BEFORE WIDENING IT -----------------------------------------------

A gate that reddens 250 unmeasured suites is a gate somebody turns off, so the
refusal was not widened on reasoning. A full PG 18 matrix ran with it armed for
every suite:

    247 suites ran, 6 skipped, 0 incomplete | ALL VERSIONS PASSED, RC=0 | 0 shared keys

Check names are static, so one major's matrix measures this class completely rather
than sampling it.

THE LOAD-BEARING EVIDENCE IS THAT RUN, not a log snapshot. An earlier draft of this
message cited "244 logs, 6651 records, 6651 distinct keys, 0 collisions" from
per-suite logs copied out of the build directory while the matrix was still running.
That figure is withdrawn: the copies were taken mid-write, and measuring the same way
again showed 150 of 253 had no `checks run:` line yet. Zero collisions across
truncated logs is not evidence of zero across complete ones. The full-matrix figure
stands because the gate reads the logs after each suite has finished.

It does still answer what commandprompt#982 said nobody had measured -- the gap between "checks
that ran" and "rows the ledger can hold" across the other 250 suites -- because the
gate saw every suite's complete log and refused nothing.

--- VERIFICATION --------------------------------------------------------------

The committed ledger still loads at 1197 rows. The gate against the COMMITTED ledger
and budget over a real 934-record harness_selftest log: rc=0, no shared-key line,
census agrees at 1189. A planted duplicate check: rc=1, naming the key. A planted
duplicate row: rc=2, naming the line and what it would lose.

The refusal arm also asserts no Traceback in the output, because an unhandled
exception exits 1 too and would satisfy an rc check on its own. Measured, it did: an
earlier draft referenced `covered_suites` before it was defined, which compiles and
fails at runtime.

guard_tests 277 -> 283, re-derived by collection. 283 passed / 698 checks with
--pgc-expect-tests 283 armed. docs_style.sh PASSED (11 checks). Long-sentence counts
unchanged: CHANGELOG 860, TESTS.md 319.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

--- AND A THIRD INSTANCE, IN THE ARM THAT POLICES THE OTHER TRACKED FILE -------

test_harness_deps.py read expected_tests.txt with `nums[f[0]] = int(f[1])`, so a
duplicated key collapsed and the last line won -- exactly as read_ledger did, one file
over. Measured on one fixture read both ways:

    the line form   names guard_tests as duplicated
    the dict form   sees two keys and keeps 280, the LAST line

NOT HYPOTHETICAL. Three PRs were open at once, each moving guard_tests, and resolving
all three keep-both produced three of those lines. ci.yml reads the value with
`awk '$1=="guard_tests"{print $2}'`, which prints one line per match. So WANT becomes
multi-line, `test -n "$WANT"` still passes, and the flag refuses it:

    pytest: error: argument --pgc-expect-tests: invalid int value: '284\n280\n283'
    exit 4

It FAILS CLOSED, so these two arms are about legibility rather than a hole. What exit 4
does not say is that a line is duplicated. The removal proof is on the real file: with a
duplicate planted, the new arm reddens and the pre-existing one stays green.

Keep-both is right for a changelog and wrong for a key-value file, and nothing in the
tree said so.

cluster_tests 205 -> 207, NOT guard_tests: test_harness_deps.py DEFINES NO_CLUSTER and
is not in it. I bumped the wrong number first and --pgc-expect-tests caught it --
`collected 283 test(s) but expected 285` -- which is the argument for the mechanism.
Both halves re-derived by collection: guard 283, cluster 207.

REBASED onto 14c9dd4 after commandprompt#1029 merged, and the conflict was the exact shape the third
arm here is about. `expected_tests.txt` conflicted on `guard_tests`:

    guard_tests 280     <- main, after commandprompt#1029
    guard_tests 283     <- this branch, from the old base

Keep-both would have produced two lines and, through ci.yml's
`awk '$1=="guard_tests"{print $2}'`, a multi-line value the flag refuses at exit 4. One
line, and both halves re-derived by collection rather than by arithmetic:

    guard_tests 286, cluster_tests 207

Re-verified: 286 passed / 707 checks with --pgc-expect-tests 286 armed; docs_style.sh
PASSED (11 checks); the gate against the committed ledger and budget over the real
934-record log still returns 0. Long-sentence counts unchanged against the new base:
CHANGELOG 860, TESTS.md 319.

--- FIXTURE CORRECTED AFTER REVIEW (@jdatcmd) -----------------------------------

test_the_gate_refuses_two_checks_sharing_one_ledger_key asserted rc == 1 on a
fixture where the gate returned 1 for TWO independent reasons, so the arm did not
discriminate. Found by mutation, and I reproduced it before changing anything:

    with `rc = 1` deleted from the shared-key block, the OLD fixture's arm PASSED
    with the same mutation, the corrected fixture's arm FAILS: got 0 want 1

The second route was the pre-existing new-check refusal. The ledger named only
`some other check` in that part, which made `demo` a covered suite whose log carried
a check the ledger had never seen:

    not in the ledger: demo   part1   shared name   (on major 18)

The ledger now NAMES `shared name`, so that refusal has nothing to say and only the
shared-key refusal can move rc.

This is the same defect class the PR is about, in my own arm: an assertion that
passes for a reason other than the one it is named for. My docstring claimed I had
measured this and closed it -- I had closed the COVERAGE-CEILING route and not the
new-check route, and the note said "routes" as though it were all of them. The
comment now enumerates all three, including the unhandled-exception route that also
exits 1.

It also described the refusal as "restricted to suites a row exists for", which was
the design I abandoned two pushes earlier when the matrix measurement said refusing
everywhere was safe. That sentence is gone.

--- REBASED onto 2ea7366 ------------------------------------------------------

guard_tests 293 and cluster_tests 207, both re-derived by COLLECTION on this base.

And a sharper point than the one I had, measured by @jdatcmd on commandprompt#1022: two branches
each derived `guard_tests 280` correctly, each against a tree holding only its own
three arms, and the merged value was 283. Both numbers were right for the tree they
were taken on and both were wrong for the merge. So "derive rather than add" is
necessary and not sufficient -- it has to be re-derived after every rebase, which
expected_tests.txt now says.

Re-verified: 293 passed / 725 checks with --pgc-expect-tests 293 armed; docs_style.sh
PASSED (11 checks); the gate against the committed ledger and budget over the real
934-record log returns 0; long-sentence counts unchanged at CHANGELOG 860 and
TESTS.md 319.

REBASED again onto 1d76ccf after commandprompt#1022 merged. Two hunks in expected_tests.txt, and the
second one is worth a note: my side carried the new `cluster_tests 207` comment PLUS two
stale fragments left by earlier keep-both resolutions of the same file ("177 -> 180",
"166 -> 182"), describing values the file no longer holds. Resolved to ONE coherent block
rather than the union, which is the same lesson the arms here are about: keep-both is
right for a changelog and wrong for a file where each key appears once.

guard_tests 296 and cluster_tests 207, both re-derived by collection on this base.
Re-verified: 296 passed / 733 checks armed; docs_style.sh PASSED; the gate over the real
934-record log returns 0.
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 13, 2026
…ndprompt#1031)

`Expect.rows` documents the argument as taking "a REASON, not a flag", and gives the
rationale: the escape hatch should cost more to type than the honest assertion. One
line below sat a truthiness test:

    if _empty(got) and _empty(want) and not allow_empty:

So `allow_empty=True` satisfied it and carried nothing, and the hatch cost LESS than
the assertion rather than more. Measured before the refusal: `allow_empty=True` and
`allow_empty=1` both passed, 3 passed. `row_set` forwards the argument, so it
inherited the hole and has its own arm asserting it does not route around the refusal.

THE CHECK FIRES WHENEVER THE ARGUMENT IS GIVEN, not only when both sides turn out to
be empty. Otherwise a flag form in a test whose sides happen to be non-empty passes
today and refuses on the day the data changes, which is the worst moment to learn it.

TWO LIVE SITES USED THE FLAG FORM AND BOTH WERE SUBSTANTIVELY CORRECT. Each had its
population premise on the line above, and test_check_records.py even stated the
argument in a comment: "an empty offender list is the answer to both, and only one of
them is good news." So nothing was hiding behind the hatch. The cost fell on the next
reader: the hatch exists so every empty-on-both-sides comparison carries its
justification where an audit of `allow_empty=` can read it, and half of them carried
none. Both now do, using the reason that was already in the comment.

Pinned in test_guards_pinned.py, which exists so a refusal is asserted by its message
rather than by "something failed" -- built after a census found 12 of 17 guards
deletable with the suite still green. Three arms: the refusal, the accepted form as
its control, and row_set inheriting it.

guard_tests 290 -> 293, derived by collection. The comment above the number now says
to RE-DERIVE AFTER EVERY REBASE rather than once: two branches can each derive
correctly against a tree holding only their own arms and both be wrong for the merge,
which @jdatcmd measured on commandprompt#1022 where two independent derivations of 280 had 283 as
the merged truth.

Verified: 293 passed / 718 checks with --pgc-expect-tests 293 armed; the layer's own
files 89 passed / 133 checks; docs_style.sh PASSED (14 checks); long-sentence counts
unchanged at CHANGELOG 868 and TESTS.md 324.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 13, 2026
…ndprompt#1031)

`Expect.rows` documents the argument as taking "a REASON, not a flag", and gives the
rationale: the escape hatch should cost more to type than the honest assertion. One
line below sat a truthiness test:

    if _empty(got) and _empty(want) and not allow_empty:

So `allow_empty=True` satisfied it and carried nothing, and the hatch cost LESS than
the assertion rather than more. Measured before the refusal: `allow_empty=True` and
`allow_empty=1` both passed, 3 passed. `row_set` forwards the argument, so it
inherited the hole and has its own arm asserting it does not route around the refusal.

THE CHECK FIRES WHENEVER THE ARGUMENT IS GIVEN, not only when both sides turn out to
be empty. Otherwise a flag form in a test whose sides happen to be non-empty passes
today and refuses on the day the data changes, which is the worst moment to learn it.

TWO LIVE SITES USED THE FLAG FORM AND BOTH WERE SUBSTANTIVELY CORRECT. Each had its
population premise on the line above, and test_check_records.py even stated the
argument in a comment: "an empty offender list is the answer to both, and only one of
them is good news." So nothing was hiding behind the hatch. The cost fell on the next
reader: the hatch exists so every empty-on-both-sides comparison carries its
justification where an audit of `allow_empty=` can read it, and half of them carried
none. Both now do, using the reason that was already in the comment.

Pinned in test_guards_pinned.py, which exists so a refusal is asserted by its message
rather than by "something failed" -- built after a census found 12 of 17 guards
deletable with the suite still green. Three arms: the refusal, the accepted form as
its control, and row_set inheriting it.

guard_tests 290 -> 293, derived by collection. The comment above the number now says
to RE-DERIVE AFTER EVERY REBASE rather than once: two branches can each derive
correctly against a tree holding only their own arms and both be wrong for the merge,
which @jdatcmd measured on commandprompt#1022 where two independent derivations of 280 had 283 as
the merged truth.

Verified: 293 passed / 718 checks with --pgc-expect-tests 293 armed; the layer's own
files 89 passed / 133 checks; docs_style.sh PASSED (14 checks); long-sentence counts
unchanged at CHANGELOG 868 and TESTS.md 324.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

stripe_row_limit=1000 is below vector_length=1024, so FSST never applies and a text column stores at 1.95x

3 participants