docs: pgcolumnar.compression picks encodings, not only a codec (#1074, #1076)

[jdatcmd]

pgcolumnar.compression does not only choose a codec. It also decides which
lightweight encodings your data gets, and we documented the opposite.

columnar_encoding.c returns "FSST helps" unconditionally when the codec is
none, before it looks at the corpus or at the margin:

if (compressionType == COLUMNAR_COMPRESSION_NONE)
    return true;

The reasoning is right — with no codec the encoded length already is the stored
length, so the per-vector test is measuring the right thing. The consequence is
that none is a different cascade, not the same cascade minus a step.

Two published claims are refuted by measurement

docs/administration.md described none as "Lowest write cost, largest size".
Measured on 200,000 rows per shape, backend instruction counts, zstd against
none:

high-entropy hex text     0.92     zstd CHEAPER to write
repetitive text           0.99     zstd CHEAPER to write

Neither shape made none the cheaper one. Repetition spreads were 0.08% and
0.19%, so the 0.67% on repetitive text is small and outside the noise. On the hex
shape zstd also produced a table 1.4 percent larger, because dropping FSST
cost more than the codec saved.

What the pages deliberately do NOT say. They do not claim zstd is cheaper in
general. These are instruction counts on one box with fsync off, and an
instruction count cannot see an I/O saving at all — the same limit
@OffgridwithJD established on #890. The wording is "not reliably the cheapest",
with both numbers shown.

docs/best-practices.md said each chunk "takes the encoding that makes it
smallest". Wrong twice: the target is the smallest stored result, so it
depends on the codec; and a FSST win below fsst_min_gain_percent is given up on
purpose.

Arms in both harnesses, every one a differential

"FSST is kept under none" on its own is satisfied by any corpus FSST always wins
on — green and empty. So each arm loads the same corpus at the same margin with
a codec, where FSST is dropped, and the only variable is the setting under test.
One arm exists only to assert the corpus is marginal rather than one-sided.

fsst_margin.sh gains three checks. test_compression_reaches_the_cascade.py is
independent: its own corpus, its own descriptor parse, neither file naming the
other.

Removal proof, on both harnesses

Deleting the early return and rebuilding moves the .so c8e5c790dbac →
6455728725e5:

fsst_margin.sh   14 passed + 2 failed + 0 unrunnable + 0 skipped = 16
pytest            8 pass + 2 fail + 0 unrun = 10

Only the codec arms redden. Every content invariant stays green on both sides,
which is the right answer — the mutant still writes correct rows, so this is a
decision changing rather than corruption.

Five majors, this tree

Own make clean, own build and own .so each:

PG15 16 PASS   PG16 16 PASS   PG17 16 PASS   PG18 16 PASS   PG19 16 PASS

cluster_tests re-derived by collection 406 → 410. guard_tests re-derived in
the same run and did not move (342) — checked rather than assumed. No ledger
change: fsst_margin has 0 rows in main's ledger.

Two tools caught me

plain_language_check.py failed my first draft on six over-long sentences and two
prose double-hyphens, in prose written to explain a measurement. Rewritten to its
rules rather than argued with. And test_docs_cover_the_corpus.py caught the new
pytest file having no numbered TESTS.md section, which is #1024's arm doing
exactly its job.

Closes #1076. Addresses the documentation half of #1074; its remaining half is the
measurement, which stays on the issue.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK

[jdatcmd]

Merged main at 675e190 to clear the conflict, which was CHANGELOG.md only.
#1080, #1081 and #1083 landed while this was open.

Both sides add entries to ### Fixed, so both are kept. Verified by count rather
than by reading the diff:

conflict markers left                  0
entries present exactly once           #1074/#1076, #1075, #1077, #1080, #1081
bodiless headings in [Unreleased]      0

That last one is not paranoia: the same resolution on #1062 would have shipped a
duplicated heading, and a bodiless one sat on main for two days before that.

Nothing else moved. Per-file patch md5 of the seven files, merged result
against the pre-merge branch, added and removed lines only:

docs/administration.md                              6a456b0edcac  same
docs/best-practices.md                              0c84c03e77bd  same
docs/configuration.md                               68b2705ebe8a  same
test/fsst_margin.sh                                 978e4e448429  same
test/pytest/TESTS.md                                787346d68405  same
test/pytest/expected_tests.txt                      d1807c9dfd54  same
test/pytest/test_compression_reaches_the_cascade.py 025da1ba7426  same

Counts re-derived by collection on the MERGED tree, not carried forward from
the pre-merge derivation:

guard half     342 tests collected     file states 342
cluster half   410 tests collected     file states 410

I also checked by md5 that main moved none of expected_tests.txt,
check_ledger.tsv or check_ledger_budget.txt — rather than assuming it from the
fact that the three merges were docs, shell suites and one src/ file. They are
unchanged, so 410 was always going to hold; deriving it was still the cheaper way
to know that than reasoning about it.

CI re-running at the new head.

[OffgridwithJD]

APPROVE at 675e190, once the two pending legs land. I attacked the corpus premise as you asked, and it is stronger than you described it.

The corpus premise is bracketed by three points, not guarded by one arm

You framed it as "there is an arm for exactly that". There are three points on the same corpus, and it is the pair that does the work:

low margin + codec   ->  kept > 0      FSST's win is at least the low margin
margin 90  + codec   ->  dropped == 0  and at most 90
margin 90  + none    ->  none_v > 0    so a keep here came from the codec

The first two bracket the win rather than asserting a point. So both drift directions redden:

• corpus drifts to FSST always wins big → at margin 90 it is kept → dropped > 0 → the second arm reddens
• corpus drifts to FSST never wins → at the low margin it is dropped → kept == 0 → the first arm reddens

A single arm at one margin would have covered one of those. The bracket covers both, and that is the answer to your question: I could not find a corpus drift that leaves the differential green and meaningless.

And the bracket guards fsst_vectors() itself, which I think is accidental

The same two arms falsify the instrument:

fsst_vectors always returns 0   -> kept == 0    first arm reddens
fsst_vectors always returns >0  -> dropped > 0  second arm reddens

That matters because fsst_vectors parses the encoding descriptor by byte offset — get_byte(descriptor, 6 + i * 13) == 8 — which is exactly the kind of reader that goes silently wrong when a format changes. A descriptor-layout change that made every vector read as FSST, or none, is caught by the same pair that catches corpus drift.

Worth a sentence in the comment, because it is a property somebody could remove by "simplifying" the low-margin arm away as redundant.

The claims I checked rather than read

0, 99 is really the GUC's range. columnar_tableam.c:3269 — DefineCustomIntVariable(..., 5, 0, 99, ...). So "99 is the maximum the GUC accepts, so if any margin could drop FSST under none this is the one that would" is exactly true, and the arm is the strongest form of that test rather than an arbitrary high value.

The docs correction is the right shape. none went from "Lowest write cost, largest size" — a claim your own arms refute — to "Not reliably the cheapest to write or the largest on disk". Declining to write a general "zstd is cheaper" claim from instruction counts on one box with fsync off is the restraint that makes the rest credible, and it is the same discipline as the ROADMAP's "the conclusion travels, the ratios do not".

What I like

Pairing section 2b with section 1 as "the same differential with the other variable moved" is what makes the codec arm mean anything. Without it, a keep under none is satisfied by any corpus FSST always wins on — and the comment says so explicitly, which is better than leaving the reader to notice.

The independent pytest twin having its own corpus and its own descriptor parse is the right kind of independence: the two harnesses can disagree, which is the only way a shared bug in the parse would show.

And plain_language_check.py failing your explanation prose, and you rewriting to its rules rather than arguing, is the tool working on its author.

One note, not blocking

#1079 has gone CONFLICTING since main moved to 410e06b. Not this PR's problem, but it is ahead of this one in your queue.

[OffgridwithJD]

APPROVE. 14/14 green. The detailed review is in my comment above; this is the formal approval now that both legs have landed.

The short version of what I checked rather than read:

• The corpus premise is bracketed by three points, not guarded by one arm, so both drift directions redden. I could not find a corpus drift that leaves the differentials green and meaningless — which was the question you put.
• That bracket also falsifies fsst_vectors() itself in both directions, which matters because it parses the encoding descriptor by byte offset. Worth a sentence in the comment so nobody removes the low-margin arm as redundant.
• 0, 99 is really the GUC's range (columnar_tableam.c:3269), so the maximum-margin arm is the strongest form of that test rather than an arbitrary high value.
• The docs correction replaces a claim your own arms refute. none as "Lowest write cost, largest size" was wrong in both halves.