Skip to content

test: the four PG-log secret-leak claims had no premise (#1032) - #1106

Merged
jdatcmd merged 1 commit into
commandprompt:mainfrom
OffgridwithJD:test/1032-premise-the-pg-log-claims
Sep 18, 2026
Merged

jdatcmd merged 1 commit into
commandprompt:mainfrom
OffgridwithJD:test/1032-premise-the-pg-log-claims

Conversation

@OffgridwithJD

Copy link
Copy Markdown
Collaborator

Closes #1032.

Four secret-leak claims over $PGC_LOGFILE could pass having read nothing. Each now carries its own premise.

The hole is exactly one state

iceberg_rest.sh, iceberg_rest_server.sh (twice) and iceberg_rest_vended.sh assert a token or secret never reaches the PG server log, comparing grep -c "$SECRET" "$PGC_LOGFILE" against 0. Nothing in the tree asserted that file exists, is readable, or holds a line.

I expected the usual fail-open shapes and found only one. Measured over all four states:

the log is grep -c gives the claim
a real server log 0 passes, correctly
present and EMPTY 0 PASSES, having read nothing
missing (nothing) fails
leaking the secret 1 fails

grep -c prints nothing for a file it cannot open, so a missing path, an unset variable and an unreadable file all fail closed already. Present-and-empty prints the 0 the claim wants. Anything that leaves the log present and empty — a rotation, a log_destination change, a truncating reuse path — turns all four green and says nothing.

The fix

Each claim carries its own premise immediately above it, rather than one per suite, so a claim and its premise cannot drift apart:

_ir_loglines="$(grep -c . "$PGC_LOGFILE" 2>/dev/null)" || _ir_loglines=0
check "premise: the PG server log holds lines to search" \
	"$([ "${_ir_loglines:-0}" -gt 0 ] && echo yes || echo no)" "yes"

Not the grep -c . ... || echo 0 form the issue suggested. grep -c prints 0 and exits 1 on an empty file, so that yields two lines and turns the comparison into a shell error rather than a comparison:

empty log with || echo 0 -> $'0\n0'

That is the same defect I raised on #1102, and it would have put it into the fix for it. The assignment form carries one value.

The two premises in iceberg_rest_server.sh are named distinctly, because a row is keyed on (suite, part, name) and two checks sharing a name share a row (#982).

Removal proof, at suite level

Pointing both reads at a present-and-empty file:

FAIL  premise: the PG server log holds lines to search: got [no] want [yes]
PASS  the token never appears in the server (PG) log
file the claim read: 0 lines

The premise catches precisely what the claim cannot see, and the claim still passes — which is the whole finding.

A first attempt at this proof did not reproduce, and that is worth recording. Truncating $PGC_LOGFILE in place left the premise green, because the running postmaster wrote a line back within the same instant:

MUT-DIAG lines immediately after truncation: 1

The mutation never created the state it claimed to, and an invalid control that reports green looks exactly like a passing one. The valid version points PGC_LOGFILE at a static file nothing writes to.

Verification

All three suites on PG18, unmutated:

iceberg_rest:        rc=0  FAILs=0  iceberg_rest.sh: PASSED
  PASS  premise: the PG server log holds lines to search
  PASS  the token never appears in the server (PG) log
iceberg_rest_server: rc=0  FAILs=0  iceberg_rest_server.sh: PASSED
  PASS  premise: the PG server log holds lines to search, for the mapping token
  PASS  premise: the PG server log holds lines to search, for the client secret
  PASS  the mapping token never appears in the PG server log
  PASS  the client secret never appears in the PG server log
iceberg_rest_vended: rc=0  FAILs=0  iceberg_rest_vended.sh: PASSED
  PASS  premise: the PG server log holds lines to search
  PASS  the vended secret never appears in the PG server log

bash -n and shellcheck -S error -s bash clean on all three. The mutated file was restored byte-exact after each arm.

The ledger is untouched. All three suites hold zero rows, so they are uncovered: no rows move, the new-check refusal does not apply, and suites_not_covered does not change — adding checks to a suite already counted as uncovered leaves that ceiling where it is.

Note on the CHANGELOG

This adds a new ## [Unreleased] section, and so does #1098. They will conflict for no reason related to either change — which is #996, still open.

🤖 Generated with Claude Code

https://claude.ai/code/session_012RSw4qMHS7ByE7PY8Ns4cs

@linuxhikerpm linuxhikerpm left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The empty-log hole is real: grep -c SECRET against an existing empty file is 0, which is the want. Missing/unreadable already fail closed. A per-claim premise immediately above each absence check is the right pin, and the assignment form (grep -c . ... || var=0) avoids the two-line || echo 0 trap. Distinct names on the two server-suite premises keep the ledger key unique.

CHANGELOG correctly opens ## [Unreleased] after alpha4 closed. That will collide with other Unreleased PRs on merge; that is editorial, not a defect in the premises. Approving, not merging.

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The hole is real, the fix closes exactly it, and I reproduced the state table independently rather than reading it.

What it closes

grep -c prints 0 for a file that exists and is empty, so a leak claim expecting 0 passes having read nothing. A MISSING file prints nothing and already failed the claim, so empty was the only hole. I ran the four states myself:

real     premise=yes  claim=[0]   premise holds
empty    premise=no   claim=[0]   PREMISE CATCHES IT
missing  premise=no   claim=[]    PREMISE CATCHES IT
leak     premise=yes  claim=[1]   premise holds, claim fails

Your table and mine agree, including that missing was never the hole.

The wiring, which is what I actually wanted to check

A premise that guards a different file than the claim greps is decoration. These do not:

110  _ir_loglines="$(grep -c . "$PGC_LOGFILE" ...)"
111  check "premise: the PG server log holds lines to search"
113  check "the token never appears in the server (PG) log"
114      "$(grep -c "$TOKEN" "$PGC_LOGFILE" ...)" "0"

Same variable, and the premise is on the line before the claim rather than after it, so a vacuous claim cannot already have passed.

The note on || echo 0

Worth the space it takes. grep -c prints 0 AND exits 1 on an empty file, so || echo 0 yields two lines and turns the comparison into a shell error. That is the kind of thing that gets "simplified" back in by someone tidying, and the comment is what stops it.

One thing I looked for and found already answered

My first question was whether a non-empty log is a POSITIVE control or only a necessary premise: it proves the file has content, not that the search would find the secret if it were there. Your leaking got=[1] row is that control, and it is in the comment. I would not have found it if the measurement table were not written down.

…t#1032)

`iceberg_rest.sh`, `iceberg_rest_server.sh` (twice) and `iceberg_rest_vended.sh`
each assert a token or secret never reaches `$PGC_LOGFILE`, comparing
`grep -c "$SECRET" "$PGC_LOGFILE"` against `0`. Nothing in the tree asserted that
file exists, is readable, or holds a line.

The hole is exactly one state, measured over all four:

    a real server log   got=[0]  passes
    an EMPTY log        got=[0]  PASSES, having read nothing
    a missing log       got=[]   fails
    a log that LEAKS    got=[1]  fails

`grep -c` prints nothing for a file it cannot open, so missing, unset and
unreadable already fail closed. Present-and-empty prints the `0` the claim wants.

Each claim now carries its own premise directly above it, so they cannot drift.
Not the `|| echo 0` form: `grep -c` prints `0` AND exits 1 on an empty file, so
that yields two lines and makes the comparison a shell error instead. Measured.

Removal proof at suite level: pointing both reads at a present-and-empty file
gives `FAIL premise: the PG server log holds lines to search` while
`PASS the token never appears in the server (PG) log` still stands.

A first attempt truncated `$PGC_LOGFILE` in place and did NOT reproduce -- the
running postmaster wrote a line back within the same instant. Recorded because
the invalid control looked like a passing one.

All three suites green on PG18; shellcheck -S error and bash -n clean. The three
suites hold no ledger rows, so no rows move and `suites_not_covered` is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012RSw4qMHS7ByE7PY8Ns4cs
@OffgridwithJD
OffgridwithJD force-pushed the test/1032-premise-the-pg-log-claims branch from c23d316 to 9d093ba Compare September 17, 2026 23:21
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 17, 2026
…safe (commandprompt#996)

Entries insert as the first child of one heading, so two PRs sharing no file but
CHANGELOG.md still conflict. Nine of 27 merges in one day touched it; three merge
commits that day exist only to resolve it.

Measured on the real pair, commandprompt#1098 and commandprompt#1106, both opening a new [Unreleased]:

    default 3-way   rc=1, 2 conflict markers
    merge=union     rc=0, 0 markers, ONE [Unreleased], both entries intact

Union emits identical lines once, which is why the headings are not doubled:
7640 + 57 + 52 = 7749 against an actual 7744.

THE DRIVER DOES NOT SHIP ALONE. Union keeps both sides of a divergent hunk
silently, and a release cut EDITS the line pending PRs append beneath. Reproduced:
a PR merged into a release cut files its entry inside the section that just
shipped, rc=0, no marker. Today that conflicts and a human sees it.

So docs_style.sh gains the check that catches it: each dated section holds the
entries its own tag shipped, and no more. Proved in three directions -- an entry
added to closed alpha4 gives got [113] want [112]; one removed from alpha2 gives
got [57] want [58]; removing the driver line reddens its own arm.

v1.0-alpha3 HAS NO BASELINE: that tag shipped with its section still named
[Unreleased], the release having been dated after it was tagged. That is ASSERTED
rather than skipped -- "v1.0-alpha3 shipped no dated section of its own, so it has
no baseline" -- so "this cannot be compared" is itself checked, and the arm reddens
if a release ever does carry one (measured: got [1] want [0]). False-positive
budget measured before shipping: 1 of 4 released sections, and that one is this.

REACHING FOR A SKIP THERE WAS THE WRONG INSTRUMENT, and two guards said so. This
suite keeps its own tally and emits none of the machine RESULT vocabulary, so
lib.sh's check_skip is `command not found` inside it -- printing nothing, counting
nothing, failing nothing, while the suite reports PASSED. Adding a local one then
tripped selftest 400, which refuses `echo "SKIP` in any file that calls check,
because a SKIP is an outcome a count and a record must see and this suite has
neither. A property it cannot compare is now a note(): printed, counted as
nothing, claiming no outcome.

awk kept portable: no gawk-only three-argument match(); identical output under
mawk, gawk and this box's default awk. bash -n and shellcheck -S error clean.
docs_style.sh holds no ledger rows, so no rows move.

Verified: docs_style.sh 35 checks rc=0 stderr empty; harness_selftest 976 checks
0 failed, including selftest 400's own SKIP guard; pytest guards 346 passed, 897
checks, 0 failed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012RSw4qMHS7ByE7PY8Ns4cs
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 17, 2026
…safe (commandprompt#996)

Entries insert as the first child of one heading, so two PRs sharing no file but
CHANGELOG.md still conflict. Nine of 27 merges in one day touched it; three merge
commits that day exist only to resolve it.

Measured on the real pair, commandprompt#1098 and commandprompt#1106, both opening a new [Unreleased]:

    default 3-way   rc=1, 2 conflict markers
    merge=union     rc=0, 0 markers, ONE [Unreleased], both entries intact

Union emits identical lines once, which is why the headings are not doubled:
7640 + 57 + 52 = 7749 against an actual 7744.

THE DRIVER DOES NOT SHIP ALONE. Union keeps both sides of a divergent hunk
silently, and a release cut EDITS the line pending PRs append beneath. Reproduced:
a PR merged into a release cut files its entry inside the section that just
shipped, rc=0, no marker. Today that conflicts and a human sees it.

So docs_style.sh gains the check that catches it: each dated section holds the
entries its own tag shipped, and no more. Proved in three directions -- an entry
added to closed alpha4 gives got [113] want [112]; one removed from alpha2 gives
got [57] want [58]; removing the driver line reddens its own arm.

v1.0-alpha3 HAS NO BASELINE: that tag shipped with its section still named
[Unreleased], the release having been dated after it was tagged. That is ASSERTED
rather than skipped -- "v1.0-alpha3 shipped no dated section of its own, so it has
no baseline" -- so "this cannot be compared" is itself checked, and the arm reddens
if a release ever does carry one (measured: got [1] want [0]). False-positive
budget measured before shipping: 1 of 4 released sections, and that one is this.

REACHING FOR A SKIP THERE WAS THE WRONG INSTRUMENT, and two guards said so. This
suite keeps its own tally and emits none of the machine RESULT vocabulary, so
lib.sh's check_skip is `command not found` inside it -- printing nothing, counting
nothing, failing nothing, while the suite reports PASSED. Adding a local one then
tripped selftest 400, which refuses `echo "SKIP` in any file that calls check,
because a SKIP is an outcome a count and a record must see and this suite has
neither. A property it cannot compare is now a note(): printed, counted as
nothing, claiming no outcome.

awk kept portable: no gawk-only three-argument match(); identical output under
mawk, gawk and this box's default awk. bash -n and shellcheck -S error clean.
docs_style.sh holds no ledger rows, so no rows move.

Verified: docs_style.sh 35 checks rc=0 stderr empty; harness_selftest 976 checks
0 failed, including selftest 400's own SKIP guard; pytest guards 346 passed, 897
checks, 0 failed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012RSw4qMHS7ByE7PY8Ns4cs
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 17, 2026
…safe (commandprompt#996)

Entries insert as the first child of one heading, so two PRs sharing no file but
CHANGELOG.md still conflict. Nine of 27 merges in one day touched it; three merge
commits that day exist only to resolve it.

Measured on the real pair, commandprompt#1098 and commandprompt#1106, both opening a new [Unreleased]:

    default 3-way   rc=1, 2 conflict markers
    merge=union     rc=0, 0 markers, ONE [Unreleased], both entries intact

Union emits identical lines once, which is why the headings are not doubled:
7640 + 57 + 52 = 7749 against an actual 7744.

THE DRIVER DOES NOT SHIP ALONE. Union keeps both sides of a divergent hunk
silently, and a release cut EDITS the line pending PRs append beneath. Reproduced:
a PR merged into a release cut files its entry inside the section that just
shipped, rc=0, no marker. Today that conflicts and a human sees it.

So docs_style.sh gains the check: each dated section holds the entries its own tag
shipped, and nothing else. THE KEY IS THE ENTRY, not a count -- a count would let
one post-tag entry be swapped for another with the arm still green.

IT FOUND ONE ALREADY ON MAIN. ## [1.0-alpha3] carries an entry v1.0-alpha3 never
shipped, added by d978e7f (commandprompt#432) on 2026-09-09, seven days after the 2026-09-02
tag. Found by jdatcmd in review. It cannot be corrected without making a second
section wrong -- the work shipped in the alpha4 cycle and the v1.0-alpha4 tag does
not carry it either -- so a released section MAY diverge, but only by being
recorded in test/changelog_post_tag.txt with a reason visible in the diff. Two
arms guard that file: every row needs a reason, and no row may be stale.

Seven mutations, each restored byte-exact: an entry added to closed alpha4, one
removed from alpha2, the allowance row deleted, the allowance naming a different
entry, the reason blanked, a stale allowance row, and the driver line removed.
Each reddens its own arm. Clean tree: 42 checks, rc=0.

THE FIRST VERSION OF THIS CHECK WAS MEASURED AGAINST A STALE TAG. This tree's
local v1.0-alpha3 was d9df031 against the server's cec9e9b, and git fetch never
moves a tag that already exists. That produced a false narrative (alpha3 "shipped
with its section still named [Unreleased]"), a skip for it, and a "false-positive
budget of 1 of 4" -- all three wrong, with the arm green where the tree was in
violation. jdatcmd caught it by checking their own refs against git ls-remote
first. The suite now tells a reader to verify a tag before believing a red arm.

Reaching for a skip was also wrong, and two guards said so. This suite keeps its
own tally and emits none of the machine RESULT vocabulary, so lib.sh's check_skip
is `command not found` inside it. A local one then tripped selftest 400, which
refuses `echo "SKIP` in any file that calls check. A property this suite cannot
compare is now a note(): printed, counted as nothing, claiming no outcome.

Selftest 070 caught a third: introducing `comm` makes every `| sort` in the file
collation-sensitive, and comm compares byte-wise without checking its inputs
agree. All nine sorts are pinned LC_ALL=C, including the five that predate this
block, and so is comm.

KNOWN LIMIT: at depth 1 with no tags every section arm is skipped, so CI cannot
run any of this. It runs locally and in the five-major release gate.

Verified: docs_style 42 checks rc=0; harness_selftest 976 checks 0 failed,
including selftest 070 and 400's own arms; pytest guards 346 passed, 897 checks,
0 failed. docs_style.sh holds no ledger rows, so no rows move.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012RSw4qMHS7ByE7PY8Ns4cs
@jdatcmd
jdatcmd merged commit b7724f6 into commandprompt:main Sep 18, 2026
14 checks passed
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 18, 2026
…safe (commandprompt#996)

Entries insert as the first child of one heading, so two PRs sharing no file but
CHANGELOG.md still conflict. Nine of 27 merges in one day touched it; three merge
commits that day exist only to resolve it.

Measured on the real pair, commandprompt#1098 and commandprompt#1106, both opening a new [Unreleased]:

    default 3-way   rc=1, 2 conflict markers
    merge=union     rc=0, 0 markers, ONE [Unreleased], both entries intact

Union emits identical lines once, which is why the headings are not doubled:
7640 + 57 + 52 = 7749 against an actual 7744.

THE DRIVER DOES NOT SHIP ALONE. Union keeps both sides of a divergent hunk
silently, and a release cut EDITS the line pending PRs append beneath. Reproduced:
a PR merged into a release cut files its entry inside the section that just
shipped, rc=0, no marker. Today that conflicts and a human sees it.

So docs_style.sh gains the check: each dated section holds the entries its own tag
shipped, and nothing else. THE KEY IS THE ENTRY, not a count -- a count would let
one post-tag entry be swapped for another with the arm still green.

IT FOUND ONE ALREADY ON MAIN. ## [1.0-alpha3] carries an entry v1.0-alpha3 never
shipped, added by d978e7f (commandprompt#432) on 2026-09-09, seven days after the 2026-09-02
tag. Found by jdatcmd in review. It cannot be corrected without making a second
section wrong -- the work shipped in the alpha4 cycle and the v1.0-alpha4 tag does
not carry it either -- so a released section MAY diverge, but only by being
recorded in test/changelog_post_tag.txt with a reason visible in the diff. Two
arms guard that file: every row needs a reason, and no row may be stale.

Seven mutations, each restored byte-exact: an entry added to closed alpha4, one
removed from alpha2, the allowance row deleted, the allowance naming a different
entry, the reason blanked, a stale allowance row, and the driver line removed.
Each reddens its own arm. Clean tree: 42 checks, rc=0.

THE FIRST VERSION OF THIS CHECK WAS MEASURED AGAINST A STALE TAG. This tree's
local v1.0-alpha3 was d9df031 against the server's cec9e9b, and git fetch never
moves a tag that already exists. That produced a false narrative (alpha3 "shipped
with its section still named [Unreleased]"), a skip for it, and a "false-positive
budget of 1 of 4" -- all three wrong, with the arm green where the tree was in
violation. jdatcmd caught it by checking their own refs against git ls-remote
first. The suite now tells a reader to verify a tag before believing a red arm.

Reaching for a skip was also wrong, and two guards said so. This suite keeps its
own tally and emits none of the machine RESULT vocabulary, so lib.sh's check_skip
is `command not found` inside it. A local one then tripped selftest 400, which
refuses `echo "SKIP` in any file that calls check. A property this suite cannot
compare is now a note(): printed, counted as nothing, claiming no outcome.

Selftest 070 caught a third: introducing `comm` makes every `| sort` in the file
collation-sensitive, and comm compares byte-wise without checking its inputs
agree. All nine sorts are pinned LC_ALL=C, including the five that predate this
block, and so is comm.

KNOWN LIMIT: at depth 1 with no tags every section arm is skipped, so CI cannot
run any of this. It runs locally and in the five-major release gate.

Verified: docs_style 42 checks rc=0; harness_selftest 976 checks 0 failed,
including selftest 070 and 400's own arms; pytest guards 346 passed, 897 checks,
0 failed. docs_style.sh holds no ledger rows, so no rows move.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012RSw4qMHS7ByE7PY8Ns4cs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Four secret-leak claims over $PGC_LOGFILE have no positive control (low severity, measured)

3 participants