Skip to content

test: attack nine index_am_support arms (#1236) - #1240

Merged
jdatcmd merged 1 commit into
commandprompt:mainfrom
OffgridwithJD:test/1236-index-am-arms
Sep 24, 2026
Merged

jdatcmd merged 1 commit into
commandprompt:mainfrom
OffgridwithJD:test/1236-index-am-arms

Conversation

@OffgridwithJD

Copy link
Copy Markdown
Collaborator

Works #1236, second batch. No test changed and no code changed.

The same shape as the first batch

index_am_support's only dated rows were two premises about docs/features.md, reddened by a docs mutation -- which cannot reach one arm about what an index does. Eight behaviour arms sat never beside them: unattacked rather than vacuous.

Two mutations with disjoint predicted sets, both confirmed exactly

  M-1  pgcolumnar_index_build_range_scan drops two rows in three
       predicted  the four "answers its operator" arms, and NOT the build arms
       observed   exactly those four          16 passed + 4 failed

  M-2  pgcolumnar_index_build_range_scan errors, so CREATE INDEX fails
       predicted  the four "builds an index" arms
       observed   those four, plus the GiST overlap premise
                                              15 passed + 5 failed

M-1 leaving every builds an index arm green is the discriminating half: CREATE INDEX still succeeds when the index is merely incomplete, so the build arms should not move, and they do not.

M-2 also shows a weakness in the four answer arms

With no index in existence at all, the four answers its operator arms stayed green.

The arm compares a count taken with enable_seqscan = off against one taken with pgcolumnar.enable_custom_scan = off. enable_seqscan is a preference rather than a prohibition, so with no index both sides fall back to the same scan and agree. An arm named answers its operator with the same rows the scan returns can be satisfied without an index scan ever happening.

The suite already guards the neighbouring case -- its own comment says "Two paths that both return nothing agree" and there is a premise that the predicate matches rows. This is the other one: both paths returning the same rows by taking the same path.

Reported on #1236 rather than repaired here. This change records evidence and alters no test, and a repair (assert the index path actually used an index) is a different change that should be reviewed as one.

Provenance

Rows produced by pgc_ledger.py merge, one merge per mutation, --target for every arm that reddened. Not by a script editing the file.

Discipline

  control  20 passed + 0 failed + 0 unrunnable + 0 skipped = 20
  M-1      16 passed + 4 failed + 0 unrunnable + 0 skipped = 20
  M-2      15 passed + 5 failed + 0 unrunnable + 0 skipped = 20
  restored 20 passed + 0 failed + 0 unrunnable + 0 skipped = 20

Constant total, nothing unrunnable, nothing skipped, so every arm outside each target set ran and passed. Each anchor asserted to match exactly once; source restored and md5-verified after each.

Accounting

  census        1519 -> 1510, counted on this tree
                1602 rows, 92 not `never`
  reading list  72 -> 64
  verified      harness_selftest PG17 rc=0, PG18 rc=0, 1138 checks, 0 FAILs
                index_am_support 20 passed + 0 failed
                docs_style.sh PASSED

🤖 Generated with Claude Code

https://claude.ai/code/session_01MpajdQbkVJ9ey1XyYHcikP

No test changed and no code changed. index_am_support's only dated rows
were two premises about docs/features.md, reddened by a DOCS mutation,
which cannot reach one arm about what an index does. Eight behaviour arms
sat `never` beside them: unattacked rather than vacuous, the same shape the
first batch had.

Two mutations with disjoint predicted sets, both confirmed exactly:

  M-1  the build drops two rows in three, so the index misses entries
       predicted the four "answers its operator" arms and NOT the build arms
       observed  exactly those four, 16 passed + 4 failed
       CREATE INDEX still succeeds, so every build arm stayed green

  M-2  the build errors, so CREATE INDEX fails
       predicted the four "builds an index" arms
       observed  those four plus the GiST overlap premise, 15 passed + 5 failed

AND M-2 SHOWS A WEAKNESS IN THE FOUR ANSWER ARMS. With no index in
existence they all stayed GREEN. The arm compares a count taken with
`enable_seqscan = off` against one taken with the custom scan off, and
enable_seqscan is a preference rather than a prohibition -- with no index
both sides fall back to the same scan and agree. An arm named "answers its
operator with the same rows the scan returns" can be satisfied without an
index scan happening. Reported on commandprompt#1236, not repaired here.

Rows produced by `pgc_ledger.py merge`, one merge per mutation, --target
for every arm that reddened. Control 20/20 before each, source restored and
md5-verified after each, clean re-run after the last.

Census 1519 -> 1510, counted. The reading list drops 72 -> 64.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpajdQbkVJ9ey1XyYHcikP
@OffgridwithJD
OffgridwithJD force-pushed the test/1236-index-am-arms branch from 77a5955 to 9db9232 Compare September 24, 2026 00:26
@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

Rebased onto ace2285 (#1241). One conflict, in test/check_ledger_budget.txt,
and it was the census line both sides carry.

Neither side's number was right for the composed tree, so neither was taken.
Main carried 1521 and this branch carried 1510, which described a main that no
longer exists. Re-derived by counting, which is the only thing that survives a
rebase:

  awk -F'\t' '$5=="never"' test/check_ledger.tsv | wc -l   ->  1512
  1606 rows total, 94 not `never`,  1512 + 94 == 1606

Cross-checked against the other route: main's 1521 minus the nine rows this
branch dates is 1512. The ledger itself auto-merged -- the two branches date
rows in different suites -- so only the derived number needed rebuilding.

The nine rows split exactly as the budget note predicts, which is worth stating
because it is the claim the rows make:

  M-1  the build drops two rows in three    4 rows   "answers its operator"
  M-2  the build errors                     5 rows   4x "builds an index" + the GiST overlap premise

Verified on the composed tree, not on the pre-rebase branch:

  harness_selftest  /usr/local/pg17a   exit=0   checks run: 1138   0 FAIL
  harness_selftest  /usr/local/pg18a   exit=0   checks run: 1138   0 FAIL
  docs_style.sh                        PASSED

No suite code changed: the diff is 3 files, all evidence -- the ledger rows, the
budget note and the CHANGELOG. test/index_am_support.sh is untouched, so the
mutations that produced these rows were run against the same suite text that is
on main.

Two instrument faults on the way, both mine and both caught before they could
be read as results.
The first run reported 21 failures on pg18a; every one was
no-repo, because I packed the tree with tar --exclude=.git and a fifth of
the selftest asks git questions. The second was an aggregate digest mismatch
between host and container that turned out to be sort collation -- 875 files,
identical content, different order. Confirmed by sorting the digest lines
themselves and by a control that mutated one byte and did separate them.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MpajdQbkVJ9ey1XyYHcikP

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving 9db9232e. Verified by deriving from the artifact rather than reading the body.

What changed

  ledger rows changed    9, every one never -> 2026-09-23, none the other way
  majors                 unchanged on all 9
  byte-identical moves   0
  rows added or removed  0
  census                 1606 rows, 1512 never, 94 not-never, 1512 + 94 == 1606
  budget states          1512, agrees, and 1521 - 9 = 1512 against main
  files                  CHANGELOG.md, check_ledger.tsv, check_ledger_budget.txt

Attribution counted on the exact mutation strings in the ledger this PR ships, not from the description:

  M-2  pgcolumnar_index_build_range_scan errors, so CREATE INDEX fails        5 arms
  M-1  ... drops two rows in three, so the index misses entries               4 arms

which matches "four, plus those four and the GiST overlap premise" exactly.

The accounting lines are in the body and they close the complement: constant total 20, nothing unrunnable, nothing skipped, so every arm outside each target set ran and passed. That is the half the ledger cannot carry — an arm that was unrunnable under a mutation is not red, is not named, and is indistinguishable afterwards from one that passed.

The finding is worth more than the rows

With no index in existence at all, the four answers its operator arms stay green. enable_seqscan is a preference and not a prohibition, so with no index both sides of the comparison fall back to the same scan and agree. An arm named btree answers its operator with the same rows the scan returns can be satisfied without an index scan ever happening.

That is not vacuity — M-1 reddens all four on a wrong answer — it is an arm whose NAME claims the index path was exercised while nothing checks that. Same class as the stat -c %F fixture facts in part 570, reached from the opposite direction: those were found by reading names, this by deleting the index and watching the arms hold.

Not repairing it here is the right call. The repair is one premise — assert the index path's plan contains an index scan — and a repair smuggled into a change whose subject is records evidence, alters no test is how a later reader loses track of which claims were measured. Reported on #1236 instead.

The pattern behind both batches

index_am_support's only dated rows were two premises about docs/features.md, reddened by a documentation mutation, which cannot reach an arm about what an index does. Eight behaviour arms sat never beside them.

That is the same shape as native_join_runtime_filter's GUC boot value in #1238, and it is why #1236's third clause is the one that does the work: a never arm in a part nobody has attacked says nothing, and a never arm in a part attacked only from outside its own subject is a question. Two for two — and the author has since refuted it as a general rule on the remaining 64, which is a better outcome than a third confirmation.

@jdatcmd
jdatcmd merged commit ac80762 into commandprompt:main Sep 24, 2026
15 checks passed
OffgridwithJD pushed a commit to OffgridwithJD/pgcolumnar that referenced this pull request Sep 24, 2026
Six mutations over validity_elision, selftest part 190 and
native_join_vector_agg. No test changed and no code changed: the arms were
attacked and the ledger records what happened.

  veC    the elision predicate forced true              5 arms
  b190A  pgc_build_needs_clean never asks for a clean    5 arms
  b190B  the stamp writes an escaped literal (commandprompt#898)      2 arms
  b190C  pgc_build_needs_clean always demands a clean    2 arms
  b190D  unknown provenance reported as a major, PG?     2 arms
  jvaB   the dim-side uniqueness refusal disabled        2 arms

b190A and b190C are opposites and redden disjoint pairs. A guard that never
cleans loses the five arms about when a clean is required; one that always
cleans loses exactly two, and only those two separate a correct guard from one
that is merely fail-closed.

Fifteen of the eighteen rows are newly dated. The other three are
validity_elision arms already dated under the inverse mutation -- the predicate
forced false rather than true -- so they now carry both strings and the later
date, rather than the newer observation overwriting the older.

Re-derived on the merged tree. All six had run once before commandprompt#1240 and commandprompt#1241
landed and were re-run against ac80762, reproducing their earlier results
exactly: same names, not merely the same counts, with a control showing the
comparison can separate two unrelated sets. The two C mutations ran with a
forced make clean, because PGXS here has no --enable-depend and a stale object
after a header edit would leave veC reddening nothing.

Census re-derived by counting: 1606 rows, 109 not never, 1497 + 109 == 1606.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpajdQbkVJ9ey1XyYHcikP
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants