Port projections to pytest, all 75 names (#432)

[OffgridwithJD]

Stacked on #1067. The first two commits are that PR's; review fc13131 alone, or wait for #1067 to merge and this becomes a single-commit diff. They are stacked deliberately — see the last section.

test/projections.sh pins the multiple-projections DDL, catalog and read path. A
projection is a second copy of some columns, and every property is about the copy
staying honest: it holds the rows the base holds, it loses the rows the base loses,
it survives a vacuum that renumbers every row underneath it, and the planner reads it
only when it can answer the whole query from it.

That makes a wrong projection a wrong answer, not a slow one. A scan that reads a
stale projection returns rows the base no longer has, and nothing downstream re-checks.

literal matches: 75 | template matches: 0 | missing: 0
VERDICT: every bash property is covered

The file is organised by what can make the copy diverge, not by feature: CATALOG,
FAN-OUT, RECONSTRUCT, PLANNER, REBUILD, MVCC, LIFECYCLE.

The port is strictly stronger in one place

A reviewer comparing the two should know which way, so it is stated rather than left to
be noticed.

The original's own expect_fail helper runs the statement and passes when it errors at
all. A misspelt table name satisfies every one of its eight refusal arms just as well
as the refusal it is named for. The port asserts the SQLSTATE instead, and every code was
measured against this build rather than guessed:

duplicate name          42710      add on heap table       42809
unknown column          42703      drop base               22023
empty columns           22023      drop unknown            42704
duplicate column        42701      read_projection base    42704
sort key not in columns 22023

The names are the bash suite's; the assertions are not.

Three other mechanism changes

• The EXPLAIN grep becomes a typed field. grep -c 'Columnar Projection: pc' is a
  substring test over text; the plan carries "Columnar Projection": "pc" as a property,
  so the port reads the value. The mutation proof below confirms it reads the name
  rather than the presence. The two negative arms use expect.plan_marker(absent=True),
  which refuses an empty plan — a plan that never arrived looks exactly like a plan
  carrying no projection.
• pgc_set_hash becomes expect.row_set — order-blind by declaration rather than by
  construction. A hash mismatch says two hashes differ; a row-set mismatch says which row.
• The second MVCC session becomes a second connection. The original drives a
  background psql -f fifo and polls its output file for a token, 200 times at 0.1s. A
  second psycopg connection removes the wait rather than shortening it. The original's
  two arms that exist only to name that timeout — session A opened snapshot and
  session A responded post-commit — are carried as the positive facts they are the
  negative of.

Arrays are cast ::text in SQL: psycopg returns a PostgreSQL array as a Python list,
so {1,2,3} arrives as [1, 2, 3] and a transcribed comparison would fail for a reason
that has nothing to do with projections.

Verified

On /usr/local/pg16a in the audit container, both legs exactly as CI runs them:

guard leg    342 collected, 0 fail             rc=0
cluster leg  406 collected, 0 fail, 0 unrun    rc=0
grader       75 literal, 0 template, 0 missing

Both counts in expected_tests.txt re-derived by collection on this tree, never by
adding a delta.

Mutation proof

Each mutation asserts it applied before its result is believed.

mutation	result
M1 — the #875 projection-writer reset removed	mid-transaction add: got 105 want 116, the defect's own signature (the back-fill plus the pre-add write, missing the 11 after). Mid-transaction drop: orphan projection storage, got 1 want 0. Both directions of the latch.
M2 — the planner never chooses a projection	covering query got None want 'pc'; after vacuum got None want 'pvp' — which is what proves the port reads the name
restored, both sources byte-identical	75 checks, 33 passed, 0 fail

Why this is stacked rather than based on main

Both PRs add a numbered TESTS.md section. Based on main they would both claim ## 40.
and whichever merged second would have to renumber — and #1066 has just shown what that
costs: a section inserted into the gap between another heading and its body left every
existing arm green, which is the defect #1067 adds an arm for.

Stacking makes the order deterministic: #1067 takes 40, this takes 41. If you would
rather have it against main, say so and I will rebase and renumber once #1067 lands.

Advances #432. I do not merge this.

🤖 Generated with Claude Code

https://claude.ai/code/session_012RSw4qMHS7ByE7PY8Ns4cs

[OffgridwithJD]

Adversarial self-review, at jd's request. One real defect, fixed in d6ae7bf. I do not approve my own PRs; this is a finding.

The aggregate arm compared two list reprs

expect.text(str(_rows(cur, "SELECT count(*), sum(a) FROM ps  WHERE ...")),
            str(_rows(cur, "SELECT count(*), sum(a) FROM ps_h WHERE ...")),
            "aggregate over projection scan matches oracle")

Same class as the one I found on #1067: comparing the repr of two lists reports these two strings differ and nothing about which row. expect.rows names the differing row. Fixed.

The bash original compares two psql outputs as text, so this is the port being stronger, not different — the same move the SQLSTATE arms make, and it should have been that way from the start.

What the audit found nothing wrong with

I went looking for the same weakness everywhere in this file:

• 15 row_set uses — all faithful. The bash originals are pgc_set_hash, which is order-blind by construction, so an order-blind comparison is the correct port. Using ordered_rows there would assert more than the original and would be wrong, not stronger: nothing about projection fan-out claims an order.
• 3 plan_marker(absent=True) — correct, and deliberately not _projection_in_plan(...) is None, because that form passes for a plan that never arrived.
• 2 remaining str() coercions (lines 187, 197) — booleans compared exactly. Redundant, not weak.
• The SQLSTATE arms — each code measured against the build rather than guessed, and all nine distinct.

Re-verified

grader       75 literal | 0 template | 0 missing
file         75 checks, 33 passed, 0 fail
guard leg    342 collected, 0 fail   rc=0
cluster leg  406 collected, 0 fail, 0 unrun   rc=0

Two things a reviewer should not take from me on trust

The counts will go stale. cluster_tests 406 and guard_tests 342 were derived by collection on this tree, which includes #1067. If anything else touching test/pytest/ merges first, re-derive by collection on the merged tree rather than taking the number in this diff — several open PRs currently carry values below what main already has.

The stack. This descends from #1067. Do not merge that one with --delete-branch (it closes this irreversibly) and use a merge commit rather than rebase-merge. Rebased and force-pushed with an explicit lease on the prior SHA.

Still outstanding from my earlier comment on #1068

## 41. here assumes #1067 takes 40. I offered #1068 section 40, so if that lands first both of mine renumber — one rebase, no work for anyone else. I have deliberately not done it yet: renumbering before #1068 actually merges would leave a gap at 40 in main.

[linuxhikerpm]

Stacked on #1067 as stated; this commit is the projections port (section 41). 75 literal names, one-for-one. The SQLSTATE refusals are strictly stronger than the bash expect_fail (any error), the EXPLAIN path reads the projection name rather than grepping for its presence, and the MVCC arm is a second connection instead of a polled fifo. expected_tests 373→406 is re-derived by collection, COMPLETE lists projections in C order, and CI is 14/14 green.

The last commit switching the aggregate arm from list-repr expect.text to expect.rows is the right instrument. Independent of test/projections.sh (docs-only cross-reference). Land #1067 first so TESTS.md numbering stays deterministic. Not merging.

[linuxhikerpm]

Re-reviewed HEAD d6ae7bf2 after #1070 landed on main.

The projections port still holds: catalog/fan-out/reconstruct/planner/rebuild/MVCC/lifecycle, expect.sqlstate instead of a bare error, and the aggregate arm compares expect.rows rather than two list reprs. COMPLETE is C-sorted (projection_privilege then projections). CI is green.

Conflicts with main are the same mechanical files as #1067. Land #1067 first, then rebase this stack.

[linuxhikerpm]

Re-reviewed HEAD a03a75eb (stack rebased onto #1067 after #1070). The projections port is unchanged in substance; land #1067 first. CI is green on the jobs that have finished.