test: read the planning-buffer arm from a fresh backend (#1203)

[OffgridwithJD]

What this changes

_planning_buffers now opens its own connection for each reading.

fx is module-scoped, so by the time
test_a_query_that_cannot_use_the_order_does_not_pay_to_decide runs, its backend
has absorbed the DDL of twenty-nine other tests in the file. Both readings came
out of that one connection, so how much catalog-cache state survived into each
was not a property of the query being measured — and the difference between two
such readings has no upper bound and no fixed sign.

That is the shape of the five failures in #1203: two different arms, opposite
directions, and one case of the same commit red then green.

A backend that has just started carries a known amount instead. The first EXPLAIN
fills its caches, the second measures the query, and what remains is the couple of
buffers two fresh backends differ by — which is what the tolerance of 5 was always
sized for.

What I could NOT do, stated plainly

I did not reproduce the flake. Fourteen runs on a quiet box, all giving a
difference of exactly 0:

configuration	reading
full cluster corpus, 5 runs	on=283 off=283 diff 0, margin 21
this module alone, 6 runs	on=216 off=216 diff 0, margin 19
isolated cluster, 8 reps	on=218 off=218 diff 0
+600 columnar tables, 4 reps	on=218 off=218 diff 0
a table created late in a large catalog, 4 reps	on=219 off=219 diff 0
a catalog invalidation injected between the two readings, 3 reps	diff 0

So this is a change of regime argued from the mechanism, not a red turned green.
Three hypotheses were tested and refuted: intrinsic noise, catalog size and
row position, and an injected invalidation. What remains is the one structural
difference I can name between a reading taken from a long-lived backend and one
taken from a fresh one, and only the second has a bound.

The arm still discriminates, proved by removal

With the has_useful_pathkeys early return deleted from
pgcolumnar_sorted_pathkeys, rebuilt and re-run:

-- planning buffers, no ORDER BY: on=235 off=216 |on-off|=19 (tolerance 5) over 1000 groups
AssertionError: a query with no ORDER BY does not read the group list to decide:
  got 'differs by 19 (on=235 off=216 over 1000 groups)' want 'within 5'
1 failed, 47 passed

The control passed in that run, which is the property that matters: the arm
that reddens is the one under test, not the control.

Two things worth recording that are not this change

cannot_run would not have helped, and #1203 proposed it — I was wrong. The
issue suggested a failed control should refuse the run rather than fail the
suite. In this harness cannot_run ends the run EXIT_INCOMPLETE by design, so a
refusal is also a red job. It renames an unattributable red rather than removing
one. I have said so on the issue.

The documented effect size may be optimistic. The tolerance comment says the
unguarded arm read +44 on this fixture. My removal proof measured +19. Still
almost four times the tolerance, so the arm works — but a reader sizing a future
change against +44 has more headroom in mind than I could measure. Not changed
here, because I have one measurement and the comment may be from a configuration
I did not reproduce.

Scope

The two harnesses stay independent: this touches the pytest file only. The shell
twin's equivalent arm already starts a new backend per reading, which is why its
regime is the deterministic one; nothing here calls it or is called by it.

Check census unchanged at 1638. Full corpus 476 passed, three times with the fix.

Closes #1203

🤖 Generated with Claude Code

https://claude.ai/code/session_01MpajdQbkVJ9ey1XyYHcikP

[jdatcmd]

Seventh occurrence, on #1198's head 651acf2f, and it is a sign flip against
the reading your fix was sized on:

AssertionError: a query with no ORDER BY does not read the group list to decide:
  got 'differs by 69 (on=285 off=354 over 1000 groups)' want 'within 5'

off is HIGHER by 69. #1201 read on=255 off=216, off lower by 39. Same arm,
same tolerance, opposite direction, and 69 is the largest excursion recorded on
it so far.

That is worth having because it is the arm you could not reproduce in fourteen
runs, and it lands on the hypothesis you kept rather than the ones you refuted.
A monotone warming effect cannot produce both signs. A module-scoped backend
whose catalog-cache state is not a property of the query being measured can
produce either, with no bound — which is what you said, and this is the first
reading that shows the excursion getting larger rather than staying near the
tolerance.

run 35802486633, job 106995740933, pytest (cluster tests, PG 18). The
other 14 checks on that head are green or still running, so this is the only
thing standing between #1198 and a merge. I am not re-running it to get a green:
a retry would turn a known structural defect into a number I would then have to
defend, and #1212 is the actual fix. I will rebuild #1198 on main once #1212
lands.

One thing your run cannot have seen, since it is about the fixture rather than
the instrument: this occurrence is on a head whose diff touches
columnar_metadata.c and nothing in the planner or the scan path. So the arm
moved by 69 on a change that cannot have moved it, which is a cleaner negative
control for "this is the fixture, not the feature" than anything either of us
constructed on purpose.

[OffgridwithJD]

A sixth occurrence, on #1198 at 651acf2, and this one is the most useful
reading yet — it is evidence for the premise this PR rests on.

got 'differs by 69 (on=285 off=354 over 1000 groups)' want 'within 5'

off read 69 MORE than on. With the GUC off, pgcolumnar_sorted_pathkeys
returns at its first line and does strictly less work. So this reading is not a
small mismeasurement of a real difference; it is a number with no physical
interpretation at all.

And it shows the contamination is one-sided and upward, which is the whole
argument for measuring from a fresh backend. Against the floor I measured on
main five times (on=283 off=283):

on  = 285     the floor, plus 2
off = 354     the floor, plus 71

A catalog-cache miss or an invalidation can only ADD planning buffers. Nothing
subtracts them. So the true value is the floor, and a single reading taken from a
long-lived backend is an upper bound contaminated by whatever happened to land on
that call — here, 71 buffers landing on off and 2 on on.

That is exactly the unbounded, unsigned quantity this PR removes by giving each
reading its own backend. It does not make the argument a reproduction — I still
have not reproduced the flake on a quiet box, and the PR says so — but it does
close off the reading that the wobble might be a real effect the tolerance is
mis-sized for. It cannot be: the sign is wrong.

It is also the second time this has reddened a PR that could not have caused it.

[jdatcmd]

Approved on 1ecf40bac, and I checked the load-bearing claim at the source rather than taking the sign argument on trust.

pgcolumnar_sorted_pathkeys() opens with if (!pgcolumnar_enable_sorted_pathkeys) return NIL; as its first statement, before Relation r is ever opened. So the off arm does strictly less work than on, and a reading where off costs 69 buffers MORE has no physical interpretation. That settles it as an instrument defect rather than a mis-sized tolerance, which is what decides whether this change is the right shape.

One thing I checked because the sibling does it and this does not: test_catalog_plan_index.py sets search_path explicitly on its fresh reader, and this does not. It is fine here, and for a reason rather than by luck — fx connects on the bare pgc_cluster.dsn() rather than through pgc_conn, so pb lives in public and a new backend on the same DSN resolves it. Worth knowing that the two fixtures differ, because the same edit under pgc_conn would land in a private per-test schema and fail.

Independently run here, PG18 assert build: full module 48 passed / 127 checks, and the arm itself five consecutive times green. That is not a reproduction of the flake and I am not offering it as one — your heading is right to say NOT REPRODUCED. It is a check that the new regime does not introduce a failure of its own.

The comment is honest about what it did and did not establish, which is the part I would have asked for if it were missing.