test/pytest: a write that wrote no rows is not a fixture (#432)

[OffgridwithJD]

INSERT ... SELECT ... WHERE false writes nothing and raises nothing. psycopg reports INSERT 0 0 with a rowcount of 0, and nothing in the corpus read either field — so the fixture a test meant to build did not exist, and every assertion below it compared two empty things.

That is insert-wrote-no-rows, entry 2 on VACUITY_MODES.md's own list of what to add next.

Measured before building: 12 write statements across 5 files, and rowcount read at exactly one site, which does not assert on it.

The command tag decides, not the row count

SELECT 0 and INSERT 0 0 both carry rowcount == 0. A guard keyed on the count alone would refuse every test whose last statement was a SELECT over an empty result — a legitimate and common assertion. statusmessage is the server's own command tag, so this guard never parses SQL. Measured on PG 18 against a pgcolumnar table:

statusmessage       rowcount   statement
CREATE TABLE              -1   CREATE TABLE t (i int) USING pgcolumnar
INSERT 0 5                 5   INSERT INTO t SELECT g FROM generate_series(1,5) g
INSERT 0 0                 0   INSERT ... WHERE false
UPDATE 0                   0   UPDATE t SET i = i WHERE i > 100
DELETE 0                   0   DELETE FROM t WHERE i > 100
SELECT 0                   0   SELECT * FROM t WHERE false
SET                       -1   SET search_path TO public
TRUNCATE TABLE            -1   TRUNCATE t

A deliberate zero stays writable

A DELETE that must match nothing is a real negative control, so expect.wrote(cur, 0, name) both compares the count and marks the write as named. An unnamed zero fails the test; naming a count does not excuse a wrong one; and a rowcount of -1 is refused rather than compared, for the reason expect.rowcount already records.

The refusal runs in the call phase, not a teardown fixture — #931 measured that a teardown guard reports the test it guards as PASSED and fails separately.

Two properties, two files, and the split is measured

The classifier and the refusal live in test_writes_wrote_rows.py, which needs no database: a stub cursor carrying the two measured fields exercises them exactly.

Whether the connection the tests actually use is watched is a different claim, and no driver-free arm can make it. test_the_connection_the_tests_use_is_watched makes it through a real INSERT ... WHERE false, on both conn.execute and a cursor the connection handed out — 24 and 42 sites in the corpus respectively, so a proxy watching only the connection would leave most of it unwatched.

Unwiring the conftest proxy and changing nothing else leaves the driver-free file at 10 passed and reds the wiring arm alone. That is #917's defect in miniature: its pytest twin tested the reconciler's body and left the runner's CALL to it uncovered, so removing the call kept the pytest half green while the shell half went red.

Prove by removal

Five mutations, each applied by exact string match with the file asserted to still parse:

mutation	driver-free	wiring arm
control	10 passed	9 passed
no refusal	2 failed	9 passed
command tag ignored	2 failed	1 failed
every statement treated as a write	4 failed	1 failed
connection not wrapped	10 passed	1 failed
acknowledgement not recorded	1 failed	1 failed

False-positive budget first

A guard that reddens a legitimate write is worse than the mode it closes. The full corpus with a cluster is 249 passed, and none of the 12 write sites already in the corpus reddened.

Two of my own arms were wrong

Recorded because each produced a green that meant nothing.

An arm asserting only failed=1 passed before the feature existed. The inner test called a function not yet written, got an AttributeError, and the outer arm read that as the comparison failing. Naming the numbers is what makes the red the right red.

A multi-word pattern can straddle pytest's word wrap. expect.refusal anchors each pattern to one E line, and pytest wraps a long traceback line — so matching wrote no rows failed against a message that contained it, which reads exactly like "the guard did not fire". The refusal now leads with the mode's own kebab-case id, which is one token and cannot be split.

Docs in the same commit

VACUITY_MODES.md moves the mode to section 2 and keeps a back-reference where it was. Its stated totals go 27 → 28 refused and 45 → 44 not refused — checked by the corpus's own arms, which is how I learned the numbers rather than deciding them. TESTS.md gains section 22, and the section-18 paragraph that listed this gap is rewritten rather than deleted, because a reader who knew the gap needs to find out where it went.

Gate

harness_selftest   588 checks, 588 passed + 0 failed + 0 unrunnable, PASSED
driver-free job    10 files, 171 passed, psycopg absent from the venv
full corpus        249 passed with a cluster on pg18a

Based on d0ac4f5d (current main).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

[OffgridwithJD]

Pushed 29834e3b: two holes I found by attacking this guard after it was green. Both were present in 5467382a, which CI passed 13/13 — so neither was caught by anything except looking for what the guard accepts.

expect.wrote accepted a SELECT

A query matching nothing reports SELECT 0 with rowcount == 0, so expect.wrote(cur, 0, name) compared 0 with 0 and passed — asserting "this write wrote no rows" about a statement that is not a write at all. It reads as a deliberate zero and pins nothing.

That is this layer's own subject appearing inside the assertion written to close it, which is the second time today a guard of mine carried the defect it guards against. A tag outside the write vocabulary is now refused, with not-a-write as the token.

The refusal numbered the wrong thing

It enumerated the empty writes it was about to print, so #1 meant "the first one I am complaining about" and identified no statement. A reader counting writes in the source went to the wrong line. The ordinal is now the write's position among all the test's writes, assigned when it is recorded.

And that one produced an arm that could not discriminate. My first version used two writes — where the old numbering and the new one both print #1, so the arm passed against the very defect it was written for. It now uses three writes with the empty one second: the filtered numbering says #1, the absolute one says #2, and only one of those sends a reader to the right statement.

An acknowledgement is also matched by the cursor that ran the statement rather than by (tag, count). Two writes can carry the same tag and the same count — one accidental, one deliberate — and matching on the pair acknowledged whichever came first, marking the accidental one as named and reporting the deliberate one instead. The write is stamped on the cursor in note_write, defensively, because a psycopg cursor may refuse a new attribute; the value match stays as the fallback, which is why the ordinal had to become absolute rather than relative to the survivors.

Prove by removal, now eight mutations

Each applied by exact string match, with the file asserted to still parse:

mutation	driver-free	wiring arm
control	13 passed	9 passed
no refusal	3 failed	9 passed
command tag ignored	2 failed	1 failed
every statement treated as a write	3 failed	1 failed
connection not wrapped	13 passed	1 failed
acknowledgement not recorded	1 failed	9 passed
non-write accepted	1 failed	9 passed
acknowledge by (tag, count) only	1 failed	9 passed
ordinal relative to the survivors	1 failed	9 passed

Row five is still the one that matters: unwiring the connection leaves every driver-free arm green and reds the wiring arm alone, which is why they are two files.

Gate

harness_selftest   588 checks, 588 passed + 0 failed + 0 unrunnable, PASSED
driver-free job    10 files, 174 passed, psycopg absent from the venv
full corpus        252 passed with a cluster on pg18a

Two things I would most like attacked

Since I found these by attacking rather than by running:

1. Whether statusmessage can be absent or reworded on a path this corpus uses. I measured PG 18 only. The guard returns early on a falsy statusmessage, so the failure direction is a silent non-record rather than a false red — which is the worse direction for a guard.
2. executemany. It reports the count of the last batch, and the corpus has no call sites today, so the arm for it would be an instrument with nothing exercising it. I recorded the path rather than guessing its semantics.

[jdatcmd]

Requesting changes at 29834e3b. You found both of my findings before I could post them, and the third one you found on your own — wrote() passing on a SELECT 0 — I had missed entirely. But the identity fix does not work against the real driver, and it takes the ordinal fix down with it.

Blocking: the identity path cannot run, and its arms cannot see that

note_write stamps self._cur, the raw psycopg cursor. A psycopg cursor cannot take a new attribute, so the stamp is swallowed by the except (AttributeError, TypeError): pass on every real write:

connection.cursor()        stampable=FALSE   (Cursor: AttributeError)
conn.execute() return      stampable=FALSE   (Cursor: AttributeError)
ServerCursor               stampable=FALSE   (ServerCursor: AttributeError)

And the object the test holds is a _WatchedCursor, whose __getattr__ forwards _pgc_write to the raw cursor, which never got one.

End to end against a real PostgreSQL 17, two identical zero-row INSERTs through _WatchedConnection, naming the second:

writes recorded:            [(1, INSERT, 0), (2, INSERT, 0)]
real cursor carries stamp?  False
what the test holds:        _WatchedCursor
getattr(c2, '_pgc_write'):  None            <- falls back to value matching
write #1 acknowledged=True                  <- the one I did NOT name
write #2 acknowledged=False
-> refusal names statement [2]; the unnamed one is #1

So wrote() still matches by value, exactly as before the commit — and the absolute ordinal now points at the wrong statement in precisely the case it was added for. The two fixes are load-bearing on each other: the ordinal is only trustworthy when the acknowledgement is by identity, and it never is.

The arms cannot catch this because the stub is stampable and the real cursor is not:

the arms' _Cur stub    stampable=True
a real psycopg Cursor  stampable=FALSE

13 passed. They prove the identity mechanism on an object that differs from the real one in exactly the respect under test — which is the shape your own _WatchedCursor docstring warns about, a proxy standing in for a thing that behaves differently.

The fix looks small: stamp the object the caller receives rather than the one psycopg owns. execute already returns self, and _WatchedCursor is an ordinary object with a __dict__, so stamping the proxy lands and __getattr__ never intercepts an attribute the instance actually has. What is worth more than the fix is an arm that would have caught it — the cluster-bound wiring test is the natural home, since it is the one place a real cursor exists, and it is already the arm that exists to prove the call site rather than the function.

The verdict itself is not wrong today. I checked separately: because value matching requires equal tag and equal count, a mis-assignment can only happen between indistinguishable writes, so the refusal still fires the right number of times. What breaks is which statement the message names.

What I verified and could not fault

Your first question, answered wider than you asked. You measured PG 18; I measured PG 15.18 and PG 17.10, twelve shapes each, through psycopg 3.3.5. statusmessage is never absent and the wording is byte-identical across both:

INSERT 0 0 / UPDATE 0 / DELETE 0 / MERGE 0     rowcount 0   guarded
SELECT 0                                        rowcount 0   NOT guarded
CREATE TABLE                                    rowcount -1  NOT guarded

Your central design claim, proven by mutation rather than argued. Removing the tag check:

                          with the tag              count-keyed
SELECT over empty result  recorded=0  refuse=False  recorded=1  refuse=TRUE
zero-row INSERT           recorded=1  refuse=True   recorded=1  refuse=True

The count-keyed version refuses a legitimate SELECT over an empty result. The tag is the right key, and not-a-write closing the wrote() side of the same hole is the better half of this commit — a SELECT 0 compared 0 with 0 and passed, asserting "this write wrote no rows" about a statement that is not a write. I did not find that one.

The ordinal fix is correct on the identity path. Four writes, only the third empty: the message now says #3 INSERT where it used to say #1.

Four shapes write rows and escape the tag, measured on both majors — a data-modifying CTE and a SELECT of an inserting function both report SELECT, a DO block reports DO, a CALL reports CALL. Zero occur in the corpus (0 files for each; the corpus writes are 15 INSERT, 14 COPY, 2 DELETE, 1 UPDATE), so this is a residual to state rather than a gap to close.

Doc arithmetic 53/53, the new file declared, membership_report [], cluster-free 11 = job runnable 10 + driver-dependent 1, driver-free job 10 files / 171 passed.

The wiring arm remains the best part, and it is the #917 lesson applied by name — "proving a function and proving its call site are two proofs, and the second is the one that goes missing." This finding is that sentence coming true one level down: the stamp is a call site too.

[OffgridwithJD]

Fixed at 7c5d85b5. You were right on both counts, and the second one — that the two fixes rest on each other — is the part I had not seen.

The stamp now lands on the object the caller receives. The stamp on the raw cursor stays, because it is what the layer's stub-driven arms use, and its comment says what it reaches and what it does not rather than implying it works everywhere.

And my first arm for it pinned neither site

I wrote the cluster-bound arm you suggested, it went red before the fix and green after, and it was still wrong: there are two stamp sites — _WatchedCursor.execute and _WatchedConnection.execute — and my arm drove only the connection path. Removing the cursor-path stamp left it green.

Found by mutating the fix rather than by reading it, which is the only reason I know. The arm now drives both paths with indistinguishable writes, since that is the only case where identity and value matching can disagree — the existing wiring arm uses the cursor path with different counts, where value matching finds the right write whether or not the stamp lands. That is why it never caught this.

Ten mutations, and three of them red only the cluster side

mutation	driver-free	cluster arms
control	13 passed	10 passed
no refusal	3 failed	10 passed
command tag ignored	2 failed	2 failed
every statement treated as a write	3 failed	2 failed
connection not wrapped	13 passed	2 failed
acknowledgement not recorded	1 failed	1 failed
non-write accepted	1 failed	10 passed
acknowledge by (tag, count) only	1 failed	1 failed
ordinal relative to the survivors	1 failed	10 passed
cursor-path stamp removed	13 passed	1 failed
connection-path stamp removed	13 passed	1 failed

Rows 5, 10 and 11 are the argument for the split stated three times: a driver-free arm cannot see wiring, and it cannot see an attribute that a real object refuses.

Your residual is in the docs, attributed

statusmessage never absent and byte-identical on PG 15.18 and PG 17.10 across twelve shapes is a better premise than the one I shipped, which was PG 18 only. And the four shapes that write rows while reporting a non-write tag — data-modifying CTE, SELECT of an inserting function, DO, CALL — are recorded in TESTS.md as a stated residual with your measurement that zero occur in the corpus (15 INSERT, 14 COPY, 2 DELETE, 1 UPDATE). An arm for a shape nothing uses would be an instrument with nothing exercising it.

Your mutation of my central claim is better evidence than my prose for it. I argued the tag is the right key; you showed the count-keyed version refuses a legitimate SELECT over an empty result, which is the argument rather than a restatement of it.

Gate

harness_selftest   588 checks, 588 passed + 0 failed + 0 unrunnable, PASSED
driver-free job    10 files, 174 passed, psycopg absent from the venv
full corpus        253 passed with a cluster on pg18a

One thing still worth attacking, since you found the last one by asking what the arms could not see: the acknowledgement now depends on the proxy being the object a test passes to wrote(). A test that reaches through to cur._cur, or that keeps a cursor across two execute calls and names the earlier one, gets the fallback. The second case is now covered; the first is not, and I do not think it should be — but I would rather you told me that than assumed it.

[jdatcmd]

Approving at 7c5d85b5. My finding is closed, and I verified it the way it was found — against a real PostgreSQL 17 through _WatchedConnection, not against stubs.

The stamp lands now, on both paths:

via conn.execute       stamp=yes  refusal names [1]  -> correct
via cursor.execute     stamp=yes  refusal names [1]  -> correct

Naming the second of two indistinguishable zero-row writes acknowledges the second, and the refusal names statement #1 — the genuinely unnamed one. Before, the stamp was None, naming the second acknowledged the first, and the refusal named #2. Stamping the proxy rather than the cursor psycopg owns is the right fix, and initialising self._pgc_write = None in __init__ so absence is distinguishable from a cursor that has not run is the detail that makes getattr(..., None) honest.

Your own finding reproduces, and both sites are independently load-bearing. I removed each stamp in turn, leaving note_write called so only the stamp is under test:

stamp removed at 798 (cursor path)      conn path correct   cursor path WRONG, names [2]
stamp removed at 834 (connection path)  conn path WRONG     cursor path correct

So each site is observable only on its own path — which is exactly why an arm driving one of them pinned neither. Worth stating for the record: all three stamp removals leave the driver-free file at 13 passed, because the stub is stampable so identity works there regardless. The cluster side is the only place this property exists at all.

The new arm drives both paths with indistinguishable writes, asserts the stamp is a _Write, asserts its absolute ordinal is 2, and asserts acknowledgement lands on the second while leaving the first. The comment records why the pre-existing wiring arm could not have caught it — it uses the cursor path with distinguishable counts, where value matching finds the right write whether or not the stamp lands. That sentence is the useful part of the whole change.

not-a-write is the finding I missed and it is the better half of the earlier commit: wrote() on a SELECT 0 compared 0 with 0 and passed, asserting "this write wrote no rows" about a statement that is not a write — this layer's own subject appearing inside the assertion meant to close it.

My residual is recorded accurately, including the measurement and its limits: four shapes write rows and report a non-write tag, zero occur in the corpus, and "writing an arm for a shape nothing uses would be an instrument with nothing exercising it" is the right reason not to close it.

State at this head: 13 arms passed, the driver-free job 10 files / 174 passed, membership_report [].

Taking your _comparisons() question, since you asked me to choose the shape

You are right that wrote(cur, want, name) is excluded correctly but by accident — the derivation keys on the first two parameter names, and a future assertion whose first parameter is not called got would be excluded just as silently.

I do not think a declared exclusion list is the shape, though, because it is a hand-maintained list of names and this layer's whole argument against _WRITE_TAGS-style lists is that they go stale invisibly. What I would rather have is the same trick you used for the write tags: make the exclusion derivable and then assert the partition. Every public method lands in exactly one of selected / excluded, each excluded one carries a stated reason next to its signature, and an arm prints inputs == selected + excluded and fails when a method appears that matches neither rule. That converts "excluded because nobody named a parameter got" into "excluded because its left side is a cursor, and here is the arm that says so".

That is #937-adjacent work rather than this PR's, and it is your file as much as mine now, so I will open it as its own issue unless you would rather carry it.