test: a pytest harness with a vacuity-refusal layer, 25 tests (#432)

CHANGELOG.md

@@ -194,6 +194,36 @@ true until the next version shipped.
 ## [1.0-alpha3] - 2026-09-02
 
 ### Added
+- A pytest harness beside the bash suites, with a layer that refuses tests which
+  assert nothing (#432).
+
+  `test/pytest/` connects through `psycopg` rather than parsing `psql` output, so a
+  test can assert the TYPE as well as the value: `psql -At` returns text, and an
+  `int4` `1` and a `text` `'1'` are the same string to a bash oracle.
+
+  This ports ONE bash suite. `test/` carries 4,429 anchored assertions across 256
+  suites, so this is 0.18% of them and is not coverage. The layer is the point.
+
+  A pytest run fails open in several ways this project has already been bitten by:
+  a test that asserts nothing passes, a filter that selects nothing exits 0, and a
+  fixture that skips greens every test under it. `pgc_vacuity.py` is loaded for
+  every run and refuses those shapes, along with an empty result compared with an
+  empty result, a value compared against itself, a plan matched by substring rather
+  than by typed key, an absence claim over an empty plan, `cursor.rowcount` of
+  `-1`, and a broad `except` found by walking the AST rather than by line regex.
+  `xfail_strict` is on, and `--pgc-expect-tests N` asserts the run's own shape and
+  refuses `N = 0`.
+
+  Every refusal has a red test in `test_layer.py` that runs pytest inside pytest
+  and asserts on the INNER run's outcome, which is what proves a guard refuses
+  rather than assuming it. Each records the bare-pytest behaviour it exists to
+  stop; every one of those measurements exited 0. Four of the ten are positive
+  controls, so a guard that starts rejecting good tests reddens there first.
+
+  Not registered in `test/run_all_versions.sh`. That would add a `psycopg` build
+  dependency to every CI leg for 0.18% of the assertions; `test/pytest/README.md`
+  records what registering would cost and what has to be true first.
+
 
 - A nanosecond Arrow import says how many values lost precision, and still
   imports every row.

test/lib.sh

@@ -134,6 +134,51 @@ pgc_so_line() {
 	fi
 }
 
+# pgc_build_and_install SRCDIR PG_CONFIG MAJOR
+#
+# Build and install the extension, or fail. Extracted from pgc_setup UNCHANGED so
+# the pytest harness can drive the same implementation instead of carrying a
+# second one: test/pytest/ never built or installed, so it reported 25 passed
+# against source carrying `#error THIS SOURCE IS BROKEN AND CANNOT BUILD`
+# (@jdatcmd, #897 review). Two implementations of "is the thing under test the
+# thing in this tree" would drift, and the drift would be invisible in exactly
+# the way that defect was.
+#
+# Returns non-zero rather than calling exit, so a caller that is not a suite --
+# the Python harness -- can turn it into its own kind of failure. pgc_setup
+# passes the exit through, so bash behaviour is unchanged.
+pgc_build_and_install() {
+	_pgc_bi_src="$1"
+	_pgc_bi_cfg="$2"
+	_pgc_bi_major="$3"
+	_pgc_bi_stamp="$_pgc_bi_src/.pgc_built_for_major"
+	_pgc_bi_had="$(cat "$_pgc_bi_stamp" 2>/dev/null | tr -dc '0-9')"
+	_pgc_bi_objs=no
+	[ -n "$(find "$_pgc_bi_src/src" -maxdepth 1 -name '*.o' -print -quit 2>/dev/null)" ] && _pgc_bi_objs=yes
+	# Objects from another major link but do not load (#536).
+	if [ "$(pgc_build_needs_clean "$_pgc_bi_had" "$_pgc_bi_major" "$_pgc_bi_objs")" = yes ]; then
+		pgc_build_stale_message "$_pgc_bi_had" "$_pgc_bi_major"
+		make -C "$_pgc_bi_src" clean PG_CONFIG="$_pgc_bi_cfg" >/dev/null 2>&1 || true
+	fi
+	echo "-- building"
+	if ! make -C "$_pgc_bi_src" PG_CONFIG="$_pgc_bi_cfg" >/dev/null; then
+		echo "FATAL: the build failed, so there is nothing new to test" >&2
+		echo "       (refusing to report checks against the previously installed .so)" >&2
+		return 1
+	fi
+	# Stamped only after a build that succeeded. printf '%s\n', NOT '%s\\n':
+	# the doubled backslash writes the four bytes 1 9 \ n, which only worked
+	# because the reader strips non-digits. Caught in review, not by a test.
+	pgc_write_build_stamp "$_pgc_bi_stamp" "$_pgc_bi_major"
+	echo "-- installing"
+	if ! make -C "$_pgc_bi_src" install PG_CONFIG="$_pgc_bi_cfg" >/dev/null; then
+		echo "FATAL: the install failed, so the .so under test is not the one just built" >&2
+		echo "       (refusing to report checks against the previously installed .so)" >&2
+		return 1
+	fi
+	return 0
+}
+
 pgc_setup() {
 	PGC_PG_CONFIG="${1:-/usr/local/pg17/bin/pg_config}"
 	PGC_BINDIR="$("$PGC_PG_CONFIG" --bindir)"
@@ -197,31 +242,7 @@ pgc_setup() {
 	# installed .so and saw the same hash either side of a source change that could
 	# not have produced it.
 	if [ -z "${PGC_SKIP_BUILD:-}" ]; then
-		# Objects from another major link but do not load (#536).
-		_pgc_stamp="$PGC_SRCDIR/.pgc_built_for_major"
-		_pgc_had="$(cat "$_pgc_stamp" 2>/dev/null | tr -dc '0-9')"
-		_pgc_objs=no
-		[ -n "$(find "$PGC_SRCDIR/src" -maxdepth 1 -name '*.o' -print -quit 2>/dev/null)" ] && _pgc_objs=yes
-		if [ "$(pgc_build_needs_clean "$_pgc_had" "$PGC_MAJOR" "$_pgc_objs")" = yes ]; then
-			pgc_build_stale_message "$_pgc_had" "$PGC_MAJOR"
-			make -C "$PGC_SRCDIR" clean PG_CONFIG="$PGC_PG_CONFIG" >/dev/null 2>&1 || true
-		fi
-		echo "-- building"
-		if ! make -C "$PGC_SRCDIR" PG_CONFIG="$PGC_PG_CONFIG" >/dev/null; then
-			echo "FATAL: the build failed, so there is nothing new to test" >&2
-			echo "       (refusing to report checks against the previously installed .so)" >&2
-			exit 1
-		fi
-		# Stamped only after a build that succeeded. printf '%s\n', NOT '%s\\n':
-		# the doubled backslash writes the four bytes 1 9 \ n, which only worked
-		# because the reader strips non-digits. Caught in review, not by a test.
-		pgc_write_build_stamp "$_pgc_stamp" "$PGC_MAJOR"
-		echo "-- installing"
-		if ! make -C "$PGC_SRCDIR" install PG_CONFIG="$PGC_PG_CONFIG" >/dev/null; then
-			echo "FATAL: the install failed, so the .so under test is not the one just built" >&2
-			echo "       (refusing to report checks against the previously installed .so)" >&2
-			exit 1
-		fi
+		pgc_build_and_install "$PGC_SRCDIR" "$PGC_PG_CONFIG" "$PGC_MAJOR" || exit 1
 	else
 		# Named because the variable is not what it says. It reads as "skip the
 		# build" and means "skip the build AND the install, and test whatever is

test/pytest/README.md

@@ -0,0 +1,59 @@
+# Running the pytest harness
+
+This is the issue #432 pilot. It runs beside `test/*.sh`, and replaces nothing.
+
+- `TESTS.md` in this directory documents every test and every assertion helper.
+- `design/ISSUE_432_PYTEST_HARNESS.md` holds the design and the measurements
+  behind each guard.
+
+## Prerequisites
+
+The interpreter is marked `EXTERNALLY-MANAGED`, so install into a virtual
+environment rather than into system Python:
+
+```sh
+apt-get install -y python3.14-venv        # ensurepip is not in the base image
+python3 -m venv /root/pyenv
+/root/pyenv/bin/pip install -r test/pytest/requirements-test.txt
+```
+
+## Running it
+
+```sh
+cd test/pytest
+PYTHONPATH=. /root/pyenv/bin/pytest                      # serial
+PYTHONPATH=. /root/pyenv/bin/pytest -n 4                 # four workers
+PYTHONPATH=. /root/pyenv/bin/pytest --pgc-expect-tests 24 # assert the run's shape
+PGC_PG_CONFIG=/usr/local/pg19a/bin/pg_config PYTHONPATH=. /root/pyenv/bin/pytest
+```
+
+Each worker builds its own throwaway cluster on a port derived from its worker id,
+and drops it at session end. Nothing survives a run.
+
+## Checking a port against its bash original
+
+```sh
+/root/pyenv/bin/python test/pytest/compare_to_bash.py \
+    test/native_projection.sh test/pytest/test_native_projection.py
+```
+
+It compares the two by assertion NAME and exits non-zero if the bash suite asserts
+a property the port does not. A port keeps this working by passing each assertion
+the same name string the bash check uses.
+
+## This is not in the gate yet
+
+`test/run_all_versions.sh` does not run these tests, and neither does CI. That is a
+decision with a price, recorded in section 1a of the design document: `pgc_skip`
+treats a missing dependency as a failure rather than a skip, so registering this run
+in `SUITES` would redden every CI job until `ci.yml` installs from
+`requirements-test.txt`. Until someone takes that decision, run it by hand.
+
+## Warnings
+
+The vacuity layer is loaded through `pytest.ini` and cannot be turned off by a test
+file. A test that concludes nothing fails, a bare skip fails the run, and a
+comparison that could not have failed is refused. If a guard blocks something
+legitimate, the escape hatches take a reason rather than a flag, and every one of
+them is listed in the design document. Adding a new escape hatch needs a red test
+that proves the guard still fires without it.