Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,61 @@ true until the next version shipped.

### Added

- A UNIQUE-constraint check passed on any psql failure, and a recursive sweep passed on a
tree it never read (#1033).

`native_recluster.sh` decided "unique still enforced" from psql's exit code:

psql ... -c "INSERT INTO n VALUES (1, 0, 0, 'x');" >/dev/null 2>&1 && echo no || echo yes

`|| echo yes` cannot tell a unique violation from any other failure, and
`>/dev/null 2>&1` discards the message so nothing else can either. Measured on the
identical expression: a missing table, a wrong port, psql absent from PATH and a syntax
error all produced `yes`.

THE CONTRAST, RUN END TO END on PG 18 rather than argued. The same mutation -- point the
probe at a table that does not exist -- against both versions of the arm:

main's arm PASS "unique still enforced" suite PASSED, rc=0, 12 passed
this arm FAIL got [42P01] want [23505] suite FAILED, rc=1

And the arm is observable in the direction that matters. Replace the unique index with a
plain one and the duplicate INSERT succeeds. The arm goes red there too, so it tests the
constraint rather than merely the SQLSTATE plumbing.

The arm now reads the SQLSTATE. 23505 is `ERRCODE_UNIQUE_VIOLATION` and comes from the
index; 42P01, 42601 and a connection failure do not. `arrow_import.sh` and `audit.sh`
already read SQLSTATE this way, so this is the convention rather than a new one.

SECOND, A SWEEP WITH NO POPULATION. `local_open_race_free.sh` asserted that a removed
helper leaves no trace:

"$(grep -rc 'PgColumnarRejectNonRegularFile' "$SRC" | awk -F: '{s+=$2} END{print s+0}')" "0"

`grep -rc` prints one `file:count` line per file and prints NOTHING for a path it cannot
open, and `END{print s+0}` then manufactures the `0` the check wants. The three arms above
it read `$OBJ`, not `$SRC`, so nothing established that `$SRC` was a source tree.
Measured:

SRC=src files=56 premise=yes arm=0 both agree
SRC=/no/such/tree files=0 premise=no arm=0 the premise catches it, the arm alone passes

A premise now counts the lines the recursive grep emitted, which is exactly what the
arm's `awk` sums over.

NEITHER SUITE NEEDS A LEDGER ROW, which is why these two and not a third.
`check_ledger.tsv` holds zero rows for `native_recluster` and `local_open_race_free`, so
the gate cannot refuse a new check there.

A third site has the same shape and is left alone: `selftest/400`'s tree-wide piped-loop
sweep. It lives in `harness_selftest`, which the ledger covers with 934 rows. Its detector
IS premised, since `:549` proves it fires on a planted offence, and only its glob
population is not. So it belongs to a change that carries the five-major ledger merge.

Verified on PG 18 in the container: `local_open_race_free.sh` PASSED, 11 checks;
`native_recluster.sh` PASSED, 12 checks; both mutations red; main's arm green under the
same mutation.

- `native_ownership.sh` has a pytest twin, and it asserts the SQLSTATE (#432).

Nine maintenance and DDL functions, each refused to a non-owner with 42501 rather
Expand Down
8 changes: 8 additions & 0 deletions test/local_open_race_free.sh
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,14 @@ check "the opener fstats the fd it holds (checks what it opened)" \
"$([ "$(opener_body | grep -c 'fstat(')" -ge 1 ] && echo yes || echo no)" "yes"
check "the opener never stats a path before opening it (no TOCTOU)" \
"$(opener_body | grep -Ec 'stat\((const )?path|stat\("|stat\(path')" "0"
# THE POPULATION, because this arm can pass on a tree it never read. `grep -rc`
# prints `file:count` per file and prints NOTHING for a path it cannot open, and
# `END{print s+0}` then manufactures the 0 the check wants. The three arms above read
# `$OBJ`, not `$SRC`, so nothing here established that `$SRC` is a source tree. Counted
# the same way the arm sums: one line of `grep -rc` output per file read.
_lorf_files="$(grep -rc 'PgColumnarRejectNonRegularFile' "$SRC" 2>/dev/null | grep -c .)"
check "premise: the recursive sweep read source files under \$SRC" \
"$([ "${_lorf_files:-0}" -ge 5 ] && echo yes || echo no)" "yes"
check "the racy stat-before-open helper is gone" \
"$(grep -rc 'PgColumnarRejectNonRegularFile' "$SRC" | awk -F: '{s+=$2} END{print s+0}')" "0"

Expand Down
19 changes: 15 additions & 4 deletions test/native_recluster.sh
Original file line number Diff line number Diff line change
Expand Up @@ -75,10 +75,21 @@ check "row count after recluster" "$(q 'SELECT count(*) FROM n;')" "40960"

# Online index maintenance: index scan returns each live row exactly once.
check "index scan returns each row once" "$(idxcount)" "40960"
check "unique still enforced" \
"$(env PATH="$PGC_BINDIR:$PATH" psql -h 127.0.0.1 -p "$PGC_PORT" -U postgres -d "$PGC_DB" -At -v ON_ERROR_STOP=1 \
-c "INSERT INTO n VALUES (1, 0, 0, 'x');" >/dev/null 2>&1 && echo no || echo yes)" \
"yes"
# THE SQLSTATE, NOT THE EXIT CODE. This arm was
# `psql ... && echo no || echo yes`, which reports "yes" for every failure psql can
# have. Measured on the identical expression: a missing table, a wrong port, psql
# absent from PATH and a syntax error all produced "yes", so the arm passed with the
# unique index doing nothing. 23505 is ERRCODE_UNIQUE_VIOLATION and comes from the
# index; 42P01, 42601 and a connection failure do not. The project already reads
# SQLSTATE this way in `arrow_import.sh` and `audit.sh`.
dup_sqlstate() { # -> the SQLSTATE of a duplicate insert, or empty if it succeeded
env PATH="$PGC_BINDIR:$PATH" psql -h 127.0.0.1 -p "$PGC_PORT" -U postgres \
-d "$PGC_DB" -qtA -v ON_ERROR_STOP=0 -v VERBOSITY=sqlstate \
-c "INSERT INTO n VALUES (1, 0, 0, 'x');" 2>&1 \
| sed -n 's/^ERROR: \([0-9A-Z]\{5\}\).*/\1/p' | head -1
}
check "unique still enforced, by SQLSTATE 23505 and not by any failure" \
"$(dup_sqlstate)" "23505"

# Lock level: ShareUpdateExclusiveLock, never AccessExclusiveLock.
locks="$(env PATH="$PGC_BINDIR:$PATH" psql -h 127.0.0.1 -p "$PGC_PORT" -U postgres \
Expand Down
Loading