Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -503,6 +503,46 @@ true until the next version shipped.
there is no aarch64 box here to test it on. What this change does is make the
next nightly report the symbol names instead of a bare exit status, so the fix
after it is measured rather than reasoned about.
- The cross-major preflight leaves no object tree behind, on a box with no
`pg_config` on `PATH` (#1219).

`test/build_all_versions.sh` ends with a clean, and its comment says why:
"Leave no object tree behind from whichever major happened to be last: the
next build against a different major would link objects compiled for this
one." It was written as

```sh
make -C "$SRCDIR" clean >/dev/null 2>&1 || true
```

with no `PG_CONFIG` and its failure swallowed. PGXS resolves `pg_config` from
`PATH`, so on a box without a packaged one that `make` fails, the `|| true`
eats it, and the tree keeps the last major's objects while the script prints
PASSED. Measured, same tree, pg18a then pg19a:

```
normal PATH built 2 of 2 PASSED objects left: 0
PATH with no pg_config built 2 of 2 PASSED objects left: 36
```

**Passing a `pg_config` is not the fix.** `make clean` needs PGXS loaded to do
anything, objects live in `src/` *and* `objstore/`, and a clean whose failure
is swallowed cannot be told from one that worked. The sweep now removes by
`find` -- needing no `pg_config` -- and then verifies, through two functions the
selftest drives directly:

```
pgc_bav_tree_has_objects DIR -> yes | no
pgc_bav_clean_tree SRCDIR [PG_CONFIG] -> clean | dirty
```

End to end, the real script under `env -i PATH=<no pg_config>` now leaves
**0 objects** where it left 36.

The consequence was bounded -- #1221's DWARF provenance catches foreign objects
on the next suite and cleans them -- so this cost a rebuild rather than a wrong
install. What it did not cost is nothing, and the comment promised something it
did not always do.

- `pgc_reconcile_records` no longer reports an impossible mismatch on a log it
cannot measure (#1242). Handed a log that states `checks run: N` and carries
Expand Down
52 changes: 50 additions & 2 deletions test/build_all_versions.sh
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,48 @@ fi
echo "== pgColumnar build check across majors =="

failed=0
# Remove every build artifact from a tree, and SAY WHETHER IT WORKED (#1219).
#
# The final sweep used to be `make -C "$SRCDIR" clean >/dev/null 2>&1 || true`
# with no PG_CONFIG. PGXS resolves `pg_config` from PATH, so on a box with no
# packaged one that make fails, the `|| true` eats the failure, and the tree
# keeps the last major's objects while the script still prints PASSED. Measured,
# same tree, pg18a then pg19a:
#
# normal PATH built 2 of 2 PASSED objects left: 0
# PATH with no pg_config built 2 of 2 PASSED objects left: 36
#
# PASSING A pg_config IS NOT THE FIX. `make clean` needs PGXS loaded to do
# anything at all, objects live in src/ AND objstore/, and a clean whose failure
# is swallowed cannot be told from one that worked. So: try make when a
# pg_config is at hand, because PGXS knows about artifacts this sweep does not
# name; then remove by find, which needs no pg_config; then VERIFY, because a
# sweep that reports nothing is the defect being fixed.
# THE DETECTOR IS ITS OWN FUNCTION so it can be judged directly. Folded into
# the cleaner it was unreachable: in every fixture the sweep works, so `clean` is
# the right answer and a cleaner that ALWAYS says `clean` reddens nothing.
# Measured -- that mutation passed all five arms. Splitting it out is what gives
# the detection logic a killer; the belt-and-braces re-check inside the cleaner
# still only fires when the sweep fails, which cannot be staged as root, and
# that residue is recorded rather than papered over.
pgc_bav_tree_has_objects() { # pgc_bav_tree_has_objects DIR -> yes|no
local _d="${1:-}"
[ -n "$_d" ] && [ -d "$_d" ] || { echo no; return; }
if [ -n "$(find "$_d" \( -name '*.o' -o -name '*.bc' -o -name '*.so' \) -type f 2>/dev/null | head -1)" ]; then
echo yes
else
echo no
fi
}

pgc_bav_clean_tree() { # pgc_bav_clean_tree SRCDIR [PG_CONFIG] -> clean|dirty
local _d="${1:-}" _pgc="${2:-}"
[ -n "$_d" ] && [ -d "$_d" ] || { echo dirty; return; }
[ -n "$_pgc" ] && make -C "$_d" clean PG_CONFIG="$_pgc" >/dev/null 2>&1
find "$_d" \( -name '*.o' -o -name '*.bc' -o -name '*.so' \) -type f -delete 2>/dev/null
[ "$(pgc_bav_tree_has_objects "$_d")" = yes ] && echo dirty || echo clean
}

built=0
for pgc in "${CONFIGS[@]}"; do
if [ ! -x "$pgc" ]; then
Expand Down Expand Up @@ -129,8 +171,14 @@ for pgc in "${CONFIGS[@]}"; do
done

# Leave no object tree behind from whichever major happened to be last: the next
# build against a different major would link objects compiled for this one.
make -C "$SRCDIR" clean >/dev/null 2>&1 || true
# build against a different major would link objects compiled for this one. The
# verdict is read rather than discarded -- the previous form swallowed its own
# failure and left the tree dirty on any box without a pg_config on PATH (#1219).
if [ "$(pgc_bav_clean_tree "$SRCDIR" "${pgc:-}")" != clean ]; then
echo "build_all_versions: objects remain in $SRCDIR after the final sweep;" >&2
echo " the next build against another major would link them" >&2
failed=1
fi

# What was actually compiled, next to what was asked for. Without this line the
# verdict below collapses "built five" and "built none" into the same word: a
Expand Down
21 changes: 19 additions & 2 deletions test/check_ledger.tsv
Original file line number Diff line number Diff line change
Expand Up @@ -514,11 +514,28 @@ harness_selftest 280-the-shared-cluster-config-must-not premise: the cluster-con
harness_selftest 280-the-shared-cluster-config-must-not premise: the detector fires on the line that caused #799 15;16;17;18;19 never -
harness_selftest 280-the-shared-cluster-config-must-not the per-suite escape hatch PGC_EXTRA_CONF is still applied to the config 15;16;17;18;19 never -
harness_selftest 280-the-shared-cluster-config-must-not the shared cluster config sets no pgcolumnar.* GUC 15;16;17;18;19 never -
harness_selftest 290-a-preflight-that-built-nothing a preflight that built nothing does not report PASSED 15;16;17;18;19 never -
harness_selftest 290-a-preflight-that-built-nothing a preflight that built nothing exits non-zero 15;16;17;18;19 never -
harness_selftest 290-a-preflight-that-built-nothing a preflight that built nothing does not report PASSED 15;16;17;18;19 2026-09-24 the zero-build refusal removed, so a preflight that built nothing reports PASSED
harness_selftest 290-a-preflight-that-built-nothing a preflight that built nothing exits non-zero 15;16;17;18;19 2026-09-24 the zero-build refusal removed, so a preflight that built nothing reports PASSED
harness_selftest 290-a-preflight-that-built-nothing a preflight that built nothing says how many it built 15;16;17;18;19 never -
harness_selftest 290-a-preflight-that-built-nothing a tree full of objects is reported clean after the sweep 15;16;17;18;19 2026-09-24 the cleaner always answers dirty, whatever the detector says;the find sweep removed, so only make clean is relied on;the sweep looks only in src/, missing objstore/
harness_selftest 290-a-preflight-that-built-nothing a tree holding objects is detected as holding them 15;16;17;18;19 2026-09-24 the detector blinded, so it never reports an object
harness_selftest 290-a-preflight-that-built-nothing a tree with nothing to remove is clean, not an error 15;16;17;18;19 2026-09-24 the cleaner always answers dirty, whatever the detector says;the find sweep removed, so only make clean is relied on;the sweep looks only in src/, missing objstore/
harness_selftest 290-a-preflight-that-built-nothing and a directory that is not there holds none 15;16;17;18;19 never -
harness_selftest 290-a-preflight-that-built-nothing and a directory that is not there is dirty rather than silently clean 15;16;17;18;19 2026-09-24 the missing-or-empty directory guard removed
harness_selftest 290-a-preflight-that-built-nothing and clean when it reports none 15;16;17;18;19 2026-09-24 the cleaner always answers dirty, whatever the detector says
harness_selftest 290-a-preflight-that-built-nothing and it left the source alone 15;16;17;18;19 2026-09-24 the sweep loses its name filter and deletes every file it finds
harness_selftest 290-a-preflight-that-built-nothing and no argument at all is dirty rather than sweeping the cwd 15;16;17;18;19 2026-09-24 the missing-or-empty directory guard removed
harness_selftest 290-a-preflight-that-built-nothing and the detector now says the tree holds none 15;16;17;18;19 2026-09-24 the find sweep removed, so only make clean is relied on;the sweep looks only in src/, missing objstore/
harness_selftest 290-a-preflight-that-built-nothing and the objects are gone, including the ones outside src/ 15;16;17;18;19 2026-09-24 the find sweep removed, so only make clean is relied on;the sweep looks only in src/, missing objstore/
harness_selftest 290-a-preflight-that-built-nothing and the old swallow-everything form is gone 15;16;17;18;19 2026-09-24 the old swallow-everything make clean put back alongside the new call
harness_selftest 290-a-preflight-that-built-nothing premise: and it built none of them 15;16;17;18;19 never -
harness_selftest 290-a-preflight-that-built-nothing premise: and the leftover detector it reads is exposed too 15;16;17;18;19 never -
harness_selftest 290-a-preflight-that-built-nothing premise: the fixture holds objects in both source directories 15;16;17;18;19 never -
harness_selftest 290-a-preflight-that-built-nothing premise: the probe run skipped every major 15;16;17;18;19 never -
harness_selftest 290-a-preflight-that-built-nothing premise: the real detector is back, and sees a real object 15;16;17;18;19 2026-09-24 the detector blinded, so it never reports an object
harness_selftest 290-a-preflight-that-built-nothing premise: the tree cleaner is exposed to be judged 15;16;17;18;19 never -
harness_selftest 290-a-preflight-that-built-nothing the cleaner says dirty when the detector reports objects left behind 15;16;17;18;19 2026-09-24 the cleaner never verifies and always answers clean
harness_selftest 290-a-preflight-that-built-nothing the script asks the cleaner rather than merely defining it 15;16;17;18;19 2026-09-24 the call site reverted, so the cleaner is defined but never asked
harness_selftest 300-a-test-script-must-be-runnable and every executable script declares one 15;16;17;18;19 never -
harness_selftest 300-a-test-script-must-be-runnable and every script a document names exists 15;16;17;18;19 never -
harness_selftest 300-a-test-script-must-be-runnable and every script a document names is executable 15;16;17;18;19 never -
Expand Down
64 changes: 63 additions & 1 deletion test/check_ledger_budget.txt
Original file line number Diff line number Diff line change
Expand Up @@ -835,4 +835,66 @@ suites_not_covered 249
#
# The three rows arriving `never` are premises: two `type -t` exposure checks
# and one asserting that a rebuild with no pg_config fails.
checks_never_observed_red 1532
#
# 1532 -> 1534 for #1219, composed onto the main that carries #1213 AND #1248.
# RE-DERIVED BY COUNTING:
#
# awk -F'\t' '$5=="never"' test/check_ledger.tsv | wc -l -> 1534
# 1690 rows total, 156 not `never`, 1534 + 156 == 1690
#
# 17 new rows: 13 arrive dated, 4 arrive `never`
# 2 pre-existing rows flip never -> dated (two of the #809 arms, via K)
# 1532 - 2 + 4 == 1534
#
# THIS BRANCH HAS NOW CARRIED THREE NUMBERS -- 1508, 1532 and 1534 -- and only the
# last describes a tree that exists. They are not corrections of each other:
# each was counted correctly on the tree it was written against. 1508 was
# against a main with neither #1213 nor #1248; 1532 against one with #1213
# alone; 1534 against one with both. The delta this branch contributes is
# constant at -2 + 4; the base moved twice.
#
# AND 1532 APPEARED TWICE FOR UNRELATED REASONS. It is what #1248 composed with
# #1213 came to, and it is what THIS branch came to when rebased onto #1213
# alone -- different rows, different deltas, same total. Had #1248 not landed
# first, this file would now read 1532 and be right; it reads 1534 and is right.
# Two trees agreeing on a number is not evidence either is correct. Recount on
# the tree you have.
#
# THE DECOMPOSITION ABOVE WAS WRONG IN AN EARLIER VERSION of this file, and the
# way it was wrong is worth keeping. It said "9 arrive dated, 8 arrive never"
# and "1506 - 2 + 8 == 1508". The split is 13/4 and 1506 - 2 + 8 is 1512: TWO
# COMPENSATING ERRORS REACHING THE RIGHT ANSWER. Nothing failed, because the
# shipped number was right -- but a reader checking the working got 1512 and
# could not tell whether to distrust the count or the comment. The cause: the
# diff shows 19 `+` lines, which is 17 new rows PLUS the 2 pre-existing rows
# that changed, and counting `+` lines as new rows produced the 9/8. Caught by
# @jdatcmd.
#
# ELEVEN MUTATIONS. Three are the point:
#
# C the cleaner never verifies and always answers `clean`
# -> reddened NOTHING, twice: inline, and again after the detector was
# split out. Its verdict is only load-bearing when the SWEEP FAILS, and
# that cannot be staged as root -- chmod does not stop an unlink, and
# anything `find -type f -delete` removes, the `-type f` re-check cannot
# see either. FIXED BY STUBBING WHAT THE CODE CONSULTS rather than by a
# cleverer fixture: the cleaner asks the detector, so the detector is
# stubbed to `yes` and the cleaner must answer `dirty`. The real one is
# restored and the restore ASSERTED against a freshly created object,
# because an unrestored stub makes every later arm measure a function
# this part wrote.
#
# G the detector's own directory guard removed -> reddens NOTHING, and that
# is correct rather than a gap. Measured with and without it over
# '/nonexistent/nope', '' and '/tmp': identical answers, because `find` on
# a missing path already yields nothing and falls through to `no`. A
# REDUNDANT CONDITION in #1236's taxonomy, not an uncovered one.
#
# A and B redden the SAME four arms, so this set catches "no sweep at all" and
# "sweep only src/" without distinguishing them.
#
# TWO NON-PREMISE ARMS REMAIN `never` HERE, both named: "and a directory that is
# not there holds none" (G, redundant) and "a preflight that built nothing says
# how many it built", which predates this change -- K reddens its two siblings
# but leaves the `built 0 of 3` line intact.
checks_never_observed_red 1534
Loading
Loading