diff --git a/.gitattributes b/.gitattributes index 2649e4af..5cdc074d 100644 --- a/.gitattributes +++ b/.gitattributes @@ -73,3 +73,21 @@ mkdocs.yml export-ignore .gitignore export-ignore .mailmap export-ignore .gitattributes export-ignore + +# ---- merge behaviour ------------------------------------------------------- +# +# CHANGELOG.md takes a UNION merge. Every PR adds its entry as the first child of +# one heading, so any two of them conflict over an anchor rather than over +# content: nine of 27 merges in one day touched this file, and three merge +# commits that day exist only to resolve it. +# +# Union takes both sides of a divergent hunk with no marker. Identical lines are +# emitted once, so two branches that each open a `## [Unreleased]` section +# produce one heading and both entries -- measured on #1098 and #1106. +# +# ITS HAZARD IS REAL HERE: a release cut EDITS `## [Unreleased]` into +# `## [1.0-alphaN] - date`, and a PR appending beneath that line then lands its +# entry inside the section that just shipped, silently. `test/docs_style.sh` +# carries the check that catches it, by comparing each released section against +# what its own tag shipped. The driver and that check belong together. +CHANGELOG.md merge=union diff --git a/CHANGELOG.md b/CHANGELOG.md index 002daf81..ab231dfd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,76 @@ true until the next version shipped. ### Fixed +- Every PR with a changelog entry conflicted with every other one (#996). + + Entries insert as the first child of a single heading, so two PRs that share no + file but `CHANGELOG.md` still collide. Nine of 27 merges in one day touched it, + and three merge commits that day exist only to resolve it. + + `.gitattributes` now gives `CHANGELOG.md` a union merge driver. Measured on the + real pair, #1098 and #1106, which both open a new `## [Unreleased]`: + + default 3-way rc=1, 2 conflict markers + merge=union rc=0, 0 markers, ONE [Unreleased], both entries intact + + The headings are not doubled because union emits identical lines once: + 7640 + 57 + 52 = 7749 against an actual 7744, the 5 being the shared + `## [Unreleased]` / blank / `### Fixed` / blank prefix. + + UNION'S HAZARD IS REAL FOR THIS FILE, so the driver does not ship alone. Union + keeps both sides of a divergent hunk with no marker, and a release cut EDITS the + line a pending PR appends beneath. Reproduced: a PR merged into a release cut + files its entry INSIDE the section that just shipped, rc=0, no marker. Today that + case conflicts and a human sees it, so union alone would trade a loud daily cost + for a silent one at every release. + + `test/docs_style.sh` therefore gains the check that catches it: each dated + section must hold the entries its own tag shipped, and nothing else. THE KEY IS + THE ENTRY, not a count -- counting would let one post-tag entry be swapped for + another with the arm still green. + + IT FOUND ONE ALREADY ON MAIN. `## [1.0-alpha3]` carries an entry that + `v1.0-alpha3` never shipped: + + - A pytest harness beside the bash suites, with a layer that refuses tests which + + added by `d978e7f` (#432) on 2026-09-09, seven days after the 2026-09-02 tag. + Found by @jdatcmd in review. + + That entry cannot be corrected without making a second section wrong: the work + shipped in the 1.0-alpha4 cycle, and the `v1.0-alpha4` tag does not carry it + either. So a released section CAN diverge from its tag, but only by being + recorded in `test/changelog_post_tag.txt` with a reason a reviewer sees in the + diff. Two further arms guard that file: every row needs a reason, and no row may + be stale. + + Seven mutations, each restored byte-exact: an entry added to closed alpha4, an + entry removed from alpha2, the allowance row deleted, the allowance naming a + different entry, the reason blanked, a stale allowance row, and the driver line + removed. Each reddens its own arm; the clean tree is 42 checks, rc=0. + + THE FIRST VERSION OF THIS CHECK WAS MEASURED AGAINST A STALE TAG and reported the + opposite. This tree's local `v1.0-alpha3` was `d9df031d` against the server's + `cec9e9b5`, and `git fetch` never moves a tag that already exists. That produced + a claim that `v1.0-alpha3` shipped with its section still named `## [Unreleased]`, + a skip for it, and a "false-positive budget of 1 of 4" -- all three false, and + the arm green where the tree was actually in violation. @jdatcmd caught it by + checking their own refs against `git ls-remote` before contradicting the result. + The suite now says how to check a tag before believing a red arm. + + Reaching for a skip there was also the wrong instrument, and two guards said so. + This suite keeps its OWN tally and emits none of the machine RESULT vocabulary, + so `lib.sh`'s `check_skip` is `command not found` inside it -- printing nothing, + counting nothing, failing nothing, while the suite reports PASSED. A local one + then tripped `selftest 400`, which refuses `echo "SKIP` in any file that calls + `check`, because a SKIP is an outcome a count and a record must see. A property + this suite cannot compare is now a `note()`: printed, counted as nothing, + claiming no outcome. + + KNOWN LIMIT, stated because it changes what a green CI check means here: at depth + 1 with no tags every section arm is skipped, so CI cannot run any of this. It + runs locally and in the five-major release gate, which is where a release is cut. + - Four secret-leak claims over the PG server log could pass having read nothing (#1032). diff --git a/test/changelog_post_tag.txt b/test/changelog_post_tag.txt new file mode 100644 index 00000000..e20c2bd3 --- /dev/null +++ b/test/changelog_post_tag.txt @@ -0,0 +1,25 @@ +# POST-TAG ENTRIES IN RELEASED SECTIONS, each recorded with the reason it stays. +# +# `test/docs_style.sh` pins every dated section to the entries its own tag shipped. +# Without this file that pin would forbid a released section from ever changing, +# which is not quite the rule anyone wants: it would mean a mistake made before a +# tag can never be described afterwards. So a divergence is allowed, and allowed +# ONLY by being written down here with a reason a reviewer sees in the diff. +# +# THE KEY IS THE ENTRY'S FIRST LINE, NOT A COUNT. An allowance of "+1" would let +# one post-tag entry be swapped for another with the check still green, which is +# the same shape as an accounting line that balances while saying nothing. +# +# Format, TAB-separated, three fields, all required: +# +# +# +# Two arms in docs_style.sh guard this file itself: every row must carry a reason, +# and no row may be stale -- an allowance for an entry that is no longer extra +# would silently widen what the section arms accept. +# +# Adding a row is not the normal response to a red arm. The normal response is that +# an entry was filed into a closed section by mistake and belongs in [Unreleased]. +# A row here says the mistake is already tagged and cannot be corrected without +# making some other section wrong. +1.0-alpha3 - A pytest harness beside the bash suites, with a layer that refuses tests which d978e7f, docs: record the pytest harness in the changelog (#432), added it on 2026-09-09 -- seven days after the v1.0-alpha3 tag. It STAYS where it is. The work shipped in the 1.0-alpha4 cycle, but the v1.0-alpha4 tag does not carry the entry either, so moving it would make a second section disagree with its own tag and deleting it would lose a true record. A changelog records what happened, and what happened is this. diff --git a/test/docs_style.sh b/test/docs_style.sh index 56dc1919..48f6c754 100755 --- a/test/docs_style.sh +++ b/test/docs_style.sh @@ -68,6 +68,23 @@ check() { fi } +# THIS SUITE KEEPS ITS OWN TALLY, not lib.sh's, and emits none of the machine +# RESULT vocabulary. So it cannot report a `SKIP` OUTCOME: selftest 400 refuses +# `echo "SKIP` in any file that calls `check`, because a skip there is a check +# result a count and a record must see, and this suite has neither to put it in. +# +# A property this suite cannot compare is therefore a NOTE: printed for a reader, +# counted as nothing, claiming no outcome. Where the reason is itself a fact about +# the tree it is asserted with `check` instead -- see the no-baseline arm below -- +# so "this cannot be checked" is itself checked rather than asserted. +# +# Reaching for lib.sh's helper of that name here is `command not found`: it prints +# nothing, counts nothing and fails nothing while the suite reports PASSED. +# Measured, and the reason this is a note rather than a second check helper. +note() { # note TEXT + echo "-- $1" +} + command -v python3 >/dev/null || { echo "FAIL python3 not found"; exit 1; } echo "== pgColumnar test: docs_style.sh ==" @@ -225,7 +242,7 @@ check "premise: the VERSION file has a version to compare against" \ # README.md is in this list because it was NOT, and drifted two versions as a # result: it said `1.0-alpha` while VERSION said `1.0-alpha3`. The check that # would have caught it excluded the only file that was wrong. -_verdocs="$(grep -rln 'recorded in `VERSION`' "$SRCDIR/CHANGELOG.md" "$SRCDIR/README.md" "$SRCDIR/docs" 2>/dev/null | sort)" +_verdocs="$(grep -rln 'recorded in `VERSION`' "$SRCDIR/CHANGELOG.md" "$SRCDIR/README.md" "$SRCDIR/docs" 2>/dev/null | LC_ALL=C sort)" check "premise: at least one document cites the VERSION file" \ "$([ -n "$_verdocs" ] && echo yes || echo no)" "yes" @@ -252,7 +269,7 @@ check "premise: the version badge is present" \ "$([ -f "$_badge" ] && echo yes || echo no)" "yes" _badgehits="$(grep -c -- "$_ver" "$_badge" 2>/dev/null || echo 0)" -_badgeold="$(grep -oE '1\.0-[a-z]+[0-9]*' "$_badge" 2>/dev/null | sort -u | grep -vxF "$_ver" | tr '\n' ' ' | sed 's/ $//')" +_badgeold="$(grep -oE '1\.0-[a-z]+[0-9]*' "$_badge" 2>/dev/null | LC_ALL=C sort -u | grep -vxF "$_ver" | tr '\n' ' ' | sed 's/ $//')" check "the version badge names no version other than VERSION's" \ "$_badgeold" "" check "premise: and it names VERSION's version at all" \ @@ -287,7 +304,7 @@ check "README's badge alt text names the version VERSION holds" \ # what happened. It CANNOT catch every document being stale together, and the # release procedure carries that step instead. _pubdocs="$(grep -rln 'latest published pre-release' \ - "$SRCDIR/CHANGELOG.md" "$SRCDIR/README.md" "$SRCDIR/docs" 2>/dev/null | sort)" + "$SRCDIR/CHANGELOG.md" "$SRCDIR/README.md" "$SRCDIR/docs" 2>/dev/null | LC_ALL=C sort)" check "premise: at least one document names the latest published pre-release" \ "$([ -n "$_pubdocs" ] && echo yes || echo no)" "yes" @@ -303,7 +320,7 @@ check "premise: at least one document names the latest published pre-release" \ # shellcheck disable=SC2086 _pubvers="$( { grep -rhoE 'latest published pre-release is `v[^`]*`' $_pubdocs 2>/dev/null grep -rhoE '`v[^`]*` is the latest published pre-release' $_pubdocs 2>/dev/null - } | grep -oE '`v[^`]*`' | tr -d '`' | sort -u)" + } | grep -oE '`v[^`]*`' | tr -d '`' | LC_ALL=C sort -u)" _pubunparsed="" for _d in $_pubdocs; do @@ -412,6 +429,150 @@ check "every pgcolumnar--*.sql in the tree is in the published distribution" \ "all shipped" +# ---- the changelog's shared anchor, and the guard the fix needs (#996) ------- +# +# Every PR that adds an entry inserts as the first child of one heading, so any +# two conflict for a reason unrelated to either change. Nine of 27 merges in one +# day touched this file, and three merge commits that day exist only to resolve +# it. `.gitattributes` now gives CHANGELOG.md a UNION merge driver, which takes +# both sides with no marker. +# +# MEASURED ON THE REAL PAIR, #1098 and #1106, both of which add a new +# `## [Unreleased]` section: +# +# default 3-way rc=1, 2 conflict markers +# merge=union rc=0, 0 markers, ONE `## [Unreleased]`, both entries intact +# +# The section headers are not doubled because union emits identical lines once: +# 7640 + 57 + 52 = 7749 against an actual 7744, and the 5 are the shared +# `## [Unreleased]` / blank / `### Fixed` / blank prefix. +# +# UNION'S HAZARD IS REAL AND THIS FILE MEETS IT AT EVERY RELEASE. Union keeps +# both sides of a divergent hunk silently, so where one branch EDITS a line that +# another appends beneath, the append survives under the edit. A release cut +# edits exactly that line -- `## [Unreleased]` becomes `## [1.0-alphaN] - date`. +# Reproduced: a PR appending an entry, merged into a release cut, lands that +# entry INSIDE the section that just shipped, with rc=0 and no marker. Today the +# same case conflicts and a human sees it. +# +# So the driver ships with the check below, which is what makes it safe: an entry +# cannot appear in a released section after that release's tag without this +# saying so. +check "CHANGELOG.md has a union merge driver, so two entries do not conflict" \ + "$(grep -c '^CHANGELOG\.md[[:space:]]\+merge=union$' "$SRCDIR/.gitattributes")" "1" + +# Portable awk: no gawk-only three-argument match(). Identical output under mawk, +# gawk and this box's default awk, checked rather than assumed. +_cl_versions() { # stdin: a CHANGELOG -> one DATED section version per line + awk ' + /^## \[[^]]+\] - / { + line = $0; sub(/^## \[/, "", line); sub(/\].*$/, "", line); print line + } + ' +} + +# THE KEY IS THE ENTRY, NOT A COUNT. Counting would let one post-tag entry be +# swapped for another with the arm still green -- the same "an aggregate that +# balances" failure this repository has been finding all week, one level up. +_cl_entries() { # _cl_entries VERSION; stdin: a CHANGELOG -> that section's entry first lines + awk -v want="$1" ' + /^## \[/ { + insec = 0; line = $0 + if (line ~ /^## \[[^]]+\] - /) { + sub(/^## \[/, "", line); sub(/\].*$/, "", line) + if (line == want) insec = 1 + } + next + } + insec && /^- / { print } + ' +} + +_cl_git() { git -C "$SRCDIR" "$@" 2>/dev/null; } +_cl_allowfile="$SRCDIR/test/changelog_post_tag.txt" + +# STALE LOCAL TAGS ARE THE FAILURE MODE HERE, and it is not hypothetical: the +# first version of this check reported `v1.0-alpha3` as shipping no dated section +# at all, because this tree's local tag was 5 commits behind the server and +# `git fetch` NEVER moves a tag that already exists. That produced a wrong +# narrative, a wrong false-positive budget, and a green arm where the tree is +# actually in violation. Caught by jdatcmd, who checked their own refs against +# `git ls-remote` before saying so. +# +# If an arm below fails and the section looks right, check the tag before +# believing it: +# +# git ls-remote --tags origin 'refs/tags/v1.0-alpha*' +# git fetch --tags --force origin +if ! _cl_git rev-parse --git-dir >/dev/null; then + _cl_why="no git repository in the tree under test" +elif [ -z "$(_cl_git tag -l 'v1.0-alpha*')" ]; then + # CI checks out at depth 1 with no tags, so NONE of this runs there and a + # green check on this job says nothing about it. It runs locally and in the + # five-major release gate, which is where a release is cut and therefore + # where an entry can be filed into a closed section. + _cl_why="no release tags in this checkout" +else + _cl_why="" +fi + +_cl_seen=0 +_cl_usedrows="" +while read -r _cl_v; do + [ -n "$_cl_v" ] || continue + _cl_seen=$((_cl_seen + 1)) + _cl_name="the $_cl_v section holds what v$_cl_v shipped, plus only what is recorded" + if [ -n "$_cl_why" ]; then + note "$_cl_name: not compared ($_cl_why)" + continue + fi + _cl_tagged="$(_cl_git show "v$_cl_v:CHANGELOG.md" | _cl_entries "$_cl_v" | LC_ALL=C sort)" + if [ -z "$_cl_tagged" ]; then + check "v$_cl_v carries a dated section of its own to compare against" \ + "no section for $_cl_v at v$_cl_v" "a section for $_cl_v at v$_cl_v" + continue + fi + _cl_nowents="$(_cl_entries "$_cl_v" < "$SRCDIR/CHANGELOG.md" | LC_ALL=C sort)" + # LC_ALL=C on BOTH the sorts and the comm. `comm` compares byte-wise and does + # not check that its inputs agree; fed two collations it returns wrong lines + # rather than an error. Selftest 070 enforces this over the whole file, which is + # why the sorts above that predate this block are pinned too. + _cl_extra="$(LC_ALL=C comm -13 <(printf '%s\n' "$_cl_tagged") <(printf '%s\n' "$_cl_nowents"))" + _cl_gone="$(LC_ALL=C comm -23 <(printf '%s\n' "$_cl_tagged") <(printf '%s\n' "$_cl_nowents"))" + _cl_allowed="$(awk -F'\t' -v v="$_cl_v" '$1==v && $2 != "" {print $2}' "$_cl_allowfile" 2>/dev/null | LC_ALL=C sort)" + [ -n "$_cl_allowed" ] && _cl_usedrows="$_cl_usedrows$_cl_allowed +" + check "$_cl_name" "$_cl_extra" "$_cl_allowed" + check "the $_cl_v section still holds every entry v$_cl_v shipped" "$_cl_gone" "" +done </dev/null || true)" +check "premise: the post-tag allowance file is present and readable" \ + "$([ -f "$_cl_allowfile" ] && echo yes || echo no)" "yes" +_cl_noreason="$(awk -F'\t' '/^[^#]/ && NF > 0 && $3 == "" {print $1 " " $2}' "$_cl_allowfile" 2>/dev/null)" +check "every recorded post-tag entry carries a reason" "$_cl_noreason" "" +if [ -z "$_cl_why" ]; then + _cl_declared="$(awk -F'\t' '/^[^#]/ && $2 != "" {print $2}' "$_cl_allowfile" 2>/dev/null | LC_ALL=C sort)" + _cl_stale="$(LC_ALL=C comm -23 <(printf '%s\n' "$_cl_declared") <(printf '%s\n' "$_cl_usedrows" | LC_ALL=C sort -u))" + check "no recorded post-tag entry is stale, so the allowance cannot widen silently" \ + "$_cl_stale" "" +else + note "no recorded post-tag entry is stale: not compared ($_cl_why)" +fi + + echo "checks run: $checks" if [ "$fail" = 0 ]; then echo "docs_style.sh: PASSED"