From 696de3b80139850e386f977ad25a3b4b6fd4a333 Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Thu, 17 Sep 2026 13:03:26 -0600 Subject: [PATCH 1/3] docs: alpha4 release notes, and the three version claims that drifted Audit of the user-facing documentation before the 1.0-alpha4 tag. Five findings, all fixed here. RELEASE NOTES. alpha2 and alpha3 each have a RELEASE_NOTES_*.md and alpha4 had none. Written from what a user can see rather than from the changelog: the upgrade script, the GUC defaults, and the feat/fix commits touching src/. The [Unreleased] section runs to 4,229 lines and is almost entirely harness work, so counting its entries would have produced the wrong highlights. THE LATEST-RELEASE CLAIM DRIFTED A WHOLE CYCLE. README.md, docs/roadmap.md and docs/installation.md all said the latest published pre-release was v1.0-alpha2 while v1.0-alpha3 was tagged. CHANGELOG.md was the only one right. docs/installation.md was the worst of the three and shows why the existing gate missed them. It never writes the phrase "recorded in `VERSION`", so it sat outside that check entirely, and it also carried a stale heading, a stale tree version, and an upgrade chain that stopped at 1.0-alpha2. A GATE FOR THE SECOND CLAIM. docs_style.sh now checks that every document naming the latest published pre-release names the same one. This is agreement rather than comparison, and the comment says so: no tracked file records the newest tag, and reading `git tag` fails in a tree copied without .git, which this harness runs from. It catches one document drifting from the others, which is what happened. It cannot catch all of them being stale together, and the release procedure now carries that step instead. THE HEADLINE FEATURE WAS FILED UNDER Fixed. Hilbert clustering sat under ### Fixed. Moved to ### Added, verified positional: the sorted multiset of lines is identical before and after, so nothing was lost or reworded. ALPHA5 HAS NO SCOPE LEFT. Both its planned items shipped in alpha4. Verified rather than assumed, each absent from v1.0-alpha3 and present in main: 9f7dcd8 perf: prune scattered IN lists by element (#752) ae623cb feat: add serial join runtime range filter (#752) cbd0c2e feat: add serial join runtime Bloom filter (#752) 60ddc10 feat: turn join runtime filter on by default (#752) Recorded in the plan. docs/roadmap.md still publishes "join acceleration" as alpha5's theme, which is now a promise about delivered work, and re-theming it is the owner's call rather than an edit to make during an audit. A RELEASE CHECKLIST. design/RELEASE_PLAN_1.0.md never mentioned release notes, which is why the step was skipped rather than deferred. It now carries the documentation steps and the tag steps, including capturing the fixture FROM THE TAG (#901). Verification: docs_style.sh 25 checks, PASSED plain_language_check.py ok on docs/*.md, README.md and the new notes test_docs_cover_the_corpus.py 98 checks, 38 passed docs_style.sh is registered but uncovered by the ledger, has no pytest twin and is in neither COMPLETE nor INCOMPLETE, so its two new checks carry no ledger or parity obligation. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK --- CHANGELOG.md | 70 ++++++++--------- README.md | 2 +- RELEASE_NOTES_1.0-alpha4.md | 150 ++++++++++++++++++++++++++++++++++++ design/RELEASE_PLAN_1.0.md | 65 +++++++++++++++- docs/installation.md | 17 ++-- docs/roadmap.md | 2 +- test/docs_style.sh | 37 +++++++++ 7 files changed, 296 insertions(+), 47 deletions(-) create mode 100644 RELEASE_NOTES_1.0-alpha4.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 3d6d121e..8b4e4516 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -208,6 +208,41 @@ true until the next version shipped. ### Added +- Hilbert clustering: `pgcolumnar.cluster_hilbert` and + `pgcolumnar.recluster_hilbert` (#889). + + **Two verbs rather than a parameter on the existing two.** PostgreSQL refuses + to extend `cluster(regclass, VARIADIC name[])` in either direction: a defaulted + parameter cannot precede a `VARIADIC` one, and an array-plus-kind overload + makes the documented `cluster('t','a','b')` call ambiguous. Both were measured + on 18.4. The new verbs match their siblings element for element in argument + types, variadic element type, return type and volatility, so a caller switches + between them by name alone. + + **What the curve buys.** Z-order jumps a long way in key space at a bit + boundary; a Hilbert curve does not. Keys that are close in the data therefore + stay closer in storage, the min/max zone maps over the clustered columns are + tighter, and a range filter reads fewer chunk groups. The key is the same width + and sorts through the same `bytea` comparator, so nothing downstream of the + sort knows which curve produced it. + + **The curve is sticky.** `sorted_kind` is the table's declared intent, not a + property of each call: + + - plain `recluster` on a Hilbert table over the same key is a no-op returning + 0, not a silent conversion back to Z-order; + - `recluster_hilbert` on a Z-ordered table over the same columns rewrites it; + - `vacuum_sorted` leaves a Hilbert table alone rather than sorting it + lexicographically and relabelling it; + - the maintenance daemon dispatches on the recorded kind, so a Hilbert table + is re-clustered with Hilbert instead of being converted on a timer; + - naming the other verb, or reclustering on a different key, is how a table + changes curve. + + Held by `test/hilbert_cluster.sh` (181 arms) over the SQL surface, the recorded + kind, both self-gates and the daemon, and by `test/hilbert_curve.sh` (184 arms) + over the encoder itself. + - Six more guards counted their callers instead of pinning their property (#1078's class). Two were blind to the defect they name. @@ -2275,41 +2310,6 @@ true until the next version shipped. Per-element evaluation is capped at 128 non-NULL entries. Larger lists retain the bounded two-key hull because exact vector refinement otherwise costs elements times rows. -- Hilbert clustering: `pgcolumnar.cluster_hilbert` and - `pgcolumnar.recluster_hilbert` (#889). - - **Two verbs rather than a parameter on the existing two.** PostgreSQL refuses - to extend `cluster(regclass, VARIADIC name[])` in either direction: a defaulted - parameter cannot precede a `VARIADIC` one, and an array-plus-kind overload - makes the documented `cluster('t','a','b')` call ambiguous. Both were measured - on 18.4. The new verbs match their siblings element for element in argument - types, variadic element type, return type and volatility, so a caller switches - between them by name alone. - - **What the curve buys.** Z-order jumps a long way in key space at a bit - boundary; a Hilbert curve does not. Keys that are close in the data therefore - stay closer in storage, the min/max zone maps over the clustered columns are - tighter, and a range filter reads fewer chunk groups. The key is the same width - and sorts through the same `bytea` comparator, so nothing downstream of the - sort knows which curve produced it. - - **The curve is sticky.** `sorted_kind` is the table's declared intent, not a - property of each call: - - - plain `recluster` on a Hilbert table over the same key is a no-op returning - 0, not a silent conversion back to Z-order; - - `recluster_hilbert` on a Z-ordered table over the same columns rewrites it; - - `vacuum_sorted` leaves a Hilbert table alone rather than sorting it - lexicographically and relabelling it; - - the maintenance daemon dispatches on the recorded kind, so a Hilbert table - is re-clustered with Hilbert instead of being converted on a timer; - - naming the other verb, or reclustering on a different key, is how a table - changes curve. - - Held by `test/hilbert_cluster.sh` (181 arms) over the SQL surface, the recorded - kind, both self-gates and the daemon, and by `test/hilbert_curve.sh` (184 arms) - over the encoder itself. - - `test/projection_rewrite.sh`, 84 checks. Nothing in the tree asserted that a projection answers after a rewrite, which is why this was silent. diff --git a/README.md b/README.md index aae7fe53..ffcb3f8d 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ pgColumnar builds from one source tree on PostgreSQL 15 through 18, with 19 validated against 19beta2, and is licensed under the [MIT License](LICENSE). It is [pre-release](docs/limitations.md#release-status); the version marker is `1.0-alpha4`, recorded in `VERSION`. That version is in development and not -tagged; the latest published pre-release is `v1.0-alpha2`. A table `USING pgcolumnar` is stored in the +tagged; the latest published pre-release is `v1.0-alpha3`. A table `USING pgcolumnar` is stored in the native on-disk format, PGCN v1. ## Documentation diff --git a/RELEASE_NOTES_1.0-alpha4.md b/RELEASE_NOTES_1.0-alpha4.md new file mode 100644 index 00000000..fbbed170 --- /dev/null +++ b/RELEASE_NOTES_1.0-alpha4.md @@ -0,0 +1,150 @@ +# pgColumnar 1.0-alpha4 release notes + +Release date: 2026-09-17 +Previous release: 1.0-alpha3 (2026-09-02) + +pgColumnar is a columnar table access method for PostgreSQL. This is the fourth +alpha. Its theme is layout and skipping. A table can now be laid out on the +Hilbert curve, which keeps neighbouring keys closer together than Z-order does. A +star-schema join now skips fact-table groups and rejects non-matching rows, and it +does so without being asked. The on-disk native format, PGCN v1, is unchanged. +Existing tables are read and written as before. + +This release requires one upgrade command. See "Upgrading" at the end. The upgrade +is the smallest of any release so far: it adds two functions and changes nothing +else. + +## Highlights + +- **Hilbert clustering**. `pgcolumnar.cluster_hilbert` and + `pgcolumnar.recluster_hilbert` lay a table out on the Hilbert curve. The curve + has no jumps at a bit boundary, so a range filter reads fewer chunk groups. + Measured on 200,000 rows over two columns, Hilbert read 1.24x to 2.04x fewer + groups than Z-order. +- **Star-schema joins skip and reject by default**. A serial inner Hash Join now + uses the build-side keys to skip fact-table chunk groups and to reject + non-matching rows. `pgcolumnar.enable_join_runtime_filter` is on. +- **An index-driven read of a wide table does less I/O**. Adjacent column reads in + one row group are coalesced into a single read. +- **A parallel index build now uses its workers**. Every participant claims + distinct row groups. Before this release one backend read the whole table while + the launched workers sat idle. +- **The planner stops preferring a fetching index scan that does far more work**. + A correlated range over tens of thousands of rows now takes the columnar scan. + +## Hilbert clustering + +`pgcolumnar.cluster_hilbert(table, VARIADIC columns)` rewrites a table in Hilbert +order. It holds `AccessExclusiveLock`, as `CLUSTER` and `VACUUM FULL` do. +`pgcolumnar.recluster_hilbert(table, VARIADIC columns)` does the same work online +under `ShareUpdateExclusiveLock`, so reads and writes continue. + +The curve is sticky. Plain `pgcolumnar.recluster` maintains a Hilbert table rather +than converting it back. Naming the Hilbert verb is how a Z-ordered table is +switched to the curve. + +Choose the curve by the predicate. Z-order is fine for point lookups. Hilbert wins +on range filters over several columns, and the gap narrows as the query box grows. +Measure your own corpus when the two look close. See +[docs/best-practices.md](docs/best-practices.md) for the guidance and +`pgcolumnar.sort_status` for how much of a table is currently in order. + +There are two verbs rather than a parameter on the existing two because PostgreSQL +cannot extend `cluster(regclass, VARIADIC name[])` in either direction. A defaulted +parameter cannot precede a `VARIADIC` one. + +## Join acceleration + +A serial inner Hash Join over a columnar fact table now builds a filter from the +join keys it has already hashed. The filter does two things. A key range skips +whole fact-table chunk groups. A Bloom filter rejects rows that cannot match. + +Three measured cases decided the default: + +| fact table | result | +| --- | --- | +| clustered on the join key | 19 of 20 chunk groups removed, 1 read | +| scattered | 0 groups removed, Bloom rejects over 15,000 of 19,800 non-matches | +| build side too large | the Bloom disables itself | + +The third case is why this is on for everyone. A filter that helps nothing turns +itself off. Group skip still needs the fact table clustered on the join key. Set +`pgcolumnar.enable_join_runtime_filter` to `off` to compare. + +An ungrouped vectorized aggregate also keeps running over a unique-key inner Hash +Join. A unique dimension acts as a filter of the fact table, so the fold does not +have to stop. Duplicate-key dimensions and LEFT joins stay on the core plan. + +## Reads and the planner + +- **Coalesced column reads**. An index-driven fetch of several columns in one row + group issues one read for adjacent chunks instead of one per column. +- **Parallel index build**. The table access method's parallel scan claims row + groups per participant from the shared counter, the way the custom scan already + did. A parallel `CREATE INDEX` now spreads across its workers. +- **Parallel scan cost**. The planner no longer divides a parallel scan's I/O by + the worker count. PostgreSQL divides CPU across workers and leaves the disk work + whole, and the columnar cost now matches. +- **Clustered index fetch cost**. The fetch penalty now charges a per-row term, + capped at half a chunk group. Without it a correlated range of 50,000 rows stayed + on a fetching index scan while doing far more work than a scan. + +## Correctness fixes + +- **A truncated column chunk is refused rather than read**. A chunk whose recorded + length exceeded 4 GB was cast to 32 bits on the index-fetch path. A fetch could + therefore read the wrong bytes and report them as data. Both cast sites now + raise `XX001`. +- **A coalesced fetch cannot read past its buffer**. The validity bitmap copy is + now bounded by the chunk length before it runs. +- **Projections survive DDL**. A rewrite re-records its projections. `ALTER TABLE + ... RENAME COLUMN` carries the new name into the projection. `ALTER TABLE ... + DROP COLUMN` is refused when a projection depends on the column. An in-place + `TRUNCATE` clears each projection's storage as well as the base. +- **The block codec frees its buffer**. Both paths abandoned it. +- **Object storage refuses URL userinfo** on `s3://` and `gs://`, as `http(s)://` + has since #706. + +## Known issues + +- **Setting a codec can make a table larger, on high-entropy text** (#1074). The + writer keeps FSST only when it beats the alternative by + `fsst_min_gain_percent`, and it measures both sides after the block codec has + run. Storing the FSST codes uncompressed is never compared. Measured on 200,000 + rows of random hex text, `zstd` wrote 1.777% more than `compression = none`. + `lz4` was unaffected. If a table stores long high-entropy text and size matters, + measure both settings. +- **The block codec compresses a region it then discards** (#1075). On + incompressible data this costs about 25% more write CPU. It does not affect what + is stored or read. + +## Upgrading + +Install this build, then run the following in every database that has the +extension: + +```sql +ALTER EXTENSION pgcolumnar UPDATE; +``` + +This is required. The upgrade creates `pgcolumnar.cluster_hilbert` and +`pgcolumnar.recluster_hilbert`. It changes nothing else. No table data is +converted, no existing function is replaced, no catalog column is added, and no +SQL you write changes. + +See [docs/installation.md](docs/installation.md) for the commands, including how +to list the databases that need the update. + +## Scope and limitations + +- This is an alpha. Interfaces may change before 1.0. +- Hilbert clustering orders whole row groups on rewrite. A table that is written + to after the rewrite drifts out of order until the next one. +- The join runtime filter applies to a serial inner Hash Join on a direct columnar + scan. It does not wrap LEFT, SEMI, ANTI, CROSS, parallel, or projection scans. +- On PGXN this release is `1.0.0-alpha.4`, while `CREATE EXTENSION` reports + `1.0-alpha4`. PGXN requires a semantic version, which needs three integer + components. The extension's own version has two. The two names refer to the same + release. + +The complete, itemized list of changes is in `CHANGELOG.md`. diff --git a/design/RELEASE_PLAN_1.0.md b/design/RELEASE_PLAN_1.0.md index 4020693a..639a654d 100644 --- a/design/RELEASE_PLAN_1.0.md +++ b/design/RELEASE_PLAN_1.0.md @@ -164,15 +164,33 @@ Nothing further should be added. It is 11 days into a 14-day cycle. ### 1.0-alpha5, target 2026-09-29. Theme: join acceleration +**BOTH ITEMS SHIPPED EARLY, IN ALPHA4. This alpha has no scope as written, and +the owner has to give it some or fold it into alpha6.** Recorded 2026-09-17 while +auditing the documentation for the alpha4 tag. Verified rather than assumed: each +commit below is absent from `v1.0-alpha3` and present in `main`. + + 9f7dcd8 perf: prune scattered IN lists by element (#752) + ae623cb feat: add serial join runtime range filter (#752) + cbd0c2e feat: add serial join runtime Bloom filter (#752) + 60ddc10 feat: turn join runtime filter on by default (#752) + +The two items as they were planned: + - **Per-element evaluation of a set predicate** (#752). Already specified, with a measured ceiling. On a clustered fact table it reaches 9 to 12 chunk groups of 27, against 25 today. On an unclustered one the ceiling is provably zero. The design, the cost bound, the buffer constraint and the negative control are all - recorded on the issue. + recorded on the issue. **Shipped in alpha4 as `9f7dcd8`.** - **Runtime filters from a join's build side**, with clustering on the join key as a stated precondition rather than an assumption. The ceiling is zero without it, which is measured, so the documentation half of this item is as important as the - code. + code. **Shipped in alpha4, and on by default.** The precondition is documented in + `docs/how-to.md` and `docs/features.md`. + +This is a scheduling fact rather than a problem. The series compressed on +2026-08-29 to reach beta sooner, and work moving forward is that decision +working. But `docs/roadmap.md` still publishes "join acceleration" as alpha5's +theme, and that is now a promise about work already delivered. ### 1.0-alpha6, target 2026-10-13. Theme: encoding and interoperability @@ -264,3 +282,46 @@ It does not claim the dates will hold. It claims three things. The cadence is observed rather than invented. Every item named is traceable to an issue or to `design/ROADMAP.md`. And the beta 1 test can be failed, which matters, because a plan whose entry criteria cannot be failed is a wish. + +## Cutting a release + +**This section exists because it did not, and alpha4 reached tag day with no +release notes**. Every previous release has a `RELEASE_NOTES_*.md`, so the step +was known and simply never written down. A step that lives only in someone's +memory is a step that gets skipped under time pressure. + +Documentation, in this order, before the tag: + +1. **Write `RELEASE_NOTES_.md`.** Model it on the previous one. Build the + highlights from what a USER can see, which means the upgrade script, the GUC + defaults, and the `feat`/`fix` commits touching `src/`. Do not build them from + the changelog's headline count: alpha4's `[Unreleased]` ran to 4,229 lines and + almost all of it was test-harness work. +2. **Check the changelog's categories.** Alpha4's headline feature sat under + `### Fixed`. Keep a Changelog wants Added for new surface, and a reader looking + for what is new will not find it under Fixed. +3. **Update the two version claims.** They are different sentences in different + places and only one of them is gated: + + ", recorded in `VERSION`" gated by docs_style.sh + "the latest published pre-release is" agreement-checked only + + The second drifted a whole cycle across README.md, docs/roadmap.md and + docs/installation.md while the first stayed green. `docs_style.sh` now checks + that every document making the claim makes the SAME claim, which catches one + file drifting. It cannot catch all of them being stale together, because no + tracked file records the newest tag. That is this step's job. +4. **Re-theme the next alpha if its scope shipped early.** Alpha5's two items both + landed in alpha4, so the published roadmap promised work already delivered. +5. **Run the gates.** `test/docs_style.sh` and `python3 + test/plain_language_check.py docs/*.md README.md`. Release notes are outside + `docs_style.sh`'s scope by design, so run the language check on the new file by + hand. + +Then the tag itself: + +6. `## [Unreleased]` becomes `## [] - `. +7. Tag, then capture the fixture FROM THE TAG (#901): + `git show v:pgcolumnar--.sql > test/fixtures/pgcolumnar--.sql`. + Capturing it from the working tree at the next cycle-open is how the alpha2 + fixture came to differ from what alpha2 actually shipped. diff --git a/docs/installation.md b/docs/installation.md index 3732adab..c31ec6e4 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -150,14 +150,15 @@ while columnar tables exist, because they depend on the access method. The form that succeeds is `DROP EXTENSION pgcolumnar CASCADE`, and it drops every columnar table with it. -### Upgrading to 1.0-alpha3 - -`1.0-alpha3` is what this source tree installs. It is in development and not -tagged; the latest published pre-release is `v1.0-alpha2`. `ALTER EXTENSION pgcolumnar UPDATE` (step 3 -above) reaches it from either previously published version: `1.0-dev`, which the -`v1.0-alpha` tag installed, or `1.0-alpha`. PostgreSQL applies the shipped upgrade -scripts in sequence, so a `1.0-dev` install is carried `1.0-dev` to `1.0-alpha` to -`1.0-alpha2` by that one command. +### Upgrading to 1.0-alpha4 + +`1.0-alpha4`, recorded in `VERSION`, is what this source tree installs. It is in +development and not tagged; the latest published pre-release is `v1.0-alpha3`. +`ALTER EXTENSION pgcolumnar UPDATE` (step 3 above) reaches it from every +previously published version: `1.0-dev`, which the `v1.0-alpha` tag installed, +`1.0-alpha`, `1.0-alpha2`, and `1.0-alpha3`. PostgreSQL applies the shipped +upgrade scripts in sequence. One command therefore carries a `1.0-dev` install +through `1.0-alpha`, `1.0-alpha2` and `1.0-alpha3` to `1.0-alpha4`. The `1.0-alpha` cycle renamed the extension's C symbols into the `pgcolumnar` namespace, so that two extensions named `columnar` can be loaded without diff --git a/docs/roadmap.md b/docs/roadmap.md index d866f2ef..528d2848 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -13,7 +13,7 @@ Issues are the authority on anything being worked now. pgColumnar is [pre-release](limitations.md#release-status). The version marker is `1.0-alpha4`, recorded in `VERSION`. That version is in development and not tagged; the -latest published pre-release is `v1.0-alpha2`. A table `USING pgcolumnar` is stored in the +latest published pre-release is `v1.0-alpha3`. A table `USING pgcolumnar` is stored in the native on-disk format, PGCN v1. ## Releases to 1.0 diff --git a/test/docs_style.sh b/test/docs_style.sh index ac486d49..692e6712 100755 --- a/test/docs_style.sh +++ b/test/docs_style.sh @@ -236,6 +236,43 @@ done check "every document citing VERSION quotes the version VERSION holds" \ "$(printf '%s' "$_stale" | sed 's/^ //')" "" +# ---- the OTHER version claim, which the check above cannot see -------------- +# +# A second version sentence sits beside the first: "the latest published +# pre-release is `vX`". It drifted for a whole cycle in THREE documents while the +# check above stayed green, because that check only looks at files carrying the +# phrase "recorded in `VERSION`", and this is a different sentence: +# +# README.md v1.0-alpha2 while v1.0-alpha3 was tagged +# docs/roadmap.md v1.0-alpha2 +# docs/installation.md v1.0-alpha2 and it named the tree's version wrong too +# CHANGELOG.md v1.0-alpha3 the only one right +# +# `docs/installation.md` is the one that shows why the scope mattered. It never +# writes the phrase "recorded in `VERSION`", so it was outside the other check +# entirely, and it also carried a stale heading and a stale upgrade chain. +# +# THIS IS AN AGREEMENT CHECK, NOT A COMPARISON, and that is a real limit. No +# tracked file holds "the newest tag" the way VERSION holds the version, and +# reading `git tag` fails in a tree copied without `.git`, which this harness is +# run from. So it catches one document drifting away from the others, which is +# what happened. It CANNOT catch every document being stale together, and the +# release procedure carries that step instead. +_pubdocs="$(grep -rln 'latest published pre-release' \ + "$SRCDIR/CHANGELOG.md" "$SRCDIR/README.md" "$SRCDIR/docs" 2>/dev/null | sort)" +check "premise: at least one document names the latest published pre-release" \ + "$([ -n "$_pubdocs" ] && echo yes || echo no)" "yes" + +# shellcheck disable=SC2086 +_pubvers="$(grep -rhoE 'latest published pre-release is `v[^`]*`' $_pubdocs 2>/dev/null \ + | grep -oE '`v[^`]*`' | tr -d '`' | sort -u)" +check "premise: the claim was parsed, not merely present" \ + "$([ -n "$_pubvers" ] && echo yes || echo no)" "yes" + +check "every document names the same latest published pre-release" \ + "$([ "$(printf '%s\n' "$_pubvers" | grep -c .)" = 1 ] && echo "" \ + || printf '%s' "$_pubvers" | tr '\n' ' ' | sed 's/ $//')" "" + # ---- and META.json, which NOTHING read at all ------------------------------ # # `META.json` is the PGXN distribution metadata. It hardcodes the version TWICE From e1083a93b798e496c22e4e236b562a4b953aecb3 Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Thu, 17 Sep 2026 13:06:37 -0600 Subject: [PATCH 2/3] docs: close the changelog section for 1.0-alpha4 `## [Unreleased]` becomes `## [1.0-alpha4] - 2026-09-17`, which is the step the alpha3 release did in its own release-notes PR (`9628414`, #886) rather than after the tag. Following that precedent so the tag points at a tree whose changelog already names the release. The paragraph above the heading still says the version is "in development and not yet tagged". That is what `v1.0-alpha2` and `v1.0-alpha3` both shipped, so it is left alone here rather than changed during a tag. It is worth settling at cycle-open: a tagged tree that says it is untagged is a sentence a reader can falsify by having downloaded it. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8b4e4516..f8dbe0aa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,7 +14,7 @@ installed, `1.0-alpha`, `1.0-alpha2`, and `1.0-alpha3`), so a single notes in this file describe `default_version` as pinned at an earlier version, each true until the next version shipped. -## [Unreleased] +## [1.0-alpha4] - 2026-09-17 ### Fixed From e8a9bb226e5f7e4ca597d671577ab4c4b8832d8c Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Thu, 17 Sep 2026 13:09:22 -0600 Subject: [PATCH 3/3] docs: re-theme alpha5, because alpha4 absorbed the work it was planned for `docs/roadmap.md` published "join acceleration" as alpha5's theme after alpha4 shipped it, so the public schedule promised work already delivered. The themes shift by one. No date moves and no scope is invented: 1.0-alpha4 layout, skipping, and join acceleration was "skipping and layout" 1.0-alpha5 encoding: adaptive cascade selection was "join acceleration" 1.0-alpha6 Parquet partition inference was both encoding and Parquet 1.0-beta1 feature freeze unchanged Alpha6 held two items and now holds one, because adaptive cascade encoding selection moves up into the slot alpha5 vacated. Alpha6 is still the last alpha. Alpha4's row also gains the two corrections an audit should make: its date is the date the tag is cut, 2026-09-17, rather than the 2026-09-15 target, and alpha3's row reads 2026-09-02 to match its tag rather than its 2026-09-01 target. design/RELEASE_PLAN_1.0.md is updated to match, with the encoding item moved between its two sections rather than restated. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK --- design/RELEASE_PLAN_1.0.md | 26 +++++++++++++++----------- docs/roadmap.md | 14 ++++++++++---- 2 files changed, 25 insertions(+), 15 deletions(-) diff --git a/design/RELEASE_PLAN_1.0.md b/design/RELEASE_PLAN_1.0.md index 639a654d..577c45f5 100644 --- a/design/RELEASE_PLAN_1.0.md +++ b/design/RELEASE_PLAN_1.0.md @@ -162,12 +162,12 @@ Nothing further should be added. It is 11 days into a 14-day cycle. Three defects found by the investigation are tracked separately and do NOT close with it: #1074, #1075, #1076. -### 1.0-alpha5, target 2026-09-29. Theme: join acceleration +### 1.0-alpha5, target 2026-09-29. Theme: encoding (adaptive cascade selection) -**BOTH ITEMS SHIPPED EARLY, IN ALPHA4. This alpha has no scope as written, and -the owner has to give it some or fold it into alpha6.** Recorded 2026-09-17 while -auditing the documentation for the alpha4 tag. Verified rather than assumed: each -commit below is absent from `v1.0-alpha3` and present in `main`. +**BOTH OF THIS ALPHA'S PLANNED ITEMS SHIPPED EARLY, IN ALPHA4**, so the encoding +item has been moved up from alpha6 to fill it. Recorded 2026-09-17 while auditing +the documentation for the alpha4 tag. Verified rather than assumed: each commit +below is absent from `v1.0-alpha3` and present in `main`. 9f7dcd8 perf: prune scattered IN lists by element (#752) ae623cb feat: add serial join runtime range filter (#752) @@ -188,19 +188,23 @@ The two items as they were planned: `docs/how-to.md` and `docs/features.md`. This is a scheduling fact rather than a problem. The series compressed on -2026-08-29 to reach beta sooner, and work moving forward is that decision -working. But `docs/roadmap.md` still publishes "join acceleration" as alpha5's -theme, and that is now a promise about work already delivered. +2026-08-29 to reach beta sooner, and work moving forward is that decision working. -### 1.0-alpha6, target 2026-10-13. Theme: encoding and interoperability +**What this alpha now carries**, moved up from alpha6: - **Adaptive cascade encoding selection.** The primitives exist; the missing piece is a sampling selector that chooses per block. High value at low to medium effort, and it changes what the writer emits. + +No date moves. alpha6 keeps Parquet partition inference and stays the last alpha. + +### 1.0-alpha6, target 2026-10-13. Theme: Parquet partition inference + - **Parquet partition inference**, the one remaining item inside Parquet. -Two items rather than three, deliberately. The Arrow C Data Interface export was -cut from this alpha on 2026-08-29. alpha6 is now the last one, and the series +One item. The Arrow C Data Interface export was cut from this alpha on 2026-08-29, +and adaptive cascade encoding selection moved up to alpha5 on 2026-09-17 when +alpha4 absorbed alpha5's join work. alpha6 is still the last one, and the series should not lose an item it needs to a slip in October. ## What compressing the series costs diff --git a/docs/roadmap.md b/docs/roadmap.md index 528d2848..75ea14f9 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -26,12 +26,18 @@ documentation, and tests. A feature deferred past beta 1 waits for 2.0. | release | target | theme | | --- | --- | --- | -| `1.0-alpha3` | 2026-09-01 | feature complete: retention, load deduplication, sort reporting, more skipping | -| `1.0-alpha4` | 2026-09-15 | skipping and layout | -| `1.0-alpha5` | 2026-09-29 | join acceleration | -| `1.0-alpha6` | 2026-10-13 | encoding and Parquet partition inference | +| `1.0-alpha3` | 2026-09-02 | feature complete: retention, load deduplication, sort reporting, more skipping | +| `1.0-alpha4` | 2026-09-17 | layout, skipping, and join acceleration | +| `1.0-alpha5` | 2026-09-29 | encoding: adaptive cascade selection | +| `1.0-alpha6` | 2026-10-13 | Parquet partition inference | | `1.0-beta1` | 2026-10-27 | feature freeze | +Alpha4 carries join acceleration because that work finished inside its cycle +rather than waiting for the alpha5 it was planned for. The serial join runtime +filter ships on by default, and per-element pruning of a set predicate ships with +it. The two items alpha6 held are therefore split across alpha5 and alpha6, and no +date moves. + Dates are a cadence, not a commitment. They follow the 14 days observed between `v1.0-alpha` and `v1.0-alpha2`, and each is the date a tag is cut.