columnar: serialize CREATE INDEX on columnar tables for PG19 by ihalatci · Pull Request #8618 · citusdata/citus

ihalatci · 2026-06-13T09:24:50Z

Stacked PR (PR3 of the PG19 stack). Base = pg19-runtime-fixes (#8617) → pg19-ci-test-matrices (#8616) → pg19-ruleutils-port (#8602) → pg19-support. Review/merge in stack order.

PG19 enables parallel CREATE INDEX by default (max_parallel_maintenance_workers defaults to 2). Columnar's TableAM is always serial — rs_parallel is stored but never read, and the build path flushes pending writes, which is disallowed inside a parallel operation. This caused CREATE INDEX on columnar tables to fail on PG19.

Three changes (all in columnar_tableam.c):

parallelscan_estimate/initialize/reinitialize now delegate to the table_block_* helpers, so callers that unconditionally size and initialize a ParallelTableScanDesc (e.g. PG19's parallel btree build) no longer abort. Columnar ignores the descriptor, so the written state is simply unused.
columnar_index_build_range_scan accepts a parallel scan descriptor by discarding it (scan = NULL) and running the serial path.
ColumnarProcessUtility forces max_parallel_maintenance_workers=0 (via NewGUCNestLevel/AtEOXact_GUC around PrevProcessUtilityHook) for CREATE INDEX on a columnar AM relation, so the build runs serially.

Closes #8611
Part of #8597

codecov · 2026-06-13T09:28:14Z

Codecov Report

❌ Patch coverage is 92.59259% with 2 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (pg19-support@5017d6b). Learn more about missing BASE report.

Additional details and impacted files

@@               Coverage Diff               @@
##             pg19-support    #8618   +/-   ##
===============================================
  Coverage                ?   88.94%           
===============================================
  Files                   ?      288           
  Lines                   ?    64391           
  Branches                ?     8096           
===============================================
  Hits                    ?    57275           
  Misses                  ?     4782           
  Partials                ?     2334

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

This PR addresses PostgreSQL 19’s default parallel CREATE INDEX behavior to ensure columnar TableAM index builds run safely in a serial mode and no longer fail due to missing/unsupported parallel-scan plumbing.

Changes:

Implement parallel-scan callback support for columnar by delegating initialization/reinitialization to the block-table helper callbacks (to satisfy PG19 callers that size/init parallel scan state unconditionally).
Ensure columnar index builds ignore any provided parallel scan descriptor and proceed via the serial scan path.
Force max_parallel_maintenance_workers=0 for CREATE INDEX targeting columnar relations by temporarily nesting/restoring the GUC around PrevProcessUtilityHook.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

ihalatci · 2026-06-25T11:07:02Z

 static Size
 columnar_parallelscan_estimate(Relation rel)
 {
-	elog(ERROR, "columnar_parallelscan_estimate not implemented");
+	return sizeof(ParallelBlockTableScanDescData);
 }


Done — columnar_parallelscan_estimate now delegates to table_block_parallelscan_estimate(rel) for symmetry with columnar_parallelscan_initialize (which already delegates to table_block_parallelscan_initialize).

One clarification on the under-allocation concern: it doesn't apply today. table_block_parallelscan_estimate() returns exactly sizeof(ParallelBlockTableScanDescData) — the serialized snapshot is sized/allocated separately by the caller (EstimateSnapshotSpace / table_parallelscan_initialize), not into this descriptor — so the returned value is identical to what was here before. Delegating is still the right call for future-proofing in case the AM-level helper ever grows variable-sized state.

PG19 enables parallel CREATE INDEX by default (max_parallel_maintenance_workers defaults to 2). Columnar's TableAM is always serial: rs_parallel is stored but never read, and the build path flushes pending writes, which is disallowed inside a parallel operation. Three changes: 1. Provide working parallelscan_estimate/initialize/reinitialize that delegate to the table_block_* helpers, so callers that unconditionally size and initialize a ParallelTableScanDesc (e.g. PG19's parallel btree build) do not abort. Columnar ignores the descriptor, so the state is simply unused. 2. In columnar_index_build_range_scan, accept a parallel scan descriptor by discarding it (scan = NULL) and running the serial path. 3. In ColumnarProcessUtility, when the statement is CREATE INDEX on a columnar AM relation, force max_parallel_maintenance_workers=0 via NewGUCNestLevel/AtEOXact_GUC around PrevProcessUtilityHook so the build runs serially without triggering "cannot update tuples during a parallel operation". DESCRIPTION: Serialize CREATE INDEX on columnar tables on PG19 Closes #8611 Part of #8597

colm-mchugh · 2026-06-18T16:16:42Z

+		saveNestLevel = NewGUCNestLevel();
+		(void) set_config_option("max_parallel_maintenance_workers", "0",
+								 PGC_USERSET, PGC_S_SESSION,
+								 GUC_ACTION_SAVE, true, 0, false);


Should the removal of parallelism also be applied for REINDEX statements ? They have a separate node tag, T_ReindexStmt, which is not directly handled by this switch statement afaict, so falls to the default branch. As is, would a REINDEX of a columnar table with pending writes get parallel mode and hit the failure ?

Good catch — confirmed this is a real gap. REINDEX (T_ReindexStmt) fell through to the default branch, so a REINDEX of a columnar table with pending writes did enter parallel mode and failed with cannot update tuples during a parallel operation while flushing the pending writes (which updates the stripe reservation entry in columnar.stripe).

Fixed by adding a T_ReindexStmt case that resolves the target relation and sets indexBuildOnColumnar when it's columnar, so the same max_parallel_maintenance_workers = 0 override now covers REINDEX too:

REINDEX_OBJECT_TABLE → the named relation;

REINDEX_OBJECT_INDEX → its IndexGetRelation() heap.

REINDEX {SCHEMA,DATABASE,SYSTEM} are intentionally not handled: they can't run inside a transaction block, so there are never pending writes to flush. Added a pending-writes REINDEX regression test (reindex_events) that reproduces the original failure on the unfixed build and passes with the fix.

colm-mchugh · 2026-06-18T16:20:40Z

+	if (indexBuildOnColumnar)
+	{
+		saveNestLevel = NewGUCNestLevel();
+		(void) set_config_option("max_parallel_maintenance_workers", "0",


nit: (void) goes against existing calls of set_config_option() (e.g. in PostprocessReassignOwnedStmt(), citus_internal_database_command())

Done — dropped the (void) cast to match the existing set_config_option() call sites (e.g. PostprocessReassignOwnedStmt(), citus_internal_database_command()).

colm-mchugh · 2026-06-18T16:31:14Z

+			AtEOXact_GUC(true, saveNestLevel);
+		}
+	}
+	PG_END_TRY();


Wondering if the current tests in columnar_index.sql are enough to validate the fixes ?
A couple of test suggestions:

set max_parallel_maintenance_workers to e..g 4 before a CREATE INDEX on a columnar table, and then show that it is still 4 after the CREATE INDEX.

Similar GUC setting for a REINDEX operation;

SET LOCAL debug_parallel_query = regress; SET LOCAL max_parallel_workers = 4; REINDEX TABLE parallel_scan_test; REINDEX TABLE CONCURRENTLY parallel_scan_test; SHOW LOCAL max_parallel_workers;

Added both suggestions. (1) Set max_parallel_maintenance_workers = 4, then SHOW it after both CREATE INDEX and REINDEX on a columnar table to confirm it's restored to 4. (2) A pending-writes REINDEX test mirroring the existing events CREATE INDEX test.

Two deliberate adjustments to the snippet:

I assert on max_parallel_maintenance_workers rather than max_parallel_workers. The override only touches the former; max_parallel_workers stays 4 regardless of the fix, so asserting on it wouldn't actually catch a regression.

I target a fresh table with actual pending writes inside a txn block instead of parallel_scan_test — the latter has had VACUUM FULL, so it has no pending writes and wouldn't exercise the flush-during-reindex path that this fix addresses.

…y cast - columnar_parallelscan_estimate now delegates to table_block_parallelscan_estimate(rel), symmetric with columnar_parallelscan_initialize (Copilot review). - Extend the serial-index-build override to REINDEX: T_ReindexStmt previously fell through to the default branch, so REINDEX TABLE/INDEX on a columnar table with pending writes could enter parallel mode and fail with "cannot update tuples during a parallel operation" while flushing the stripe reservation update. Set indexBuildOnColumnar for REINDEX_OBJECT_TABLE/INDEX on columnar relations. SCHEMA/DATABASE/ SYSTEM cannot run in a transaction block, so they carry no pending writes and need no handling (colm-mchugh review). - Drop the (void) cast on set_config_option to match existing call sites (colm-mchugh nit). - Tests: add a REINDEX-with-pending-writes regression mirroring the existing CREATE INDEX "events" test, and assert that a user-set max_parallel_maintenance_workers is restored after both CREATE INDEX and REINDEX on a columnar table. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

colm-mchugh

Lgtm. One nit - is the OidIsValue check in the REINDEX branch strictly necessary ? Wouldn't REINDEX on an invalid relation have errored out at that point?

ihalatci · 2026-06-26T15:05:19Z

Lgtm. One nit - is the OidIsValue check in the REINDEX branch strictly necessary ? Wouldn't REINDEX on an invalid relation have errored out at that point?

will check and address with a follow up later

…8619) Stacked PR (PR3b of the PG19 stack). **Base = `pg19-columnar-index` (#8618)** → `pg19-runtime-fixes` (#8617) → `pg19-ci-test-matrices` (#8616) → `pg19-ruleutils-port` (#8602) → `pg19-support`. Review/merge in stack order. PG19 removed `get_relation_info_hook`, which is where pre-PG19 builds disable parallel query and index-only scans for columnar relations (columnar scans are always serial and cannot return tuples from an index). The Phase-1 build left a `TODO(PG19, #8614)` placeholder; this PR re-implements that suppression inside `ColumnarSetRelPathlistHook`: - forbid future parallel workers (`rel_parallel_workers = 0`, `consider_parallel = false`) and drop any already-generated partial paths; - clear each index's `canreturn` flags and strip any `IndexOnlyScan` paths that were already created (`set_rel_pathlist_hook` runs after path generation). The pre-PG19 `ColumnarGetRelationInfoHook` path is unchanged. Closes #8614 Part of #8597 --------- Co-authored-by: Ibrahim Halatci <ihalatci@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

ihalatci mentioned this pull request Jun 13, 2026

columnar: re-implement parallel/index-only-scan suppression on PG19 #8619

Merged

ihalatci linked an issue Jun 13, 2026 that may be closed by this pull request

PG19: columnar CREATE INDEX fails (parallel-scan callbacks + serialization) #8611

Closed

ihalatci force-pushed the pg19-columnar-index branch from a31f940 to 83042a9 Compare June 13, 2026 16:32

ihalatci force-pushed the pg19-runtime-fixes branch 2 times, most recently from 9d9144a to 3a0a90e Compare June 13, 2026 16:38

ihalatci force-pushed the pg19-columnar-index branch from 83042a9 to 59c5dc3 Compare June 13, 2026 16:38

ihalatci requested a review from Copilot June 13, 2026 18:21

Copilot started reviewing on behalf of ihalatci June 13, 2026 18:22 View session

Copilot AI reviewed Jun 13, 2026

View reviewed changes

ihalatci force-pushed the pg19-runtime-fixes branch from 3a0a90e to 2ff1cfd Compare June 18, 2026 08:24

Base automatically changed from pg19-runtime-fixes to pg19-support June 18, 2026 14:38

ihalatci force-pushed the pg19-columnar-index branch from 59c5dc3 to c70d228 Compare June 18, 2026 14:43

colm-mchugh reviewed Jun 18, 2026

View reviewed changes

colm-mchugh approved these changes Jun 26, 2026

View reviewed changes

ihalatci merged commit 1ef844c into pg19-support Jun 26, 2026
170 of 197 checks passed

ihalatci deleted the pg19-columnar-index branch June 26, 2026 15:05

ihalatci mentioned this pull request Jun 27, 2026

PG19: fix regress-only client-backend crashes #8608

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

columnar: serialize CREATE INDEX on columnar tables for PG19 - #8618

columnar: serialize CREATE INDEX on columnar tables for PG19#8618
ihalatci merged 2 commits into
pg19-supportfrom
pg19-columnar-index

ihalatci commented Jun 13, 2026

Uh oh!

codecov Bot commented Jun 13, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

ihalatci Jun 25, 2026

Uh oh!

colm-mchugh Jun 18, 2026 •

edited

Loading

Uh oh!

ihalatci Jun 25, 2026

Uh oh!

colm-mchugh Jun 18, 2026

Uh oh!

ihalatci Jun 25, 2026

Uh oh!

colm-mchugh Jun 18, 2026

Uh oh!

ihalatci Jun 25, 2026

Uh oh!

colm-mchugh left a comment

Uh oh!

ihalatci commented Jun 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

ihalatci commented Jun 13, 2026

Uh oh!

codecov Bot commented Jun 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

ihalatci Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

colm-mchugh Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ihalatci Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

colm-mchugh Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

ihalatci Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

colm-mchugh Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

ihalatci Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

colm-mchugh left a comment

Choose a reason for hiding this comment

Uh oh!

ihalatci commented Jun 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov Bot commented Jun 13, 2026 •

edited

Loading

colm-mchugh Jun 18, 2026 •

edited

Loading