diff --git a/CHANGELOG.md b/CHANGELOG.md index 93841bcb..c023e97f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,62 @@ true until the next version shipped. ## [Unreleased] +### Changed + +- `docs/limitations.md` now says GIN and BRIN can never be chosen, rather than that + nothing has been seen to choose them (#1143). + + The old wording left GIN as an open question and suggested the fixture might have + been too small. It is not a cost problem. Both are bitmap-only access methods, and + the columnar table access method implements no bitmap-scan callback, so the planner + generates no path either index could serve. + + Measured on 200,000 rows, same data and same two indexes on both storages, with + `enable_seqscan`, `enable_indexscan` and `enable_indexonlyscan` off. The only + difference between the rows is the table access method: + + | storage | GIN plan | BRIN plan | + | --- | --- | --- | + | heap | Bitmap Heap Scan | Bitmap Heap Scan | + | columnar | Seq Scan | Seq Scan | + + The page now also states the maintenance cost, because "never used" understates + it. A columnar insert touches very few buffers, so any index maintenance is a large + multiple of it: on 100,000 rows an `INSERT` takes 202 shared hits with no index and + 37,484 with a BRIN index, against 101,468 and 118,923 on heap. The index is essentially + the same size on both storages -- equal on this fixture, and about 0.5% apart on a + second run on a different fixture. The work is real and buys an index that cannot + be chosen. + Measured by @jdatcmd and reproduced here, whose columnar delta agreed within 0.8%. + + A BRIN index on a columnar table also never summarizes. `brin_summarize_new_values` + returns 0 where heap returns a count, and `brin_summarize_range` on a range that has + work raises "columnar: partial-range index build is not supported". The two readings + look contradictory and are not: a 0 means BRIN found no range and never called into + the access method, so it is silence rather than success. @jdatcmd found the guard in + `index_build_range_scan`; the condition that reaches it is reproduced here by naming + a range that has work. + + **The refusal is consumable, which is worse than the refusal.** The same range + returns 0 on the next call and the index never grows, so a reader who checks twice + is told the maintenance function worked. Sweeping ranges 0 to 6 three times gives + `0 0 E 0 0 0 0`, then all zeros, then all zeros, with the index at 24,576 bytes + throughout. That is also why neither session could reproduce the other's result by + re-running the same call: each of us was at a different point in the same + consumption sequence. + + On PostgreSQL 18 the columnar plan carries `Disabled: true`, which is the planner + reporting that it used a node it had been told not to use because no alternative + path existed. A row count cannot change that. BRIN is settled by the same + measurement, which #1143 records as never having been probed past the build. + + `src/columnar_tableam.c` gains a comment at the access-method routine, because the + callback set is not the same on every major and a half-implementation fails badly: + 15 to 17 declare `scan_bitmap_next_block` and `scan_bitmap_next_tuple`, 18 removed + the former, and `table_scan_bitmap_*` calls through the pointer after guarding only + against logical decoding, so a NULL member is a null function-pointer call rather + than an error. Reported by @jdatcmd. + ### Fixed - The session that deleted rows still scanned `delete_vector` sequentially, once per row diff --git a/docs/features.md b/docs/features.md index 2f422bb9..7b04ba84 100644 --- a/docs/features.md +++ b/docs/features.md @@ -120,7 +120,7 @@ coverage. - **The suite checks only the methods that bullet names.** An exclusion written anywhere else on this page is checked against nothing. So do not write one here. A method that does not work belongs in - [limitations](limitations.md#gin-and-brin-build-and-nothing-has-been-seen-to-use-them), + [limitations](limitations.md#gin-and-brin-build-and-no-plan-can-use-them), with its own evidence. - Every row is assigned a stable row number and synthetic item pointer at insert time, so ordinary index scans fetch rows by item pointer. diff --git a/docs/limitations.md b/docs/limitations.md index 39d83e42..8c021a81 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -684,25 +684,72 @@ empty range contributes no bound in either direction. Both are recorded distinctly from "no summary at all". A table written before this existed keeps today's behaviour, rather than being pruned on a statistic nobody wrote. -### GIN and BRIN build, and nothing has been seen to use them +### GIN and BRIN build, and no plan can use them `CREATE INDEX` accepts `gin` and `brin` on a columnar table and the build -succeeds. That is all that is established. No plan has been observed choosing -either one. - -For GIN the question is open. A `jsonb` containment query on a 20,000-row table -still planned a sequential scan with four scan settings turned off. GIN supports -only bitmap scans, and those settings are cost penalties rather than -prohibitions. - -For BRIN the question is deeper. BRIN summarises ranges of physical blocks, and a -columnar table's block layout is not a heap's. Whether such a summary means -anything here is a design question, not a tuning one. It may be that the build -should be refused instead of accepted. - -[Issue #1143](https://github.com/commandprompt/pgcolumnar/issues/1143) tracks -both. Until it is settled, treat a successful `CREATE INDEX` with either method -as a build, not as a plan. +succeeds. Neither index can ever be chosen. + +Both are bitmap-only access methods. A GIN index has no `amgettuple` at all. So +the only path either one can produce is a bitmap index scan feeding a bitmap heap +scan. The columnar table access method implements no bitmap-scan callback. The +planner therefore generates no such path, and this is not a cost the row count can +change. + +Measured on 200,000 rows, with the same data and the same two indexes on both +storages, and with `enable_seqscan`, `enable_indexscan` and `enable_indexonlyscan` +turned off: + +| storage | GIN plan | BRIN plan | +| --- | --- | --- | +| heap | Bitmap Heap Scan | Bitmap Heap Scan | +| columnar | Seq Scan | Seq Scan | + +The only difference between the two rows is the table access method. Both return +the same rows. + +On PostgreSQL 18 the columnar plan is reported with `Disabled: true`. The planner +used a node it had been told not to use, because the alternative did not exist. + +Queries still answer correctly, through a sequential scan or the custom scan. A +`gin` or `brin` index on a columnar table is simply never read. + +It is still maintained on every insert, and that cost is not small in proportion. +A columnar insert touches very few buffers, so any index maintenance is a large +multiple of it. Measured on 100,000 rows, shared buffer hits on the `INSERT` +itself: + +| table | no index | with BRIN | +| --- | ---: | ---: | +| columnar | 202 | 37,484 | +| heap | 101,468 | 118,923 | + +The index is also essentially the same size on both storages: on this fixture, +24,576 bytes for BRIN and 5,726,208 for GIN on each. Read that as "the storage +does not change what the index costs", not as a byte-for-byte identity. A second +run on a different fixture put the two GIN indexes about 0.5% apart rather than +exactly equal. The work is real and the bytes are real. On a columnar table they +buy an index that cannot be chosen. + +A BRIN index also never summarizes. `brin_summarize_new_values` returns 0 on a +columnar table where it returns a count on heap, because it finds no range to +summarize. Asking it to summarize one range directly reaches a path that refuses: + +``` +SELECT brin_summarize_range('cr_brin', 2); +ERROR: columnar: partial-range index build is not supported +``` + +Ranges with nothing to summarize return 0, so the error appears only for a range +that has work. **It also appears only once.** Sweeping the same ranges three times: + +| pass | ranges 0 to 6 | index size | +| --- | --- | ---: | +| 1 | 0 0 E 0 0 0 0 | 24,576 | +| 2 | 0 0 0 0 0 0 0 | 24,576 | +| 3 | 0 0 0 0 0 0 0 | 24,576 | + +So a second call reports 0 over an index that is still empty. Do not read that 0 as +a repair. The index never grew, and nothing was ever summarized into it. **Use a GiST or an SP-GiST index for a selective overlap or containment query.** Both build on a columnar table and both answer the query. A columnar index scan diff --git a/src/columnar_tableam.c b/src/columnar_tableam.c index 16b81877..7366aeb6 100644 --- a/src/columnar_tableam.c +++ b/src/columnar_tableam.c @@ -2297,6 +2297,23 @@ static const TableAmRoutine pgcolumnar_am_methods = { .relation_estimate_size = pgcolumnar_relation_estimate_size, + /* + * NO BITMAP-SCAN CALLBACK IS SET, AND THE PLANNER IS WHAT MAKES THAT SAFE. + * GIN and BRIN are bitmap-only, so with no callback here no bitmap path is + * generated and neither index can be chosen (#1143). Measured: with + * enable_seqscan off on 200,000 rows the plan is a Seq Scan reported + * "Disabled: true", while the same data on heap gives a Bitmap Heap Scan. + * + * ANYONE IMPLEMENTING THIS MUST SET EVERY CALLBACK THE MAJOR DEFINES, and + * the set is not the same on all of them: 15 to 17 declare + * scan_bitmap_next_block and scan_bitmap_next_tuple, 18 removed the former + * in the read-stream rework and declares only the latter. + * + * A HALF-IMPLEMENTATION CRASHES RATHER THAN ERRORS. table_scan_bitmap_* + * guards only against logical decoding and then calls through the pointer, + * so a NULL member is a null function-pointer call in the executor. + * Reported by @jdatcmd. + */ .scan_sample_next_block = pgcolumnar_scan_sample_next_block, .scan_sample_next_tuple = pgcolumnar_scan_sample_next_tuple, };