The parallel covering clamp is no longer provably unreachable after #1155

[OffgridwithJD]

What

In the parallel covering-projection block of PgColumnarSetRelPathlist
(src/columnar_customscan.c):

ioRunProj = ioRun * projScale;
if (ioRunProj > projRun)
    ioRunProj = projRun;
cpuRunProj = projRun - ioRunProj;

#1127 shipped a comment stating this clamp was unreachable, and gave its reason:
projRun = serialRun * projScale, so multiplying an already-clamped ioRun by
the same non-negative factor preserves the order. It also named what would
falsify it — "if projRun is ever computed independently (for example from the
projection's own pages)".

#1155 did exactly that:

projRun = cpuRun * projScale + ioProj;
ioProj  = seq_page_cost * projPages * sel;

So the stated reason is gone. #1155 has corrected the comment in place. This
issue is the open question the comment now records: whether the clamp can bind at
all under the new formula.

What is established

The old premise is false. Measured with a probe at the clamp site, on a build
where the harness confirms source ... matches the binary under test:

test/projection_parallel.sh     block reached 3 times, clamp bound 0 times
    ioRunProj=24.4794   projRun=2122.2110
    ioRunProj=24.4794   projRun=2122.2110
    ioRunProj=0.2473    projRun=163.8619
test/projection_scan_io.sh      block not reached
test/base_scan_io.sh            block not reached

What is NOT established

Reachability is unproven. I built a fixture aimed at binding it — an
incompressible base relation with a covering projection of the single int
column, forced parallel — and it did not bind either:

reached 1   bound 0     ioRunProj=0.2504   projRun=148.2537

Binding requires projScale * (ioRun - cpuRun) > ioProj. sel cancels, leaving

2 * ioBase - serialRun  >  baseSurvival * seq_page_cost * projPages

so base I/O must exceed base CPU by more than roughly projPages. Both attempts
moved that margin the wrong way, 87x then 592x, because
pgcolumnar_scan_io_run_cost prices only the columns actually read and a single
int column's decode CPU dwarfs its page I/O.

So this is not a claim that the branch is live. It is a claim that the reason we
had for believing it was dead no longer applies, and nobody has replaced it.

Why it matters if it does bind

The comment records the consequence: when the clamp binds fully, cpuRunProj is
zero and the partial covering path totals exactly like the serial covering path,
so Gather loses. That would mean the parallel covering projection path added in
#1127 silently stops being chosen — in the shape #1155 targets, a wide base with
a narrow covering projection.

Suggested next step

Either construct a case that binds it — the algebra above says where to look:
maximise base page I/O relative to base decode CPU for the columns read, with
projPages small — or establish it cannot bind and restore a proof to the
comment. Until one of those, the clamp stays, because removing it would allow a
negative cpuRunProj, which is a reason that does not depend on reachability
either way.

Please do not let the comment drift to "reachable"; UNPROVEN is the measured
state.

[OffgridwithJD]

Answered: the clamp IS reachable, and when it binds the parallel path loses

Driven with an elog(WARNING) probe at the clamp site itself, on 4266d37d built for pg18a in the audit container. The probe prints ioRun, projScale, projRun, the pre-clamp product and the verdict, so nothing here is inferred from a total.

The cost GUCs, read from the running server rather than from the docs

  cpu_index_tuple_cost = 0.005    cpu_operator_cost = 0.0025
  cpu_tuple_cost = 0.01           random_page_cost = 4     seq_page_cost = 1

The clamp across seq_page_cost, on projection_parallel's own fixture

20,000 rows, pages=122, a covering projection of all three columns, parallel_workers=2.

  seq_page_cost   ioRun        pre = ioRun*projScale   projRun     bound
       1             62.0000          3.1000            265.5000    no
       4            248.0000         12.4000            274.5000    no
      16            992.0000         49.6000            310.5000    no
      64           3968.0000        198.4000            454.5000    no
     256          15872.0000        793.6000           1030.5000    no
    1024          63488.0000       3174.4000           3334.5000    no
    2048         126976.0000       6348.8000           6406.5000    no
    4096         253952.0000      12697.6000          12550.5000    YES

The model, fitted and then used to predict

pre is linear in seq_page_cost and so is projRun, because the CPU term is not:

  pre     = spc * A                 A = 3.1
  projRun = C + spc * B             B = 3.0    C = 262.5

Those three constants reproduce all six of the first measurements to the decimal. Binding needs spc*(A-B) > C, so spc > 2625. Predicted before running them: 2048 must not bind, 4096 must. Both held, and 2048 missed by 0.9% (6348.8 against 6406.5), which is the sharpest confirmation available.

The consequence is the one the comment predicts

At every seq_page_cost up to 2048 the plan is

  Gather  ->  Parallel Custom Scan (PgColumnarScan) on cvppar

and at 4096, where the clamp binds, it is

  Custom Scan (PgColumnarScan) on cvppar   (cost=0.00..12550.50)

Gather is gone. cpuRunProj collapses to zero, the partial covering path totals exactly like the serial covering path, and the parallel covering projection added in #1127 stops being chosen. That is not a deduction from the totals; both plans are quoted above, which is the only way to tell a clamp that never binds from one that binds and is then beaten.

Why it cannot bind at the default GUCs, which is the other half

A - B = sel * (basePagesRead - projPages) and C = cpuRun * projScale. In units of one row, binding at seq_page_cost = 1 needs

  bytes the projection saves per row  >  8192 * (0.0025 * W/4 + 0.01)
                                      =  5.12 * W + 82

where W is the analyzed width of the columns read. The saving is bounded by what the base stores for those columns, so unless the relation stores more than about 5.12 times the analyzed width per row, the left side cannot reach the right. That is the cpu_operator_cost * W/4 against seq_page_cost * W/8192 ratio, which is 5.12 at these defaults.

Measured, the storage goes the other way by a wide margin:

  fixture A   122 pages / 20,000 rows =  50.0 bytes stored per row   W ~ 392
  fixture B    42 pages / 20,000 rows =  17.2 bytes stored per row   W ~ 392

0.13x and 0.04x, against the 5.12x that binding needs. A second fixture built specifically to shrink the projection -- the blob constant across runs of 500 ik, inserted in md5 order so only the clustered projection sees the runs -- did not help, because the base compresses the same data almost as well: basePagesRead - projPages was 2 pages in both fixtures, so both have the same threshold of 2625.

So, for the comment

• Reachable: YES, and by a supported knob. seq_page_cost is a planner GUC; 4096 is unusual but it is not a hack, and any relation whose stored size exceeds ~5.12x its analyzed width reaches it at a far lower setting.
• Not reachable by data shape at the defaults, and the reason is a constant ratio rather than a property of these fixtures.
• The consequence when it binds is real and is exactly what fix: offer a parallel covering-projection scan #1127 wrote down: the parallel covering path stops being chosen.

I will put up the comment change with these numbers, and an arm that pins the binding case rather than describing it -- the threshold is exact, so the arm can assert both sides of it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MpajdQbkVJ9ey1XyYHcikP