Optimize leading ranges on composite sort keys - #29513
Merged
XuPeng-SH merged 3 commits intoSep 30, 2026
Merged
Conversation
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
4 tasks done
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Which issue(s) this PR fixes:
Fixes #29507
What this PR does / why we need it:
A range on the first component of a composite primary or cluster key did not produce a hidden sort-key predicate for object pruning. The scan could load metadata for objects whose sort-key zone map already ruled them out.
The planner adds a supplemental hidden-key block filter for compatible bounded ranges,
BETWEEN, and one-sided comparisons, including reversed operands. The original SQL row predicate remains authoritative. The change uses the existing composite-key serializer,BlockFilterList, and readutil's sort-key zone-map fast path. First-component paired bounds stay separate, avoiding a premerge cast that could change bound semantics.Block filters now retain metadata-only column references without adding those columns to the scan reader. Row-needed columns keep their compact positions; omitted metadata columns receive distinct positions for the combined runtime/block zonemap path. This also removes unnecessary row reads for existing composite-part block filters. The relation's full schema still supplies physical sequence numbers and zonemaps; no new reader, storage format, or persistent state is introduced.
Partition pruning also consumes block filters. It now matches named predicates to the partition column instead of assuming scan-local and stored partition positions are equal. Ambiguous dotted alias/column names conservatively skip partition pruning. The readutil debug diagnostic likewise resolves by column name rather than indexing the full schema with a scan-local position.
The leading-range optimization remains limited to supported integer, temporal, and same-scale decimal bounds. Float and byte-string leading ranges retain the original scan path because signed zero or byte-prefix ordering could otherwise cause false-negative pruning. Some first-component
BETWEENplans on those types therefore lose their former hidden-key rewrite; exact row filtering remains in place.Validation
pkg/sql/plan,pkg/sql/compile,pkg/partitionprune, andpkg/vm/engine/readutil.FilterObjectstests verify object rejection before metadata loading. Partition tests cover same-column compaction, different-column position collision, and dotted-name ambiguity.go vetandgolangci-lintpassed;git diff --checkpassed.Against base
a36da7f72d, the final diff has production net +210 lines, tests net +389, and no document changes. The planner mapping and partition identity checks account for the production increase; the tests exercise public planning and pruning behavior rather than only helper internals.The historical 329-object local reproduction establishes the missed early-pruning opportunity but did not reproduce the reported production latency. Deterministic tests establish object rejection and removal of hidden-key reader attributes; service-level latency and byte savings have not been measured. CI is pending.