Skip to content

Preserve Partitioning::Range output partitioning through joins #23450

Description

@gene-bordegaray

Related:

Is your feature request related to a problem or challenge?

#23184 lets partitioned joins preserve compatible range-partitioned inputs, but join output currently degrades to non-range partitioning. Downstream operators then may insert avoidable hash repartitions after a range co-partitioned join.

Example

AggregateExec: mode=SinglePartitioned, gby=[range_key@0]
  HashJoinExec: mode=Partitioned, join_type=Inner, on=[(range_key@0, range_key@0)]
     DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(10), (20), (30)], 4)
     DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(10), (20), (30)], 4)

The join output should preserve Partitioning::Range when the range key survives the join output. The aggregate above the join should not need an additional hash repartition.

Describe the solution you'd like

Teach join property calculation to preserve or rewrite Partitioning::Range when it is correct for the join output.

Acceptance criteria

  • Preserve range output only when the output still contains the range key expressions with correct column remapping.
  • Handle projection, swapped joins, and join semantics conservatively.
  • Fall back to UnknownPartitioning when range properties cannot be proven.
  • Add SLT or optimizer tests for nested joins and join-to-aggregate plans.

Additional context

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions