Related:
Is your feature request related to a problem or challenge?
General Partitioning::Range satisfaction for Distribution::KeyPartitioned is tracked by #23266. Multi-child operators are harder than single-input operators because independently key-partitioned inputs are not always co-partitioned by partition index see #23184 for more contect on co-partitioning.
Some multi-child operators still declare per-child KeyPartitioned requirements directly. Before range satisfaction becomes general, these operators should either use a co-partitioning requirement to guard them to not be satisfied by independent children requirements alone.
Example
SortMergeJoinExec: join_type=Inner, on=[(range_key@0, range_key@0)]
DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(10), (20), (30)], 4)
DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(15), (20), (30)], 4)
Each input may independently satisfy KeyPartitioned(range_key), but the split points are not compatible. A partition-index aware operator needs co-partitioning proof before it can skip repartitioning.
Describe the solution you'd like
Audit multi-child physical operators that declare KeyPartitioned input requirements and convert partition-index aware operators to InputDistributionRequirements::co_partitioned(...).
Operators to audit (please do your own pass as well):
SortMergeJoinExec
SymmetricHashJoinExec
Acceptance criteria
- Partition-index aware multi-child operators use explicit co-partitioning requirements.
- Compatible range/range inputs avoid repartitioning only when cross-child boundaries, partition counts, sort options, and key expressions are compatible.
- Incompatible range/range inputs require repartitioning path.
- Tests cover independently range-partitioned inputs with incompatible cross-child distributions.
Additional context
This should happen before or as part of #23266 so general range satisfaction does not accidentally treat per-child key colocation as cross-child co-partitioning.
Related:
Is your feature request related to a problem or challenge?
General
Partitioning::Rangesatisfaction forDistribution::KeyPartitionedis tracked by #23266. Multi-child operators are harder than single-input operators because independently key-partitioned inputs are not always co-partitioned by partition index see #23184 for more contect on co-partitioning.Some multi-child operators still declare per-child
KeyPartitionedrequirements directly. Before range satisfaction becomes general, these operators should either use a co-partitioning requirement to guard them to not be satisfied by independent children requirements alone.Example
Each input may independently satisfy
KeyPartitioned(range_key), but the split points are not compatible. A partition-index aware operator needs co-partitioning proof before it can skip repartitioning.Describe the solution you'd like
Audit multi-child physical operators that declare
KeyPartitionedinput requirements and convert partition-index aware operators toInputDistributionRequirements::co_partitioned(...).Operators to audit (please do your own pass as well):
SortMergeJoinExecSymmetricHashJoinExecAcceptance criteria
Additional context
This should happen before or as part of #23266 so general range satisfaction does not accidentally treat per-child key colocation as cross-child co-partitioning.