Skip to content

fix(substrait): preserve grouping set output order#23468

Draft
bvolpato wants to merge 1 commit into
apache:mainfrom
bvolpato:agent/substrait-grouping-set-output-order
Draft

fix(substrait): preserve grouping set output order#23468
bvolpato wants to merge 1 commit into
apache:mainfrom
bvolpato:agent/substrait-grouping-set-output-order

Conversation

@bvolpato

@bvolpato bvolpato commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

For multiple grouping sets, Substrait orders aggregate output as grouping expressions, measures, then grouping-set ID. DataFusion puts its internal __grouping_id before measures. The consumer applied RelCommon.emit to DataFusion's order, so a mapping such as [0, 1, 2] returned __grouping_id where the plan requested its first measure.

See the Substrait AggregateRel output mapping specification.

What changes are included in this PR?

  • Reorder consumer output to Substrait's direct order before applying emit mappings.
  • Emit a producer mapping back to DataFusion's aggregate order.
  • Deduplicate producer grouping expressions used by grouping references.
  • Test both issue reproducer and two-measure roundtrip.

Are these changes tested?

  • cargo test -p datafusion-substrait --features protoc
  • cargo clippy --all-targets --all-features -- -D warnings

Are there any user-facing changes?

Substrait plans with multiple grouping sets now return emitted columns in specified order. No public API change.

@github-actions github-actions Bot added the substrait Changes to the substrait crate label Jul 10, 2026
@bvolpato bvolpato force-pushed the agent/substrait-grouping-set-output-order branch from ae7ab3f to 9c4d8b7 Compare July 11, 2026 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

substrait Changes to the substrait crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Handling of Aggregates with multiple groupings sets is incorrect

1 participant