[CALCITE-7484] Add a rule to eliminate redundant aggregates functions over GROUP BY keys by xuzifu666 · Pull Request #4903 · apache/calcite

xuzifu666 · 2026-04-21T13:12:59Z

jira: https://issues.apache.org/jira/browse/CALCITE-7484

xiedeyantu · 2026-04-21T14:22:54Z

    sql(sql).withRule(CoreRules.AGGREGATE_REDUCE_FUNCTIONS, CoreRules.PROJECT_MERGE).check();
  }

+  /** Test case for


Please look at this PR #4808, we should create new test files for the rule tests.

Thanks for the suggestion! done.

mihaibudiu · 2026-04-21T20:23:57Z

+ *   <li>{@code MIN}</li>
+ *   <li>{@code AVG}</li>
+ *   <li>{@code ANY_VALUE}</li>
+ *   <li>{@code FIRST_VALUE}</li>


I think FIRST_VALUE and LAST_VALUE only work for window aggregates.

Yes, should not support FIRST_VALUE/LAST_VALUE, I had removed them.

mihaibudiu · 2026-04-21T20:28:04Z

+   */
+  @Test void testAggregateFunctionOfGroupByKeys() {
+    final String sql = "select sal, max(sal) as sal_max, min(sal) as sal_min,\n"
+        + "avg(sal) sal_avg, any_value(sal) as sal_val, first_value(sal) as sal_first,\n"


This query should be illegal, since FIRST_VALUE and LAST_VALUE are only defined for window aggregates. Please file an issue - this program should not be accepted.

Thanks for pointing out this issue; I have already documented the relevant jira https://issues.apache.org/jira/browse/CALCITE-7485.

xiedeyantu · 2026-04-23T01:10:16Z

+    default:
+      return null;
+    }
+    final int groupIndex = aggregate.getGroupSet().asList().indexOf(arg);


-- query 1 SELECT deptno, MAX(deptno + sal) AS "max(deptno + sal)" FROM emp GROUP BY deptno, sal ORDER BY deptno; -- output deptno | max(deptno + sal) --------+------------------- 10 | 1310.00 10 | 5010.00 10 | 2460.00 20 | 2995.00 20 | 3020.00 20 | 820.00 20 | 1120.00 30 | 1530.00 30 | 1280.00 30 | 1630.00 30 | 2880.00 30 | 980.00 (12 rows) -- query 2 SELECT deptno, deptno + sal AS "deptno + sal" FROM emp GROUP BY deptno, sal ORDER BY deptno; -- output deptno | deptno + sal --------+-------------- 10 | 1310.00 10 | 5010.00 10 | 2460.00 20 | 2995.00 20 | 3020.00 20 | 820.00 20 | 1120.00 30 | 1530.00 30 | 1280.00 30 | 1630.00 30 | 2880.00 30 | 980.00 (12 rows)

Could you rewrite complex expressions such as AggFunc(col1 + col2)or something similar?

That makes sense！This scenario likely involves many branching scenarios and requires consideration of many factors. Perhaps we could first enable basic single-field optimization in this version, and I would document an issue to gradually improve the relevant rules.

You can file 2 new issues:

handle aggregates whose argument is a deterministic function involving only constants and group keys

handle aggregates that return something other than the input, e.g., COUNT, STDDEV, STDDEV_POP, where results are constant (1, 0, NULL respectively)

Thank you for your suggestions! I think they provide clear directions for further optimization.
I've filed two jira for future follow-up.(https://issues.apache.org/jira/browse/CALCITE-7492 and https://issues.apache.org/jira/browse/CALCITE-7493)

xiedeyantu · 2026-04-23T01:18:56Z

+ * @see CoreRules#AGGREGATE_FUNCTION_OF_GROUP_BY_KEYS
+ */
+@Value.Enclosing
+public class AggregateFunctionOfGroupByKeysRule


I wonder if the name AggregateReduceFunctionsOnGroupKeysRule might be more appropriate?

Thanks for the suggestion, done.

xuzifu666 · 2026-04-27T06:20:21Z

Hi @mihaibudiu , do you agree with the current changes? I have documented you mentioned window syntax issue separately in jira and will resolve it in coming days.

mihaibudiu · 2026-04-27T18:17:33Z

+    default:
+      return null;
+    }
+    final int groupIndex = aggregate.getGroupSet().asList().indexOf(arg);


You can file 2 new issues:

handle aggregates whose argument is a deterministic function involving only constants and group keys

handle aggregates that return something other than the input, e.g., COUNT, STDDEV, STDDEV_POP, where results are constant (1, 0, NULL respectively)

xuzifu666 · 2026-04-28T03:41:04Z

If there are no further comments, I will merge this PR in 24 hours.

… over GROUP BY keys

sonarqubecloud · 2026-04-28T08:20:20Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
86.7% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

xiedeyantu reviewed Apr 21, 2026

View reviewed changes

mihaibudiu reviewed Apr 21, 2026

View reviewed changes

xuzifu666 changed the title ~~[CALCITE-7484] Add AggregateFunctionOfGroupByKeysRule to eliminate redundant aggregates over GROUP BY keys~~ [CALCITE-7484] Add a rule to eliminate redundant aggregates over GROUP BY keys Apr 22, 2026

xuzifu666 changed the title ~~[CALCITE-7484] Add a rule to eliminate redundant aggregates over GROUP BY keys~~ [CALCITE-7484] Add a rule to eliminate redundant aggregates functions over GROUP BY keys Apr 22, 2026

xiedeyantu reviewed Apr 23, 2026

View reviewed changes

xiedeyantu approved these changes Apr 27, 2026

View reviewed changes

xuzifu666 requested a review from mihaibudiu April 27, 2026 06:20

mihaibudiu approved these changes Apr 27, 2026

View reviewed changes

xuzifu666 force-pushed the agg_function_rule branch from 671518d to f7cd0df Compare April 28, 2026 03:02

xuzifu666 added the LGTM-will-merge-soon Overall PR looks OK. Only minor things left. label Apr 28, 2026

[CALCITE-7484] Add a rule to eliminate redundant aggregates functions…

554c0ee

… over GROUP BY keys

xuzifu666 force-pushed the agg_function_rule branch from f7cd0df to 554c0ee Compare April 28, 2026 07:45

xuzifu666 merged commit 3505dba into apache:main Apr 29, 2026
19 checks passed

Conversation

xuzifu666 commented Apr 21, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xuzifu666 Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xuzifu666 commented Apr 27, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xuzifu666 commented Apr 28, 2026

Uh oh!

sonarqubecloud Bot commented Apr 28, 2026

Quality Gate passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

xuzifu666 Apr 22, 2026 •

edited

Loading