[CALCITE-7657] Apply the absorption law to simplify boolean expressions by xuzifu666 · Pull Request #5110 · apache/calcite

xuzifu666 · 2026-07-19T08:48:26Z

jira: https://issues.apache.org/jira/browse/CALCITE-7657

Other modified tests in the PR have also been simplified.

sonarqubecloud · 2026-07-19T09:10:23Z

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

xuzifu666

I have provided an explanation regarding the reasons why the previous tests were modified.

xuzifu666 · 2026-07-19T09:08:31Z

 !ok
 !if (use_old_decorr) {
-EnumerableCalc(expr#0..14=[{inputs}], expr#15=[0], expr#16=[=($t8, $t15)], expr#17=[IS NULL($t7)], expr#18=[IS NOT NULL($t13)], expr#19=[AND($t14, $t18)], expr#20=[<($t9, $t8)], expr#21=[OR($t17, $t19, $t18, $t20)], expr#22=[IS NOT TRUE($t21)], expr#23=[OR($t16, $t22)], proj#0..7=[{exprs}], $condition=[$t23])
+EnumerableCalc(expr#0..13=[{inputs}], expr#14=[0], expr#15=[=($t8, $t14)], expr#16=[IS NULL($t13)], expr#17=[>=($t9, $t8)], expr#18=[IS NOT NULL($t7)], expr#19=[AND($t16, $t17, $t18)], expr#20=[OR($t15, $t19)], proj#0..7=[{exprs}], $condition=[$t20])


Since IS NOT NULL($t13) is already part of the OR clause and is subsumed by AND($t14, IS NOT NULL($t13)), the term AND($t14, IS NOT NULL($t13)) is absorbed by IS NOT NULL($t13). After converting NOT(<) to >= and reordering the variables, we obtain:

OR(=($t8, 0), AND(IS NULL($t13), >=($t9, $t8), IS NOT NULL($t7)))

xuzifu666 · 2026-07-19T09:09:23Z

            EnumerableValues(tuples=[[{ true }, { true }]])
    EnumerableSort(sort0=[$0], dir0=[ASC])
-      EnumerableCalc(expr#0..1=[{inputs}], expr#2=[true], expr#3=[IS NOT NULL($t1)], proj#0..3=[{exprs}])
+      EnumerableCalc(expr#0..1=[{inputs}], expr#2=[true], proj#0..2=[{exprs}])


After the main Calc node is simplified, the subquery no longer needs to additionally output the IS NOT NULL($t1) column, resulting in this structure.

xuzifu666 · 2026-07-19T09:10:40Z

 !ok
 !if (use_old_decorr) {
-EnumerableCalc(expr#0..13=[{inputs}], expr#14=[0], expr#15=[=($t8, $t14)], expr#16=[IS NULL($t3)], expr#17=[IS NULL($t7)], expr#18=[IS NOT NULL($t12)], expr#19=[AND($t13, $t18)], expr#20=[<($t9, $t8)], expr#21=[OR($t16, $t17, $t19, $t18, $t20)], expr#22=[IS NOT TRUE($t21)], expr#23=[OR($t15, $t22)], proj#0..7=[{exprs}], $condition=[$t23])
+EnumerableCalc(expr#0..12=[{inputs}], expr#13=[0], expr#14=[=($t8, $t13)], expr#15=[IS NULL($t12)], expr#16=[>=($t9, $t8)], expr#17=[IS NOT NULL($t3)], expr#18=[IS NOT NULL($t7)], expr#19=[AND($t15, $t16, $t17, $t18)], expr#20=[OR($t14, $t19)], proj#0..7=[{exprs}], $condition=[$t20])


Redundant terms within OR(...) are absorbed by IS NOT NULL/IS NULL-related terms, shortening the expression.

xuzifu666 · 2026-07-19T09:13:51Z

        EnumerableCalc(expr#0..1=[{inputs}], expr#2=[IS NOT NULL($t0)], expr#3=[IS NOT NULL($t1)], expr#4=[OR($t2, $t3)], $f2=[$t4])
          EnumerableValues(tuples=[[{ 3, null }, { null, null }, { 1, 2 }]])
-    EnumerableCalc(expr#0..1=[{inputs}], expr#2=[true], expr#3=[IS NOT NULL($t0)], expr#4=[IS NOT NULL($t1)], expr#5=[AND($t3, $t4)], expr#6=[OR($t3, $t4)], proj#0..2=[{exprs}], $f20=[$t5], $condition=[$t6])
+    EnumerableCalc(expr#0..1=[{inputs}], expr#2=[true], expr#3=[IS NOT NULL($t0)], expr#4=[IS NOT NULL($t1)], expr#5=[OR($t3, $t4)], proj#0..2=[{exprs}], $condition=[$t5])


The original plan involved calculating two columns simultaneously:

expr#3=[IS NOT NULL($t0)] expr#4=[IS NOT NULL($t1)] expr#5=[AND($t3, $t4)] -- Neither column is empty expr#6=[OR($t3, $t4)] -- At least one non-empty column

After simplification, the upper-level condition only requires "at least one non-null column" to handle the NULL semantics of NOT IN; AND($t3, $t4) has been absorbed:

expr#3=[IS NOT NULL($t0)] expr#4=[IS NOT NULL($t1)] expr#5=[OR($t3, $t4)]

xuzifu666 · 2026-07-19T09:14:28Z

 !ok
 !if (use_old_decorr) {
-EnumerableCalc(expr#0..15=[{inputs}], expr#16=[0], expr#17=[=($t8, $t16)], expr#18=[IS NULL($t7)], expr#19=[IS NOT NULL($t14)], expr#20=[AND($t15, $t19)], expr#21=[<($t9, $t8)], expr#22=[OR($t18, $t20, $t19, $t21)], expr#23=[IS NOT TRUE($t22)], expr#24=[OR($t17, $t23)], proj#0..7=[{exprs}], $condition=[$t24])
+EnumerableCalc(expr#0..14=[{inputs}], expr#15=[0], expr#16=[=($t8, $t15)], expr#17=[IS NULL($t14)], expr#18=[>=($t9, $t8)], expr#19=[IS NOT NULL($t7)], expr#20=[AND($t17, $t18, $t19)], expr#21=[OR($t16, $t20)], proj#0..7=[{exprs}], $condition=[$t21])


Similarly, the expression is compressed because the OR(...) clause resulting from the expansion of NOT IN contains redundant terms that can be absorbed.

xuzifu666 · 2026-07-19T09:14:58Z

              EnumerableValues(tuples=[[{ 7369, 20 }, { 7499, 30 }]])
    EnumerableSort(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC])
-      EnumerableCalc(expr#0..1=[{inputs}], expr#2=[true], proj#0..2=[{exprs}], $f20=[$t2])
+      EnumerableCalc(expr#0..1=[{inputs}], expr#2=[true], proj#0..2=[{exprs}])


After simplification in Calc, the helper column $f20 is no longer needed.

xuzifu666 · 2026-07-19T09:25:45Z

-            + "      JdbcAggregate(group=[{0, 1}], i=[LITERAL_AGG(true)], em=[MAX($2)])\n"
-            + "        JdbcProject(DEPTNO=[$7], ENAME=[CAST($1):VARCHAR(14)],"
-            + " $f2=[AND(IS NOT NULL($7), IS NOT NULL($1))])\n"
+            + "      JdbcAggregate(group=[{0, 1}], i=[LITERAL_AGG(true)])\n"


The intermediate variable $f2 is no longer needed in the condition; OR(IS NULL($7), IS NULL($1)) suffices. Consequently:
$f2=[AND(IS NOT NULL($7), IS NOT NULL($1))] is removed from JdbcProject;
em=[MAX($2)] is removed from JdbcAggregate;
At the same time, JdbcFilter(condition=[OR(IS NOT NULL($7), IS NULL($1))]) is added to the subquery side to filter out rows where both columns are NULL at an earlier stage.

mihaibudiu · 2026-07-20T19:05:34Z

+      final RexNode term = terms.get(i);
+      if (term.getKind() == compositeKind) {
+        final List<RexNode> components = compositeKind == SqlKind.OR
+            ? RelOptUtil.disjunctions(term)


This looks useful, but these two function calls are not cheap.
I hope that this does not matter in practice, but this could be expensive for some very complex boolean expressions.

[CALCITE-7657] Apply the absorption law to simplify boolean expressions

0445bd8

xuzifu666 commented Jul 19, 2026

View reviewed changes

mihaibudiu approved these changes Jul 20, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[CALCITE-7657] Apply the absorption law to simplify boolean expressions#5110

[CALCITE-7657] Apply the absorption law to simplify boolean expressions#5110
xuzifu666 wants to merge 1 commit into
apache:mainfrom
xuzifu666:absorption_law

xuzifu666 commented Jul 19, 2026

Uh oh!

sonarqubecloud Bot commented Jul 19, 2026

Uh oh!

xuzifu666 left a comment

Uh oh!

xuzifu666 Jul 19, 2026

Uh oh!

xuzifu666 Jul 19, 2026

Uh oh!

xuzifu666 Jul 19, 2026

Uh oh!

xuzifu666 Jul 19, 2026

Uh oh!

xuzifu666 Jul 19, 2026

Uh oh!

xuzifu666 Jul 19, 2026

Uh oh!

xuzifu666 Jul 19, 2026

Uh oh!

mihaibudiu Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

xuzifu666 commented Jul 19, 2026

Uh oh!

sonarqubecloud Bot commented Jul 19, 2026

Quality Gate passed

Uh oh!

xuzifu666 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants