[fix](rbo) Preserve semantics in predicate inference - #67919
Open
feiniaofeiafei wants to merge 2 commits into
Open
[fix](rbo) Preserve semantics in predicate inference#67919feiniaofeiafei wants to merge 2 commits into
feiniaofeiafei wants to merge 2 commits into
Conversation
### What problem does this PR solve?
Problem Summary: Predicate inference records retained greater-than and greater-than-or-equal predicates as equalities. The false equality can make another necessary input predicate appear redundant. A QUALIFY condition a > b AND rn > b AND a > rn can consequently lose a > rn and return extra rows. Record the retained predicate's actual relation in the working graph.
### Release note
Fix extra rows returned when inequality predicate inference removes a necessary filter, including QUALIFY queries over window outputs.
### Check List (For Author)
- Test: Unit Test / Regression test / Manual test
- All 25 UnequalPredicateInferTest tests passed. New semantic checks cover 234 relation/order/qualifier combinations and 64 value assignments per combination; the new test fails on the unmodified baseline.
- infer_predicate_qualify passed after generating expected output with the regression runner and rerunning comparisons.
- DISABLE_BUILD_UI=ON ./build.sh --fe passed, including Checkstyle. Deployed the FE library and verified the SQL wrong-result reproducer.
- Behavior changed: Yes. Necessary inequality predicates are retained.
- Does this need documentation: No.
### What problem does this PR solve?
Problem Summary: Comparison equality does not imply that raw operands are interchangeable inside arbitrary expressions. Replacing a DATE slot with an equal DATETIME slot changes CAST-to-string output, and replacing negative floating-point zero with positive zero changes SIGNBIT. Predicate inference can therefore add filters not implied by the query and discard matching rows.
Restrict substitution inside expressions to exactly matching supported scalar types with value-preserving equality. Keep direct comparisons, IN and their negations on the unwrapped operand, while retaining the existing determinism and cast-analysis checks. Do not change the shared cast extraction or inequality inference paths.
### Release note
Fix missing rows caused by inferred predicates that substitute comparison-equal values inside type-sensitive or representation-sensitive expressions.
### Check List (For Author)
- Test: Unit Test / Regression test / Manual test
- All 65 tests in InferPredicateByReplaceTest and UnequalPredicateInferTest passed. New tests cover type/precision boundaries, floating-point signed zero, safe same-type substitutions, widening casts, and direct comparison/IN/NOT IN propagation.
- Six regression suites passed: infer_predicate_replace_type, infer_predicate_qualify, infer_unequal_predicates, extend_infer_equal_predicate, infer_predicate, infer_datetimev2_cast_precision. Existing expected files are unchanged; new expected files were generated with the regression runner.
- Date/string-length and signed-zero SQL witnesses returned no rows before the fix and the expected rows after deployment.
- DISABLE_BUILD_UI=ON ./build.sh --fe passed, including Checkstyle. Deployed the final FE lib to the test cluster and verified the jar checksum.
- Behavior changed: Yes. Unsafe expression substitutions are rejected while direct comparison propagation is preserved.
- Does this need documentation: No.
feiniaofeiafei
requested review from
924060929,
englefly,
morrySnow and
starocean999
as code owners
September 14, 2026 01:44
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Collaborator
Author
|
run buildall |
morrySnow
changed the title
Contributor
TPC-H: Total hot run time: 16668 ms |
Contributor
TPC-DS: Total hot run time: 82069 ms |
Contributor
ClickBench: Total hot run time: 14.7 s |
Contributor
FE UT Coverage ReportIncrement line coverage |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Problem Summary:
Fix two predicate inference errors that change query results:
chooseInputPredicatesrecords a retainedGT/GTEpredicate asEQ. The false equality can make another necessary predicate appear redundant. For example,a > b AND rn > b AND a > rncan losea > rnand return extra rows in a QUALIFY query. Record the actual relation in the working graph.The fixes are in separate commits. Shared cast extraction is unchanged, and existing regression expectations are unchanged.
Release note
Fix extra or missing rows caused by incorrect predicate inference, including QUALIFY inequality chains and substitutions inside type-sensitive or representation-sensitive expressions.
Check List (For Author)
infer_predicate_qualify,infer_predicate_replace_type,infer_unequal_predicates,extend_infer_equal_predicate,infer_predicate, andinfer_datetimev2_cast_precision. New expected files were generated by the regression runner and verified by rerunning the suites.UnequalPredicateInferTestandInferPredicateByReplaceTestpassed. New inequality checks cover 234 relation/order/qualifier combinations with 64 value assignments per combination. Substitution tests cover type/precision boundaries, signed zero, safe same-type substitutions, widening casts, and direct comparison/IN/NOT IN propagation.DISABLE_BUILD_UI=ON ./build.sh --fepassed, including Checkstyle.Check List (For Reviewer who merge this PR)