[fix](fe) Preserve Paimon CAST predicate semantics on branch-4.1 - #67933
Merged
yiguolei merged 1 commit intoSep 15, 2026
Merged
Conversation
### What problem does this PR solve? Problem Summary: Paimon predicate conversion on branch-4.1 strips CAST from column references before building source filters. For a STRING column containing '05', ' 5', and '5', `CAST(code AS INT) = 5` can become the source predicate `code = '5'`, incorrectly pruning matching rows before Doris evaluates the original predicate. Decimal casts that reduce scale have the same risk. Only convert bare column references. Keep CAST predicates in Doris, preserving their value and null semantics. Ordinary column predicates can still be pushed down independently. This follows the conservative CAST policy already used by the connector on master. ### Release note Fix missing rows in Paimon queries with predicates on casted columns on branch-4.1. ### Check List (For Author) - Test: FE planner regression tests for CAST comparisons, IN/NOT IN, null checks, OR, independent AND conjuncts, and ordinary column pushdown. The 11 predicate-converter tests pass after reproducing 8 failures on the original code. FE Checkstyle passes with 0 violations. All 59 tests across PaimonPredicateConverterTest, PaimonScanNodeTest, and PaimonSourceTest pass. No external Paimon cluster was required for these FE tests. - Behavior changed: Yes. CAST column predicates remain in Doris instead of being rewritten into potentially stricter Paimon filters. This may reduce pushdown for otherwise safe casts. - Does this need documentation: No.
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
Author
|
/review |
Contributor
There was a problem hiding this comment.
Static review opinion: approve; no blocking findings on the reviewed head.
- CAST predicate correctness: the converter now accepts only bare column references, so explicit or implicit casts cannot be stripped into a potentially narrower Paimon filter. Binary comparisons, IN/NOT IN, null predicates, and compound OR fail closed when a casted branch is present.
- Conjunct behavior: top-level conjuncts remain independently eligible, so an ordinary predicate beside a rejected CAST predicate can still push down without changing query semantics.
- Planning and execution: conversion does not mutate the original Doris conjuncts. The derived predicate list is used consistently for FE split planning, split-cache identity, and JNI serialization, while the unchanged conjuncts remain available for Doris-side residual evaluation across normal, cached, snapshot/file-creation, native, and JNI paths.
- Tests and compatibility: the added cases cover the unsafe string/integer and decimal-scale casts, reversed comparison, IN/NOT IN, null checks, OR, independent AND, and ordinary pushdown controls. Rejecting all casted columns may reduce pushdown for safe casts, but it is a conservative compatibility/performance tradeoff rather than a correctness risk.
- User focus: no additional review focus was provided; the full two-file patch and related control flow were reviewed.
This was a static review under the workflow contract; no builds or tests were run independently. The authoritative bundle and live PR matched base 80ff138cfa1532240c119a75a659fd6b2aaa1616 and head 9d333ed633c53b2f971615f1145ae7d6a2296677 at review time, and no existing inline comments required duplicate suppression.
yiguolei
approved these changes
Sep 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Problem Summary: Paimon predicate conversion on branch-4.1 strips CAST from column references before building source filters. For a STRING column containing '05', ' 5', and '5',
CAST(code AS INT) = 5can become the source predicatecode = '5', incorrectly pruning matching rows before Doris evaluates the original predicate. Decimal casts that reduce scale have the same risk.Only convert bare column references. Keep CAST predicates in Doris, preserving their value and null semantics. Ordinary column predicates can still be pushed down independently. This follows the conservative CAST policy already used by the connector on master.
Release note
Fix missing rows in Paimon queries with predicates on casted columns on branch-4.1.
Check List (For Author)