Skip to content

[fix](fe) Preserve Paimon CAST predicate semantics on branch-4.1 - #67933

Merged
yiguolei merged 1 commit into
apache:branch-4.1from
Gabriel39:fix/paimon-cast-predicate-branch-4.1
Sep 15, 2026
Merged

yiguolei merged 1 commit into
apache:branch-4.1from
Gabriel39:fix/paimon-cast-predicate-branch-4.1

Conversation

@Gabriel39

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Problem Summary: Paimon predicate conversion on branch-4.1 strips CAST from column references before building source filters. For a STRING column containing '05', ' 5', and '5', CAST(code AS INT) = 5 can become the source predicate code = '5', incorrectly pruning matching rows before Doris evaluates the original predicate. Decimal casts that reduce scale have the same risk.

Only convert bare column references. Keep CAST predicates in Doris, preserving their value and null semantics. Ordinary column predicates can still be pushed down independently. This follows the conservative CAST policy already used by the connector on master.

Release note

Fix missing rows in Paimon queries with predicates on casted columns on branch-4.1.

Check List (For Author)

  • Test: FE planner regression tests for CAST comparisons, IN/NOT IN, null checks, OR, independent AND conjuncts, and ordinary column pushdown. The 11 predicate-converter tests pass after reproducing 8 failures on the original code. FE Checkstyle passes with 0 violations. All 59 tests across PaimonPredicateConverterTest, PaimonScanNodeTest, and PaimonSourceTest pass. No external Paimon cluster was required for these FE tests.
  • Behavior changed: Yes. CAST column predicates remain in Doris instead of being rewritten into potentially stricter Paimon filters. This may reduce pushdown for otherwise safe casts.
  • Does this need documentation: No.

### What problem does this PR solve?

Problem Summary: Paimon predicate conversion on branch-4.1 strips CAST from column references before building source filters. For a STRING column containing '05', ' 5', and '5', `CAST(code AS INT) = 5` can become the source predicate `code = '5'`, incorrectly pruning matching rows before Doris evaluates the original predicate. Decimal casts that reduce scale have the same risk.

Only convert bare column references. Keep CAST predicates in Doris, preserving their value and null semantics. Ordinary column predicates can still be pushed down independently. This follows the conservative CAST policy already used by the connector on master.

### Release note

Fix missing rows in Paimon queries with predicates on casted columns on branch-4.1.

### Check List (For Author)

- Test: FE planner regression tests for CAST comparisons, IN/NOT IN, null checks, OR, independent AND conjuncts, and ordinary column pushdown. The 11 predicate-converter tests pass after reproducing 8 failures on the original code. FE Checkstyle passes with 0 violations. All 59 tests across PaimonPredicateConverterTest, PaimonScanNodeTest, and PaimonSourceTest pass. No external Paimon cluster was required for these FE tests.
- Behavior changed: Yes. CAST column predicates remain in Doris instead of being rewritten into potentially stricter Paimon filters. This may reduce pushdown for otherwise safe casts.
- Does this need documentation: No.
@Gabriel39
Gabriel39 requested a review from yiguolei as a code owner September 14, 2026 05:15
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Gabriel39

Copy link
Copy Markdown
Contributor Author

run buildall

@Gabriel39

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review opinion: approve; no blocking findings on the reviewed head.

  • CAST predicate correctness: the converter now accepts only bare column references, so explicit or implicit casts cannot be stripped into a potentially narrower Paimon filter. Binary comparisons, IN/NOT IN, null predicates, and compound OR fail closed when a casted branch is present.
  • Conjunct behavior: top-level conjuncts remain independently eligible, so an ordinary predicate beside a rejected CAST predicate can still push down without changing query semantics.
  • Planning and execution: conversion does not mutate the original Doris conjuncts. The derived predicate list is used consistently for FE split planning, split-cache identity, and JNI serialization, while the unchanged conjuncts remain available for Doris-side residual evaluation across normal, cached, snapshot/file-creation, native, and JNI paths.
  • Tests and compatibility: the added cases cover the unsafe string/integer and decimal-scale casts, reversed comparison, IN/NOT IN, null checks, OR, independent AND, and ordinary pushdown controls. Rejecting all casted columns may reduce pushdown for safe casts, but it is a conservative compatibility/performance tradeoff rather than a correctness risk.
  • User focus: no additional review focus was provided; the full two-file patch and related control flow were reviewed.

This was a static review under the workflow contract; no builds or tests were run independently. The authoritative bundle and live PR matched base 80ff138cfa1532240c119a75a659fd6b2aaa1616 and head 9d333ed633c53b2f971615f1145ae7d6a2296677 at review time, and no existing inline comments required duplicate suppression.

@yiguolei
yiguolei merged commit e3bae97 into apache:branch-4.1 Sep 15, 2026
35 of 37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants