Avoid position delete metadata column collisions - #89
Merged
osipovartem merged 1 commit intoSep 23, 2026
Merged
osipovartem merged 1 commit into
osipovartem merged 1 commit into
Conversation
osipovartem
force-pushed
the
fix/position-delete-column-collisions-df55
branch
from
September 23, 2026 09:12
38d2e9f to
1335780
Compare
Author
|
Rebased onto Post-rebase validation:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PartitionedFilevalue ordering when the internal data path and public manifest path are both presentThis is the df55 counterpart of the upstream contribution in JanKaul#412, adapted to the fork's v3 row-lineage and deletion-vector implementation.
Performance
The added name selection and index remapping happen once during scan planning. V2 joins and v3 deletion-vector filters continue to use native DataFusion physical expressions; no row-wise conversion, interpreted evaluation, sorting, or materialization was added.
Validation
cargo +1.95.0 test -p datafusion_iceberg --test position_delete(v2 position deletes and v3 Puffin deletion vectors)cargo +1.95.0 test -p datafusion_iceberg position_deletes_include_equal_data_sequence_numbers --libcargo +1.95.0 clippy -p datafusion_iceberg --all-targets -- -D warningscargo +1.95.0 fmt --all -- --checkgit diff --checkThe integration test covers a partitioned table with colliding user column names, reordered projections, public manifest metadata, v2 position deletes, and subsequent equality deletes.
Independent review
An independent read-only review initially found a manifest/data-file path ordering defect in the df55 adaptation. The ordering was fixed and the test was strengthened to assert the exact manifest path and deleted rows through the affected provider configuration. Final review verdict: APPROVED, with no blocking or non-blocking findings.