What is the problem the feature request solves?
The native Iceberg write eligibility gate (CometIcebergNativeWrite, documented in iceberg-writes.md) is an allowlist, and each restriction it enforces was added conservatively in #5298 so the writer could land incrementally. None of the restrictions has an issue saying whether it is permanent or what would lift it, so there is no way to tell how much of the fallback surface is intended.
Current restrictions, each of which makes a write fall back to iceberg-java:
Describe the potential solution
For each restriction, record one of three decisions in this issue and in iceberg-writes.md:
- Lift: the native path already reproduces iceberg-java's outcome, or can with a bounded change; file a follow-up and link it.
- Keep, pending upstream: needs an iceberg-rust or parquet-rs feature; link the upstream issue.
- Keep, permanent: intentionally out of scope (for example encryption); say why.
The ordering should follow how often the setting appears in real tables. Object-storage layout (write.object-storage.enabled) and bloom filters are the two most likely to be hit by production tables and should be decided first.
Additional context
Part of the native Iceberg writes epic, #5649. Related: #5298 (introduced the allowlist), #5361 (native writer).
What is the problem the feature request solves?
The native Iceberg write eligibility gate (
CometIcebergNativeWrite, documented iniceberg-writes.md) is an allowlist, and each restriction it enforces was added conservatively in #5298 so the writer could land incrementally. None of the restrictions has an issue saying whether it is permanent or what would lift it, so there is no way to tell how much of the fallback surface is intended.Current restrictions, each of which makes a write fall back to iceberg-java:
MinUnsupportedFormatVersion)encryption.*propertywrite.parquet.bloom-filter-max-bytesand per-column bloom filter enablementwrite.parquet.page-version=v2write.parquet.shred-variants=true(also depends on [Variant] Support whole-value VariantType in native Parquet writes #5433 / the Variant epic [EPIC] Native Variant Support #5438)write.parquet.*key, and anyparquet.*table property (includingparquet.enable.dictionary)parquet.*key in the session Hadoop configuration (other thanparquet.hadoop.vectored.io.enabled)write.parquet.row-group-check-min-record-count/max-record-countwrite.object-storage.enabled=trueandwrite.location-provider.implio-implset, or aFileIOoutside the recognized class hierarchyfile,memory,s3,s3a,gs(ossfalls back becauseoss.*properties are not forwarded; see Native Iceberg writes drop Hadoop GCS configuration for HadoopFileIO + gs:// #5637 forgs)Describe the potential solution
For each restriction, record one of three decisions in this issue and in
iceberg-writes.md:The ordering should follow how often the setting appears in real tables. Object-storage layout (
write.object-storage.enabled) and bloom filters are the two most likely to be hit by production tables and should be decided first.Additional context
Part of the native Iceberg writes epic, #5649. Related: #5298 (introduced the allowlist), #5361 (native writer).