Writer: nullable columns go through vortex.masked, so the cascade ignores validity cost and picks differently from Rust #604

Java encodes a nullable primitive column as vortex.masked(values, validity) and runs the cascade over the values alone. Rust keeps validity inside the encoding: vortex-jni 0.89 writes a nullable I32 column (20k rows, 1000 distinct, one null in 7) as fastlanes.bitpacked with a vortex.bool validity child, where Java writes vortex.masked over bit-packed values.

Why it matters beyond the tree shape

Rust's scheme selection charges validity to sample-measured schemes but not to estimated ones: bit-packing is measured on samples that include the validity bitmap, while IntDictScheme's estimate ignores nulls. Java's cascade under vortex.masked never sees the validity, so the two compressors rank schemes differently on nullable data.

Concrete case (the first 8192 rows of a nullable I32, 10 distinct values, nulls every 7th row):

• Rust: dict estimate ~7.99 vs bit-packing sampled at (4 + 1) bits a row = 6.4, so dict wins.
• Java: dict ~7.99 vs bit-packing 32/4 = 8 on the values alone, so bit-packing wins.

This is why #599 left nullable columns on the old cardinality check for the dict layout: running Rust's probe through Java's nullable cascade would diverge more, not less. A nullable integer column Rust bit-packs (taxi payment_type, a 1000-distinct nullable I32) still gets a dict layout in Java only (DictAdmissionInteropIntegrationTest documents the case it leaves out).

Direction

Encode nullable primitives as Rust does: validity carried by the encoding (bit-packed, FoR, ALP... with their validity child), the cascade sampling sizes with the validity included, then drop the cardinality pre-check for nullable columns and let the probe decide, as for non-null ones. Cross-check: same data through vortex-jni and vortex-java, asserting the same encoding tree and dict-layout choice for nullable shapes.

Related: #583 (per-node stats parity).