[SPARK-59183][SDP] SCD2 Ignore-null support; Populate version map per event row at ingest

[AnishMahto]

This is a stacked PR. See incremental diff here: AnishMahto/spark@SPARK-59358-project-missing-columns-during-preprocessing...SPARK-59183-populate-verison-map-at-ingest

See previous PR in stack here: #58709

What changes were proposed in this pull request?

When processing an incoming microbatch, if ignore-null is enabled for the AutoCDC flow, a non-null version map needs to be populated for every row in the microbatch.

As per the version map contract (see Scd2VersionMap's scaladoc), the version map is populated with null-authorship information for every event, respecting the currently active ignore-null selection.

This ticket is not concerned with reconciling the microbatch's version maps with existing rows in the aux/target table.

Why are the changes needed?

Core part of the algorithm for supporting ignore-null API in SCD2.

Does this PR introduce any user-facing change?

No. The user facing interfaces for ignore-null are not exposed yet, this logic cannot be hit yet.

How was this patch tested?

Unit tests.

Was this patch authored or co-authored using generative AI tooling?

Co-authored with GPT-5.6 Sol.

[AnishMahto]

@jose-torres for review, see incremental diff. Diff is not as scary as line count suggests, 700+ is just unit tests.

This PR simply adds logic to construct the version map per incoming row in the microbatch. Reconciliation of existing rows in the target/aux is not touched yet by this PR, that comes next.