feat: support atomic conditional overwrite writes

[FANNG1]

Summary

• add mode="overwrite_where" with a required predicate to daft_lance.write_lance: one Lance commit deletes the rows matching the predicate and adds the DataFrame's rows
• run the per-fragment delete as a Daft job, pruning the fragments to visit through a scalar index when one answers the predicate
• check by default that every input row satisfies the predicate, and refuse the check rather than trust it where Daft and Lance read the filter differently
• normalize the new mode to a physical append so the existing schema, Blob-column and storage-version checks cover it

No Lance-side change: this composes LanceFragment.delete() with LanceOperation.Update.

Closes #61

Semantics

start() pins the dataset version; workers write the input into new fragments exactly as append does; finalize applies the predicate to the pinned snapshot's fragments and commits one LanceOperation.Update(updated_fragments, removed_fragment_ids, new_fragments, fields_modified=[]) against that version. Deletions write deletion files rather than rewriting data, so row addresses and the indexes built on them survive; the new fragments are unindexed and get scanned flat. Nothing matched and nothing written means no commit and no empty version.

Fragment pruning is opportunistic: when explain_plan shows the filter answered by a ScalarIndexQuery, a streamed row-address scan gives the exact fragment set; otherwise the delete visits every fragment, which costs the same single pass the pruning scan would have.

The input check

The delete range comes from predicate alone and the input is appended verbatim, so a row outside the predicate is never replaced by a re-run — it accumulates. validate_predicate=True (the default) rejects such rows per micropartition.

That check evaluates the predicate with Daft while the delete runs with Lance, so it is only worth as much as the two engines' agreement. Two cases are refused up front, before any data is written, both confirmed against lance 8.0.0:

• float32/float16 columns. Daft widens 0.1f32 to 0.10000000149 and calls score > 0.1 true; Lance narrows the literal and calls it false. A row like that passed the check and was then never deleted, so re-running the write piled up duplicates.
• predicates Daft types differently, such as a bare TIMESTAMP literal against a naive timestamp column. Daft parses it into a UTC-aware literal; previously only parsing was checked, so the write failed mid-flight with a raw DaftTypeError.

Both name validate_predicate=False in the error. The refusal is scoped to the columns the predicate actually reads, so a table that merely has a float32 column elsewhere still gets the check.

Concurrency caveat (documented, not detected)

Lance does not treat a concurrent Append as conflicting with this Update. A commit against a stale read version succeeds — verified with max_retries=0 — and rows another writer added during the overwrite survive it even when they match the predicate. The write_lance docstring and the README say so, and test_concurrent_append_survives_the_overwrite pins the behavior so a future Lance change surfaces here rather than silently.

Test plan

• full suite on the native runner: 374 passed, 5 skipped, 2 xfailed, 2 xpassed
• ruff format --check and ruff check clean
• mypy clean for daft_lance/ and the new test file
• Ray runner not exercised — ray is not installed in this environment. The delete job reuses the DatasetOpenContext + daft.cls shape that compaction and column merging already run on Ray, and the fragment frame is repartitioned by fragment id there (on the native runner repartition is a no-op that only warns), but that path is untested here.

26 tests in tests/io/lancedb/test_overwrite_where.py cover: multi-fragment replacement in exactly one version, idempotent re-runs including rows the previous run appended, empty input, a predicate matching nothing, a fully matched fragment being removed, input rows outside the predicate with and without the opt-out, NULL rows counting as not matching, the argument-validation matrix, storage-version and schema parity with append, invalid Lance filters, both dialect refusals, index pruning as a unit and end to end through two overwrites, scalar plus IVF_PQ index correctness after an overwrite, and a namespace-addressed table.

Mutation-checked rather than assumed: stubbing the overwrite out to a no-op fails 13 of the 26, and making pruning report no fragments fails 2.

Review history

The first commit is the feature; the second is the fixes from an independent review of it — the two dialect refusals above, repartitioning the delete job (it had been running as a single task), and two tests that had asserted so little they passed with the overwrite stubbed out.