feat(namespace)!: accept multiple columns for the merge insert on key

[wjones127]

Merge insert (upsert) through the namespace API only accepted a single column as the match key, so there was no way to upsert on a composite key — even though Lance core has always supported it (MergeInsertBuilder::try_new takes a list of join columns). The limitation was purely in the request and transport layer.

MergeInsertIntoTableRequest.on is now a list of Lance field paths, and both implementations pass it straight through to the builder. An empty list or a repeated column is rejected with InvalidInput.

This is the implementation half of lance-format/lance-namespace#363, which made the same change in the spec and the generated clients.

Example

POST /v1/table/orders/merge_insert?on=customer_id&on=order_date&when_matched_update_all=true

Breaking changes

MergeInsertIntoTableRequest.on changes from Option<String> to Option<Vec<String>>. Rust callers passing a single column need to wrap it in a list.

The HTTP wire format is unchanged for single-column callers: the query parameter uses style: form, explode: true, so a one-element list still serializes to ?on=id. An older client keeps working against a server built from this change.

Moving from one match column to several also changes how NULL keys behave, because core switches NULL join semantics on the arity of the key: a single-column key treats NULL as equal to NULL, while a composite key uses standard SQL equality, under which a NULL key matches nothing — not even a byte-identical NULL. That is pre-existing core behavior, but composite keys are reachable through the namespace API for the first time here, so it is newly visible. test_merge_insert_composite_key_never_matches_a_null_key_column pins it down.

Java and Python SDK requests reach the Rust implementations as JSON across JNI and pyo3, so a jar or wheel predating this change sends "on": "id" where the model now expects "on": ["id"]. Those four bridge sites deserialize through LenientMergeInsertIntoTableRequest, which promotes a scalar to a one-element list, so mismatched SDK builds keep working. This is inbound only — a Java namespace implementation called from Rust still needs a matching jar.

Not included

The Java and Python lance-namespace pins stay on 0.11, so their generated models still send on as a bare string and rely on the promotion described above. Moving those pins to 0.12 and adding binding-level merge-insert coverage is a follow-up.

The LanceDB Enterprise namespace server needs to accept the repeated on query parameter separately; that change is backward compatible on its own and does not depend on this one.

Part of ENT-2084.

[lance-gatekeeper]

❌ Gate recommendation: request changes.

The composite-key model and transport change is sound, but the dependency sequence must keep the workspace publishable. Land lance-namespace#363, publish its 0.12 crates, and then replace the temporary Git dependency with that released version before this PR merges.

[lance-gatekeeper]

This Git-only dependency makes lance-namespace fail Cargo packaging, so the workspace no longer satisfies the explicit “Check crates are publishable” contract in .github/workflows/rust.yml and cannot follow the configured publish workflow. The upstream schema change is still open and no v0.12 tag exists. Please land and release lance-namespace#363, then use the released version here.

Reproducer

$ CARGO_TARGET_DIR=/home/agent/tmp/gate-8915-package cargo package -p lance-namespace --no-verify --locked
error: failed to verify manifest at `/home/repo/rust/lance-namespace/Cargo.toml`

Caused by:
  all dependencies must have a version requirement specified when packaging.
  dependency `lance-namespace-reqwest-client` does not specify a version

[jackye1995]

looks good!