feat(search): support distributed batch vector queries

[ddupg]

Background

Some workloads need to search a large number of query vectors against the same dataset and index.

The existing vector_search() API is designed for single-vector queries. Calling it repeatedly processes each query independently, introducing repeated Ray task scheduling and preventing search workers from efficiently batching queries and reusing warmed index state. In our benchmark, this approach achieved only about 9 QPS.

This PR keeps vector_search() unchanged and introduces a separate streaming API for large batch-query workloads.

Summary

• add open_vector_search() for streaming vector queries
• use long-lived Ray actors to reuse dataset sessions and warmed index caches
• batch queries into configurable micro-batches
• use a bounded in-flight pipeline to overlap query submission and execution without loading all queries into memory
• merge distributed candidates independently for each query using a stable query_index
• preserve the existing vector_search() behavior and Global Pool implementation

Streaming API

with lance_ray.open_vector_search(
    dataset_uri,
    nearest={
        "column": "vector",
        "k": 10,
        "nprobes": 9,
    },
) as session:
    for result in session.map_batches(query_batches):
        ...

query_batches can be an iterator, so callers do not need to materialize all query vectors in memory.

The main execution parameters are configurable:

• number of Ray actors
• query batch size
• worker micro-batch size
• scanner concurrency
• maximum in-flight batches
• actor CPU resources
• index cache size
• index prewarming

Dataset Semantics

The streaming session pins a fixed dataset snapshot for its lifetime.

It supports:

• dataset URI or LanceDataset input
• namespace-based dataset resolution
• branch and version selection
• serialized manifest snapshot reconstruction on workers
• indexed fragments with flat-search fallback for uncovered fragments
• fast_search=True to intentionally skip uncovered fragments
• multivector queries following Lance Core query semantics

Performance

Benchmarked with an IVF_RQ index on a 100M-row, 768-dimensional dataset using k=10, nprobes=9, and accurate mode.

Workload	Result
Recall@10	90.06%
Existing vector_search() serial single-query	9.00 QPS
Streaming batch search	1,559.02 QPS
Streaming improvement	173.3x

The streaming result preserved the reference Recall@10 while substantially improving throughput.