Recall calculation is very slow for small batch

[sherylll]

The recall calculation loop is very slow for small batch size, for wiki1M dataset (k=100, n_queries=1, total_queries=10k), recall calculation can take more than 5 min.

I think the issue could be inside this loop: https://github.com/NVIDIA/cuvs/blob/main/cpp/bench/ann/src/common/benchmark.hpp#L372, where chunk_size is 0 when n_queries is very small. At small batch, there are too many worker threads and too little work each.

I modified the code and the timed the recall calculation, it drops to under 0.1s. Recall after the change looks similar.

Before:

for every query:
  create threads
  process one row
  join threads

After:

create threads once
each thread processes many rows
join threads once

Part of the benchmark output:

dataset: wiki_all_1m_cagra_smoke
dim: 768
distance: euclidean
gpu_driver_version: 13.2
gpu_gpuDirectRDMASupported: 1
gpu_hostNativeAtomicSupported: 0
gpu_mem_bus_width: 7680
gpu_mem_freq: 3996000000.000000
gpu_mem_global_size: 287421038592
gpu_mem_shared_size: 233472
gpu_name: NVIDIA B300 SXM6 AC
gpu_pageableMemoryAccess: 1
gpu_pageableMemoryAccessUsesHostPageTables: 0
gpu_runtime_version: 13.3
gpu_sm_count: 148
gpu_sm_freq: 2032000000.000000
host_cores_used: 14
host_cpu_freq_max: 4000000000
host_cpu_freq_min: 800000000
host_pagesize: 4096
host_processors_sysconf: 224
host_processors_used: 28
host_total_ram_size: 2164173221888
host_total_swap_size: 0
max_k: 100
max_n_queries: 10000