Skip to content

perf(proximity): broaden SIMD acceleration and remove reranking copies - #108

Merged
forhappy merged 1 commit into
mainfrom
codex/s3-production-readiness
Aug 18, 2026
Merged

perf(proximity): broaden SIMD acceleration and remove reranking copies#108
forhappy merged 1 commit into
mainfrom
codex/s3-production-readiness

Conversation

@forhappy

Copy link
Copy Markdown
Contributor

Summary

  • Add runtime-dispatched AVX-512, AVX2, and NEON kernels for exact and scalar-quantized scoring.
  • Preserve scalar-order f64 reductions and bit-identical scoring semantics.
  • Score validated encoded vectors directly, removing intermediate reranking scratch copies across native, HNSW, PQ, composite, sync, and async paths.
  • Reduce HNSW node cloning with Arc and use a lookup-only HashMap for native score caching.
  • Add repeat, cache-reset, and SIMD-first benchmark controls.

Performance evidence

Preliminary local release measurements on an Apple M2 Max:

  • Adaptive SQ8 at 1,536 dimensions improved from approximately 3.65–3.83 ms to 3.28–3.52 ms; logical work stayed at 1,005 quantized evaluations.
  • HNSW at 128 dimensions improved from approximately 0.90–0.95 ms to 0.72–0.75 ms in the same harness; node reads and distance evaluations stayed unchanged at 194 and 212.
  • Exact-search result counts and scalar/SIMD score bits remained unchanged.

These are preliminary one-shot measurements; the benchmark now supports repeated runs, cold/warm cache comparison, and randomized kernel order for more robust follow-up measurements.

Validation

  • cargo fmt --all -- --check
  • cargo check --target x86_64-apple-darwin --lib --tests
  • cargo clippy --all-features --lib -- -D warnings
  • Focused all-feature proximity tests: 23 passed
  • cargo bench --all-features --no-run

The full all-feature test invocation remains blocked by pre-existing unresolved imports in tests/async_store.rs (catalog_map_id, control_record_key, control_root_name, ActiveIndexControl, and IndexControl). No persisted vector format changes are introduced.

@forhappy
forhappy marked this pull request as ready for review August 18, 2026 06:53
@forhappy
forhappy merged commit 6bf6e58 into main Aug 18, 2026
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant