Dependency discussion per CONTRIBUTING.md. Gated on the benchmark-harness issue — this should
not turn into a PR before there is a repeatable measurement.
What exists today
Hand-rolled, and deliberately minimal:
src/align/simd_scan.rs — find_stop, a 16-byte chunked "does this chunk contain a stop"
reduction in SSE2 / NEON intrinsics, with a scalar re-scan to locate the exact position. The
module comment states the design rationale: keep the arch-specific surface to a trivial boolean
reduction, do position extraction in portable scalar code.
src/cpu.rs — runtime feature detection, the build-flavour guard, and prefetch_read
(_mm_prefetch / prfm).
That is a good design and the bar for replacing it is correspondingly high.
Candidates
| Option |
Version |
License |
Note |
wide |
1.6.0 |
Zlib OR Apache-2.0 OR MIT |
Mature, fixed-width vector types, SSE2/AVX2/NEON/WASM. No multiversioning |
pulp |
0.22.3 |
MIT |
Built-in multiversioning, proven by faer; native-width only, so code must handle variable-width chunks |
std::simd |
— |
— |
Nightly-only; the crate pins rust-version = "1.89" stable, so it is out |
The actual questions
- Does a portable crate buy wider vectors? Today's scan is fixed at 16 bytes, i.e. SSE2/NEON
width. AVX2 (32 B) or AVX-512 would halve or quarter the chunk count, but only with runtime
dispatch — which is exactly what wide does not do and pulp does.
- Five platforms, including Windows.
CONTRIBUTING.md calls this out. Any candidate has to
build cleanly everywhere, including the aarch64 macOS path.
- Results must be bit-identical across dispatch paths. A scalar fallback, an SSE2 path and an
AVX2 path must return the same find_stop index. That needs a test exercising all compiled
paths, not just the native one.
- Is
find_stop even hot enough to matter? Answer with a profile before touching it.
Checklist
Dependency discussion per
CONTRIBUTING.md. Gated on the benchmark-harness issue — this shouldnot turn into a PR before there is a repeatable measurement.
What exists today
Hand-rolled, and deliberately minimal:
src/align/simd_scan.rs—find_stop, a 16-byte chunked "does this chunk contain a stop"reduction in SSE2 / NEON intrinsics, with a scalar re-scan to locate the exact position. The
module comment states the design rationale: keep the arch-specific surface to a trivial boolean
reduction, do position extraction in portable scalar code.
src/cpu.rs— runtime feature detection, the build-flavour guard, andprefetch_read(
_mm_prefetch/prfm).That is a good design and the bar for replacing it is correspondingly high.
Candidates
wide1.6.0pulp0.22.3faer; native-width only, so code must handle variable-width chunksstd::simdrust-version = "1.89"stable, so it is outThe actual questions
width. AVX2 (32 B) or AVX-512 would halve or quarter the chunk count, but only with runtime
dispatch — which is exactly what
widedoes not do andpulpdoes.CONTRIBUTING.mdcalls this out. Any candidate has tobuild cleanly everywhere, including the aarch64 macOS path.
AVX2 path must return the same
find_stopindex. That needs a test exercising all compiledpaths, not just the native one.
find_stopeven hot enough to matter? Answer with a profile before touching it.Checklist
find_stop's share of align wall time (blocked on the benchmark harness)is_x86_feature_detected!first,since that costs zero dependencies and answers question 1 directly
pulp(multiversioning is the differentiator)