Skip to content

dev-deps: add a benchmark harness (divan or criterion) — there is no benches/ today #204

Description

@BenjaminDEMAILLE

There is no benches/ directory and no benchmark harness in Cargo.toml. Several open dependency
questions (SIMD crates, packed-seq, external sort, sufr vs caps-sa, #162) all start with
"measure first", and each currently has to invent its own measurement. One harness, once, unblocks
all of them.

Candidates

Crate Version License Note
divan 0.1.21 MIT OR Apache-2.0 Simple API, generic benchmarks, allocation counting
criterion 0.8.2 Apache-2.0 OR MIT Incumbent, statistics and regression reports, heavier

Both are dev-dependencies only, so nothing ships in the binary and the supply-chain argument in
CONTRIBUTING.md is much weaker than for a runtime dep.

What to benchmark

The paths the open questions actually care about:

  1. genomeGenerate suffix-array construction (src/index/sa_build.rs), in-mem and ext-mem arms
  2. Seed extension / find_stop (src/align/simd_scan.rs)
  3. FASTQ decode, plain and gzipped (src/io/fastq.rs)
  4. BAM write and coordinate sort (src/io/bam.rs)
  5. End-to-end align on a fixed small fixture, as a coarse regression signal

Constraints

  • Benchmarks must not gate CI correctness; a slow machine must never fail the build.
  • Numbers are machine-dependent. Useful as A/B within one machine and one session, not as absolute
    thresholds checked into tests.
  • Keep the fixture small enough to run locally, large enough not to be dominated by process startup.

Checklist

  • Pick one harness, not both
  • Add the five benches above
  • Document in CONTRIBUTING.md how to run them and how to report an A/B in a PR
  • Link this issue from the dependency issues that are gated on measurement

Metadata

Metadata

Assignees

No one assigned

    Labels

    dependenciesPull requests/issues that update a dependency file

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions