Phase 7: authoritative pinned-hardware benchmark matrix (Linux)

[alexander-yevsyukov]

Phase 7 of the elastic-hashing plan. Run the authoritative, pinned-hardware time benchmark for the comparative matrix (built in Phase 6 / PR #11) on a Linux box, and fold the numbers into the README.

Split out of Phase 6 as its own job — configuring a pinned box and running a multi-hour matrix is separate work from writing the harness, and shouldn't block the Phase 6 PR from landing. Maven Central publication is the sibling Phase 7 job (runbook: docs/publishing.md), tracked separately. This benchmark is deliberately not automatable on a laptop — Apple Silicon can't be pinned.

Why this is needed

The Phase 6 matrix produces two kinds of numbers:

• Memory footprint — already authoritative. JOL measures object layout deterministically and hardware-independently, so LongLongMap 19.4 B/entry (4.59× vs HashMap), IntIntMap 10.3 (7.11×), etc. reproduce everywhere. Nothing to do here.
• Time (ns/op) — needs pinning. Wall-clock is sensitive to CPU frequency scaling, turbo, thermal throttling, and other processes stealing cores. The distribution-robustness ratio (ours flat, fastutil/Eclipse degrade ~5–6× on adversarial keys) already survives noise, but the precise absolute ns/op for the README table must come from a pinned run to survive external scrutiny.

Apple Silicon can't be pinned (no turbo/governor/P-E control), so this must run on Linux. Method reference: docs/benchmarking.md.

Prerequisites

• A Linux box you can free up for ~2–3 hours (ideally otherwise idle).
• JDK 17 (Amazon Corretto 17 is what the project uses); ./gradlew picks up the toolchain.
• taskset (util-linux) and cpupower (or equivalent) available.
• The repo checked out on the phase-6 branch (git pull so run-matrix.sh has the pinned mode).

Step 1 — Configure the environment (pinning)

Run once per boot, as root/sudo. Substitute your own core numbers (pick 2 physical cores; see note below).

# Fix the clock to the max sustained frequency (no ramping).
sudo cpupower frequency-set -g performance

# Disable turbo/boost so the frequency is stable and repeatable.
# Intel:
echo 1 | sudo tee /sys/devices/system/cpu/intel_pstate/no_turbo
# AMD (instead of the Intel line):
# echo 0 | sudo tee /sys/devices/system/cpu/cpufreq/boost

# (Best, optional) Reserve cores so the scheduler puts nothing else on them:
# add `isolcpus=2,3 nohz_full=2,3` to the kernel cmdline and reboot.

# Verify:
cpupower frequency-info | grep -i 'current policy\|governor'
cat /sys/devices/system/cpu/intel_pstate/no_turbo   # expect 1 (Intel)

Core choice: pin to two cores (e.g. 2,3). The comparison benchmarks are @Threads(1), so one core runs the measured thread and the other absorbs GC/JIT — otherwise those background JVM threads steal time from the thing being timed. Prefer two cores on the same physical die / L3 but not SMT siblings of each other. If you have isolcpus, use the isolated cores here.

Step 2 — Run the matrix

./benchmarks-jvm/run-matrix.sh linux-pinned 2,3

This:

• captures environment.txt (JDK/OS/CPU + governor + turbo state + the pinned cores, so the bundle proves it was pinned),
• runs the JOL footprint report,
• runs the single-threaded comparison matrix (LongLong / IntInt / LoadFactor) as the JMH jar directly under taskset — @Fork 3, warmup 5×1s, measurement 5×1s, AverageTime — writing jmh-results.json,
• drops everything in benchmarks-jvm/results/linux-pinned/.

Takes roughly 2–3 hours. The multi-threaded read-scaling / mixed-load benchmarks are intentionally excluded (they measure scaling and need all cores); if you want those too, run ./gradlew :benchmarks-jvm:jmh separately, unpinned.

Step 3 — Hand the results back to Claude Code

Claude Code plays no part during the run (it would steal cycles); its job is to analyze the bundle afterward.

1. Commit and push the results bundle, or just have the JSON handy:

   git add benchmarks-jvm/results/linux-pinned/ && git commit -m "Add pinned Linux benchmark results" && git push

2. Start (or resume) a Claude Code session in the repo and give it a prompt like:

   The pinned Linux benchmark bundle is in benchmarks-jvm/results/linux-pinned/. Parse jmh-results.json into a ns/op table in the README performance section, cross-check the distribution-robustness ratios (dense vs clustered) against the indicative M4 Max run, confirm the memory numbers still hold, and flag anything that looks off. Keep the honest positioning (competitors are a reference ceiling; baseline named).

   If you're on a machine without the repo, paste the contents of jmh-results.json instead and ask the same.

3. Claude will read the JSON, produce the per-op / per-distribution table, update README.md + docs/benchmarking.md (replacing the "indicative" caveats with the pinned figures + the environment.txt provenance), and call out any surprises.

Scope notes

• Footprint is already authoritative — no need to re-measure it, though run-matrix.sh regenerates it anyway (it's deterministic and cheap).
• Maven Central publication is postponed (separate decision) — not part of this task.
• The seeds are fixed (42) and shared across maps, so a rerun on the same pinned box reproduces the numbers within noise.

Definition of done

• benchmarks-jvm/results/linux-pinned/ committed, with environment.txt showing performance governor + turbo disabled + pinned cores.
• README performance section updated with the pinned ns/op numbers and their provenance, replacing the "indicative" caveat.
• Distribution-robustness ratios and memory numbers confirmed against the pinned run.