Skip to content

ci(bench): pin fork-JVM allocation ergonomics — B/op counted TLAB-retire waste, not just allocation - #121

Merged
kryptt merged 1 commit into
mainfrom
ci/bench-pinned-ergonomics
Sep 29, 2026
Merged

kryptt merged 1 commit into
mainfrom
ci/bench-pinned-ergonomics

Conversation

@kryptt

@kryptt kryptt commented Sep 29, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #116 (merged in 6809e732): the TLAB-ergonomics pinning landed on that branch ~15 minutes after the merge, so it never reached main. This PR carries it — cherry-picked, applies clean on current main.

Why

gc.alloc.rate.norm counts TLAB-retire waste, whose size adapts to per-JVM heap/TLAB ergonomics — so the A/B profile's B/op "gate" metric was measuring environment state, not just allocation. Measured A/A on identical source (bench sources byte-identical on both sides):

configuration worst A/A B/op delta on the flagged rows
-f 1 unpinned (the A/B profile) +78–81% (PlatedBench.visitorUniverseJson n=4096: 295 KB ↔ 524 KB)
-f 3 unpinned ±11–41% — fork counts are the wrong axis: per-invocation environment state, not per-fork sampling noise
-f 1 pinned (-XX:-ResizeTLAB + fixed -Xms1g -Xmx1g) ±0.0% on n=4096, bit-identical n=512

CI-hardware validation

workflow_dispatch mode=aa on the pinned profile (run 35803316970), filtered to the flagged families — A/A noise floor from the run's own noise-report:

family |ns/op Δ| p50 / p90 / max |B/op Δ| p50 / p90 / max
OrderJsoniterBench 1.2% / 1.2% / 9.1% 0.000% / 0.000% / 0.002%
PlatedBench 6.7% / 6.7% / 7.7% 9.1% / 9.1% / 24.7%

vs the unpinned ±78–81% that produced the false alarms on #116.

What changed

  • bench-pr.yml / bench-sweep.yml: -jvmArgsAppend -XX:-ResizeTLAB -jvmArgsAppend -Xms1g -jvmArgsAppend -Xmx1g on both JMH profiles; profile strings gain -rt1g so the R5 binding invalidates any stale calibration.
  • bench_tools.py noise-report advice + README calibration step 2: pin the ergonomics before the bump--wi advice — fork counts don't fix this (measured).

Notes

  • thresholds.json stays absent: the gate remains advisory until an N≥3 A/A calibration on CI hardware per the runbook; PlatedBench's 24.7% max on one pair says either sample more or raise -wi for that family first.
  • Caveat: the pinned 1g heap must hold every swept bench's live set — if the nightly sweep OOMs, bump both files' -Xms/-Xmx together and recalibrate.

…/op noise floor

The A/B profile's B/op gate assumed deterministic allocation counting,
but gc.alloc.rate.norm counts TLAB-retire waste, whose size adapts to
per-JVM heap/TLAB ergonomics. Measured A/A on identical source:
PlatedBench.visitorUniverseJson n=4096 swung 295KB -> 524KB B/op
(+77.8% — the exact delta CI flagged on this PR) between single-fork
runs, and OrderJsoniterBench.eoReadStreet size=8 wobbled 72/128/152.
Fork counts don't fix it (-f 3 unpinned still swung +/-11-41%: per-
invocation environment state, not per-fork sampling noise).

Pinning does: -jvmArgsAppend -XX:-ResizeTLAB plus fixed -Xms1g/-Xmx1g
took the same A/A to +/-0.0% on n=4096 and bit-identical on n=512.
Applied to both JMH profiles (bench-pr + bench-sweep); the profile
strings gain -rt1g so R5 binding invalidates any future thresholds.json
until recalibration. noise-report advice + README calibration step 2
now name the pinning before the bump-wi advice. Caveat: a pinned 1g
heap must hold every swept bench's live set — bump both files together
if a bench OOMs.
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Cloudflare Pages preview for ci/bench-pinned-ergonomics is live:

https://241c5298.cats-eo-docs.pages.dev

Branch alias: https://ci-bench-pinned-ergonomics.cats-eo-docs.pages.dev

Built from commit 490dc5f50a5f8c8791aeef1894fa342337dcf24f · updated on every push.

@kryptt
kryptt merged commit 96bc4e2 into main Sep 29, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant