ci(bench): pin fork-JVM allocation ergonomics — B/op counted TLAB-retire waste, not just allocation - #121
Merged
Conversation
…/op noise floor The A/B profile's B/op gate assumed deterministic allocation counting, but gc.alloc.rate.norm counts TLAB-retire waste, whose size adapts to per-JVM heap/TLAB ergonomics. Measured A/A on identical source: PlatedBench.visitorUniverseJson n=4096 swung 295KB -> 524KB B/op (+77.8% — the exact delta CI flagged on this PR) between single-fork runs, and OrderJsoniterBench.eoReadStreet size=8 wobbled 72/128/152. Fork counts don't fix it (-f 3 unpinned still swung +/-11-41%: per- invocation environment state, not per-fork sampling noise). Pinning does: -jvmArgsAppend -XX:-ResizeTLAB plus fixed -Xms1g/-Xmx1g took the same A/A to +/-0.0% on n=4096 and bit-identical on n=512. Applied to both JMH profiles (bench-pr + bench-sweep); the profile strings gain -rt1g so R5 binding invalidates any future thresholds.json until recalibration. noise-report advice + README calibration step 2 now name the pinning before the bump-wi advice. Caveat: a pinned 1g heap must hold every swept bench's live set — bump both files together if a bench OOMs.
Contributor
|
🚀 Cloudflare Pages preview for https://241c5298.cats-eo-docs.pages.dev Branch alias: https://ci-bench-pinned-ergonomics.cats-eo-docs.pages.dev Built from commit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #116 (merged in
6809e732): the TLAB-ergonomics pinning landed on that branch ~15 minutes after the merge, so it never reached main. This PR carries it — cherry-picked, applies clean on current main.Why
gc.alloc.rate.normcounts TLAB-retire waste, whose size adapts to per-JVM heap/TLAB ergonomics — so the A/B profile's B/op "gate" metric was measuring environment state, not just allocation. Measured A/A on identical source (bench sources byte-identical on both sides):-f 1unpinned (the A/B profile)PlatedBench.visitorUniverseJson n=4096: 295 KB ↔ 524 KB)-f 3unpinned-f 1pinned (-XX:-ResizeTLAB+ fixed-Xms1g -Xmx1g)CI-hardware validation
workflow_dispatch mode=aaon the pinned profile (run 35803316970), filtered to the flagged families — A/A noise floor from the run's ownnoise-report:OrderJsoniterBenchPlatedBenchvs the unpinned ±78–81% that produced the false alarms on #116.
What changed
bench-pr.yml/bench-sweep.yml:-jvmArgsAppend -XX:-ResizeTLAB -jvmArgsAppend -Xms1g -jvmArgsAppend -Xmx1gon both JMH profiles; profile strings gain-rt1gso the R5 binding invalidates any stale calibration.bench_tools.pynoise-reportadvice + README calibration step 2: pin the ergonomics before the bump--wiadvice — fork counts don't fix this (measured).Notes
thresholds.jsonstays absent: the gate remains advisory until an N≥3 A/A calibration on CI hardware per the runbook; PlatedBench's 24.7% max on one pair says either sample more or raise-wifor that family first.-Xms/-Xmxtogether and recalibrate.