Reduce contention on the bdwgc global lock - #608
Conversation
Profiling 'nix search nixpkgs --no-eval-cache' showed that ~65% of all futex waits came from the Boehm GC, the largest share being the global allocation lock (GC_allocate_ml): with the default GC_TINY_FREELISTS of 25, only allocations up to 384 bytes are served from the per-thread freelists, and everything larger takes the global lock. Nixpkgs evaluation does ~755k such allocations per search — e.g. a typical derivation attrset (~46 attrs) is a 752-byte Bindings, of which one is allocated per package — all serialized on the lock across the worker threads. Building bdwgc with GC_TINY_FREELISTS=96 (thread-local up to 1520 bytes) eliminates 93% of the global-lock allocations (755k -> 56k; what remains is essentially the >16 KiB attrset giants), halves the context-switch count at 8-16 workers, and makes the search ~10% faster at 24 workers (~3.7s -> ~3.3s). The per-thread memory cost is at most a partial heap block per size class and object kind. Also add a patch (from the bdwgc "nix-patches" branch) that makes gctest's fixed "unexpected heap growth" limit scale with GC_TINY_FREELISTS, since the larger freelists exceed it by design. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
Included review availability: 3 reviews are currently available. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour. 📝 WalkthroughWalkthroughThe BoehmGC override now enables ChangesBoehmGC tiny freelists
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to This localized build configuration change adjusts Boehm GC freelists to reduce allocation contention without any identified product or production-impacting defect; no actionable merge-blocking risk remains after normal checks and review. Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (2 skipped: 2 unsupported.) ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
2 participants
Motivation
Taken from #597.
Profiling 'nix search nixpkgs --no-eval-cache' showed that ~65% of all futex waits came from the Boehm GC, the largest share being the global allocation lock (GC_allocate_ml): with the default GC_TINY_FREELISTS of 25, only allocations up to 384 bytes are served from the per-thread freelists, and everything larger takes the global lock. Nixpkgs evaluation does ~755k such allocations per search — e.g. a typical derivation attrset (~46 attrs) is a 752-byte Bindings, of which one is allocated per package — all serialized on the lock across the worker threads.
Building bdwgc with GC_TINY_FREELISTS=96 (thread-local up to 1520 bytes) eliminates 93% of the global-lock allocations (755k -> 56k; what remains is essentially the >16 KiB attrset giants), halves the context-switch count at 8-16 workers, and makes the search ~10% faster at 24 workers (~3.7s -> ~3.3s). The per-thread memory cost is at most a partial heap block per size class and object kind.
Assisted-by: Claude Fable 5 noreply@anthropic.com
Context
Summary by CodeRabbit