From 2e99865be13ebeb493f65e17e06bb1064b63186a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 5 Sep 2026 21:27:42 +0200 Subject: [PATCH 1/2] perf(gc): route the trigger path and dirty-page barrier through hot TLS MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `gc_check_trigger` runs on every `gc_malloc`, and `gc_budgeted_due_trigger` resolved eleven raw `thread_local!` declarations one `_tlv_get_addr` call at a time. Measured with `sample` on the compiled claude-code TUI streaming a 3300-char reply (14,578 active main-thread samples, callers resolved by an explicit ancestor walk): `_tlv_get_addr` was 380 main-thread leaf samples, 71 of them with `gc_budgeted_due_trigger` as the immediate caller, 36 in `old_page_account_dirty_slots`, 31 in `scan_dirty_object_slots`, 27 in `gc_malloc_header_is_tracked`. Sixty-seven declarations move to `crate::perry_thread_local!`. Why they were still cold is a measurement bug in the gate, not an oversight: `scripts/check_thread_locals.py` ratchets on raw `thread_local!` BLOCKS per file, and a block holds any number of declarations — so `gc/policy.rs` counted as 6 while declaring 28, and adding a `static` to a recorded block passed silently. In the same unit as the hot side, main was 318 hot against 339 cold declarations. The gate now ratchets on declarations (385/272) and `--self-test` gained the direction that catches it. `ARENA_TOTAL_BYTES`, `BLOCK_POOL` and `BLOCK_POOL_BYTES` stay raw and say so: they are read from `Arena::new`, which runs as `tls_hot::fill`'s first provider, so a `HotKey` there re-enters `fill` — which has not yet written the `temp_roots` field it gates on — and re-runs `ARENA`'s initializer without bound. Claude-Session: https://claude.ai/code/session_014UZWia6L37DpA93VLtNK9m --- changelog.d/9830-gc-trigger-path-hot-tls.md | 55 +++++++++++ crates/perry-runtime/src/arena/block.rs | 44 +++++++++ crates/perry-runtime/src/arena/mod.rs | 2 + crates/perry-runtime/src/arena/page_meta.rs | 9 +- crates/perry-runtime/src/gc/barrier/mod.rs | 8 +- crates/perry-runtime/src/gc/malloc.rs | 17 +++- crates/perry-runtime/src/gc/old_free.rs | 10 +- crates/perry-runtime/src/gc/policy.rs | 62 ++++++++++-- crates/perry-runtime/src/gc/tenuring.rs | 6 +- crates/perry-runtime/src/gc/tests/mod.rs | 2 + .../src/gc/tests/tls_fill_reentrancy.rs | 43 +++++++++ .../src/gc/tests/trigger_path_tls.rs | 67 +++++++++++++ crates/perry-runtime/src/gc/trace.rs | 6 +- scripts/check_thread_locals.py | 62 +++++++++--- scripts/gc_runtime_root_holders.json | 4 +- scripts/thread_local_cold_allowlist.json | 94 +++++++++---------- 16 files changed, 408 insertions(+), 83 deletions(-) create mode 100644 changelog.d/9830-gc-trigger-path-hot-tls.md create mode 100644 crates/perry-runtime/src/gc/tests/tls_fill_reentrancy.rs create mode 100644 crates/perry-runtime/src/gc/tests/trigger_path_tls.rs diff --git a/changelog.d/9830-gc-trigger-path-hot-tls.md b/changelog.d/9830-gc-trigger-path-hot-tls.md new file mode 100644 index 0000000000..e791ac3b7d --- /dev/null +++ b/changelog.d/9830-gc-trigger-path-hot-tls.md @@ -0,0 +1,55 @@ +**The GC trigger path and the dirty-page barrier stop paying `_tlv_get_addr` +per read, and the policy gate that let them stop paying it now counts the +thing it is bounding.** + +`gc_check_trigger` runs on every `gc_malloc`, and its predicate +(`gc_budgeted_due_trigger`) resolved eleven raw `thread_local!` declarations +one out-of-line call at a time. Measured with `sample` on the compiled +claude-code TUI streaming a 3300-char reply (14,578 active main-thread +samples, callers resolved by an explicit ancestor walk rather than +nearest-symbol labels): `_tlv_get_addr` was 380 main-thread leaf samples, +**71 of them with `gc_budgeted_due_trigger` as the immediate caller**, 36 in +`old_page_account_dirty_slots`, 31 in `scan_dirty_object_slots`, 27 in +`gc_malloc_header_is_tracked`. `crates/perry-runtime/src/tls_hot.rs` has +existed to abolish exactly this since #7469; the allocation path's *fields* +were covered and the trigger path never was. + +Sixty-seven declarations across `gc/policy.rs`, `gc/malloc.rs`, `gc/old_free.rs`, +`gc/tenuring.rs`, `gc/trace.rs`, `gc/barrier/mod.rs`, `arena/block.rs` and +`arena/page_meta.rs` move to `crate::perry_thread_local!` — same syntax, same +`.with()` at every call site, the address served from this thread's hot cache +instead of a libdyld call. + +**Why they were still cold is a measurement bug in the gate, not an oversight +anyone could have noticed.** `scripts/check_thread_locals.py` ratchets on the +number of raw `thread_local!` **blocks** per file, while `thread_local! { … }` +holds any number of declarations — so `gc/policy.rs` counted as **6** while +declaring **28**, and adding a `static` to an already-recorded block passed +the gate silently. Counted in the same unit as the hot side, `main` was **318 +hot declarations against 339 cold ones** — cold was the majority, reported as +a 2.6:1 minority. The gate now ratchets on declarations (`385 hot / 272 +cold`), and `--self-test` grew a seventh direction that fails when a `static` +is added to a recorded block; restoring the block count makes that case, and +only that case, fail. + +Three declarations stay deliberately raw and say so at their declaration: +`ARENA_TOTAL_BYTES`, `BLOCK_POOL` and `BLOCK_POOL_BYTES` are read from +`Arena::new`, which runs as `tls_hot::fill`'s **first** provider, so a +`HotKey` there re-enters `fill` — which by design has not yet written the +`temp_roots` field it gates on — and re-runs `ARENA`'s initializer without +bound. It is a stack overflow at thread start, not a slow path, and it is the +first documented instance of the rule that a declaration read from inside a +`fill` provider cannot use the macro. `gc::tests::tls_fill_reentrancy` is the +standing guard, and it is sabotage-proved: moving `ARENA_TOTAL_BYTES` alone +into the neighbouring hot block aborts that test with `fatal runtime error: +stack overflow`. + +`gc::tests::trigger_path_tls` is the runtime half of the gate: it drives +`gc_check_trigger` on a fresh thread and asserts every trigger-path +declaration owns a hot slot and that the path publishes slots at all. +Reverting any one of them to a raw `thread_local!` removes `slot_index` and +breaks the build at that declaration's own name. It is a test that can fail +and did: the first run rejected `GC_DEFERRED_REQUEST` with `index 4294967295`, +correctly — `defer_gc_request` reads it only while a root lock is held, so it +is not a fast-path read and never claims a slot. The list is what the fast +path reads, not what the module declares. diff --git a/crates/perry-runtime/src/arena/block.rs b/crates/perry-runtime/src/arena/block.rs index 0e8a09e487..9f19920394 100644 --- a/crates/perry-runtime/src/arena/block.rs +++ b/crates/perry-runtime/src/arena/block.rs @@ -122,6 +122,29 @@ impl Drop for BlockPool { } } +// --------------------------------------------------------------------------- +// These three stay RAW, and not because nobody got to them: they are read from +// inside `Arena::new`, which runs as `ARENA`'s lazy initializer — and `ARENA` +// is the FIRST provider `tls_hot::fill` resolves. Routing them through the hot +// cache closes a cycle that ends in a stack overflow at thread start: +// +// HotKey::get -> hot() -> hot_uncached -> hot_via_tls +// -> (temp_roots still null) fill() +// -> arena_hot_addr() -> ARENA.with(..) -> Arena::new +// -> ARENA_TOTAL_BYTES / BLOCK_POOL{,_BYTES}.with(..) +// -> HotKey::get -> hot() -> ... (temp_roots STILL null) ... +// +// `fill` writes `temp_roots` last precisely so a re-entrant reader cannot see a +// half-filled cache as ready — which makes the nesting re-run `fill`, and +// re-run `ARENA`'s initializer, without bound. It bites on any thread where +// the first `hot()` precedes the first arena touch, i.e. every freshly spawned +// one, and it fails as a crash rather than as a slow path. +// +// The rule this is an instance of: a declaration read from the dynamic extent +// of a `tls_hot::fill` provider cannot use `crate::perry_thread_local!`. Today +// `ARENA` is the only provider whose initializer runs code, so this is the +// whole set. +// --------------------------------------------------------------------------- thread_local! { static BLOCK_POOL: RefCell = const { RefCell::new(BlockPool { blocks: Vec::new(), @@ -981,7 +1004,9 @@ thread_local! { /// alloc into a tombstone slot or the end, and release inside /// `arena_reset_empty_blocks`). pub(crate) static ARENA_TOTAL_BYTES: std::cell::Cell = const { std::cell::Cell::new(0) }; +} +crate::perry_thread_local! { /// Cached running sum of `block.offset` across the old-gen arena — /// the delta-maintained twin of `ARENA_TOTAL_BYTES` above, same /// rationale: `gc_budgeted_due_trigger()` reads the old-gen in-use @@ -1007,8 +1032,18 @@ thread_local! { /// the OldReclaim trigger. pub(crate) static OLD_GEN_IN_USE_BYTES: std::cell::Cell = const { std::cell::Cell::new(0) }; +} + +// `ARENA` and `INLINE_STATE` below stay raw: they are NAMED `HotTls` fields +// (`arena`, `inline_state`) whose addresses `tls_hot::fill` reads through the +// providers further down, and a named field is one dependent load cheaper +// than a claimed slot. Their neighbours in this block were never migrated. +thread_local! { pub(crate) static ARENA: UnsafeCell = UnsafeCell::new(Arena::new(HeapGeneration::Nursery, HeapSpace::NurseryEden)); +} + +crate::perry_thread_local! { /// Segregated long-lived arena (issue #179). Holds objects that are /// intentionally pinned for the lifetime of the program by explicit @@ -1056,6 +1091,9 @@ thread_local! { pub(crate) static OLD_ARENA: UnsafeCell = UnsafeCell::new(Arena::new_lazy(HeapGeneration::Old, HeapSpace::Old)); +} + +thread_local! { /// Inline allocator state — a cache of the current arena block's /// `(data, offset, size)` tuple, exposed via a stable pointer so /// codegen can emit inline bump-allocate IR without going through @@ -1072,6 +1110,12 @@ thread_local! { }) }; } +/// Hot-cache slot claimed by `OLD_GEN_IN_USE_BYTES`. See above. +#[cfg(test)] +pub(crate) fn old_gen_in_use_bytes_slot_index() -> u32 { + OLD_GEN_IN_USE_BYTES.slot_index() +} + // --- #7469 hot-TLS address providers. See `crate::tls_hot`. --- /// Address of this thread's `ARENA`. Resolving it once and caching it is what diff --git a/crates/perry-runtime/src/arena/mod.rs b/crates/perry-runtime/src/arena/mod.rs index 9ba5401b19..4a92a805c2 100644 --- a/crates/perry-runtime/src/arena/mod.rs +++ b/crates/perry-runtime/src/arena/mod.rs @@ -46,6 +46,8 @@ pub(crate) use block::{ /// allocation path uses instead of a per-access `_tlv_get_addr`. pub(crate) use block::{arena_hot_addr, hot_arena, hot_inline_state, inline_state_hot_addr}; #[cfg(test)] +pub(crate) use block::old_gen_in_use_bytes_slot_index; +#[cfg(test)] pub(crate) use block::{ block_pool_bytes_for_test, block_pool_explicit_drained_bytes_for_test, block_pool_put, force_next_block_alloc_failure, gc_trigger_arena_borrow_depth, gc_trigger_arena_calls, diff --git a/crates/perry-runtime/src/arena/page_meta.rs b/crates/perry-runtime/src/arena/page_meta.rs index bc6984dff3..0fe26f3c9b 100644 --- a/crates/perry-runtime/src/arena/page_meta.rs +++ b/crates/perry-runtime/src/arena/page_meta.rs @@ -319,13 +319,20 @@ pub(crate) struct OldArenaSourceBlockSelection { pub(crate) pages: crate::fast_hash::PtrHashSet, } +// `PAGE_GENERATIONS` and `PAGE_GENERATION_CACHE` stay raw because they are +// NAMED fields of `HotTls` (`page_generations`, `page_generation_cache`), whose +// providers below hand `tls_hot::fill` their addresses. A named field is one +// dependent load cheaper than a claimed slot, which is why the closed set +// exists; everything else in this file was simply never migrated. thread_local! { static PAGE_GENERATIONS: RefCell = RefCell::new(crate::fast_hash::new_ptr_hash_map()); static PAGE_GENERATION_CACHE: UnsafeCell = const { UnsafeCell::new(PageGenerationCacheSet::empty()) }; +} +crate::perry_thread_local! { static OLD_GEN_PAGE_OBJECTS: RefCell = RefCell::new(crate::fast_hash::new_ptr_hash_map()); @@ -1138,7 +1145,7 @@ pub(crate) fn register_old_object_pages(header_addr: usize, total_size: usize) { // which fails if a new toucher of either table appears without one. // --------------------------------------------------------------------------- -thread_local! { +crate::perry_thread_local! { /// Old-object page registrations not yet folded into `OLD_GEN_PAGE_OBJECTS`. /// Entries are `(header_addr, total_size)`; nothing here is dereferenced, so /// a deferred entry never keeps an object alive and is not a GC root — and diff --git a/crates/perry-runtime/src/gc/barrier/mod.rs b/crates/perry-runtime/src/gc/barrier/mod.rs index 808a66427c..a4bbd8e407 100644 --- a/crates/perry-runtime/src/gc/barrier/mod.rs +++ b/crates/perry-runtime/src/gc/barrier/mod.rs @@ -697,6 +697,10 @@ pub(super) unsafe fn scan_dirty_object_slots( // retained only as a test fallback for the previous object-level // HashSet behavior. +// The three declarations below stay raw: they are NAMED `HotTls` fields +// (`incremental_mark_valid_ptrs`, `birth_extra_flags`, +// `incremental_mark_minor_only`), one dependent load cheaper than a claimed +// slot. Everything after them was simply never migrated. thread_local! { /// Active incremental mark barrier state (Full AND budgeted Minor /// cycles — a Minor cycle sliced across mutator turns has exactly the @@ -737,7 +741,9 @@ thread_local! { /// later. Old children need no shading in a minor anyway: minors never /// collect live old-gen objects. pub(super) static INCREMENTAL_MARK_BARRIER_MINOR_ONLY: Cell = const { Cell::new(false) }; +} +crate::perry_thread_local! { /// Dirty old-generation pages that have received a YOUNG-gen /// pointer since the last collection. This is Perry's compact /// modbuf: barriers log bounded page regions, and minor GC scans @@ -1761,7 +1767,7 @@ fn mark_dirty_old_page_uncached(page: usize) -> bool { inserted } -thread_local! { +crate::perry_thread_local! { /// PERRY_GC_VERIFY_EVACUATION diagnostic only: every old page EVER marked /// dirty over the process lifetime (never cleared). Lets the verifier's /// missing-edge report say whether the slot's page was recorded at some diff --git a/crates/perry-runtime/src/gc/malloc.rs b/crates/perry-runtime/src/gc/malloc.rs index 7203300abf..df85b12362 100644 --- a/crates/perry-runtime/src/gc/malloc.rs +++ b/crates/perry-runtime/src/gc/malloc.rs @@ -196,7 +196,7 @@ pub(super) const MALLOC_STATE_HEAVY_LEN_THRESHOLD: usize = 64 * 1024; /// 7/8 load factor without further rehashes). pub(super) const MALLOC_STATE_HEAVY_CAPACITY: usize = 256 * 1024; -thread_local! { +crate::perry_thread_local! { pub(crate) static MALLOC_STATE: RefCell = RefCell::new(MallocState { objects: Vec::with_capacity(MALLOC_STATE_INITIAL_CAPACITY), set: crate::fast_hash::PtrHashSet::with_capacity_and_hasher( @@ -209,12 +209,25 @@ thread_local! { heavy_capacity_reserved: false, kind_telemetry: [MallocKindTelemetry::zero(); MALLOC_KIND_BUCKET_COUNT], }); +} +// `ARENA_FREE_LIST` / `ARENA_FREE_LIST_NONEMPTY` stay raw: they are NAMED +// `HotTls` fields (`arena_free_list`, `arena_free_list_nonempty`), one +// dependent load cheaper than a claimed slot. +thread_local! { pub(crate) static ARENA_FREE_LIST: RefCell> = const { RefCell::new(Vec::new()) }; pub(crate) static ARENA_FREE_LIST_NONEMPTY: std::cell::Cell = const { std::cell::Cell::new(false) }; } +/// Hot-cache slot claimed by `MALLOC_STATE`, whose length is the +/// `MallocCount` trigger's basis on every `gc_malloc`. Liveness +/// instrumentation for `gc::tests::trigger_path_tls`. +#[cfg(test)] +pub(crate) fn malloc_state_slot_index() -> u32 { + MALLOC_STATE.slot_index() +} + pub fn gc_malloc(size: usize, obj_type: u8) -> *mut u8 { let total = GC_HEADER_SIZE + size; let layout = Layout::from_size_align(total, 8).unwrap(); @@ -480,7 +493,7 @@ pub(super) fn malloc_sweep_revalidate_header( }) } -thread_local! { +crate::perry_thread_local! { pub(super) static MALLOC_REGISTRY_REBUILD_COUNT: Cell = const { Cell::new(0) }; } diff --git a/crates/perry-runtime/src/gc/old_free.rs b/crates/perry-runtime/src/gc/old_free.rs index 360a75c4c8..de64ba30e2 100644 --- a/crates/perry-runtime/src/gc/old_free.rs +++ b/crates/perry-runtime/src/gc/old_free.rs @@ -47,7 +47,7 @@ use super::*; -thread_local! { +crate::perry_thread_local! { /// total_size -> user_ptrs of swept holes of exactly that size. static OLD_FREE_MAP: RefCell>> = RefCell::new(crate::fast_hash::new_ptr_hash_map()); @@ -60,6 +60,14 @@ pub(crate) fn old_free_bytes() -> usize { OLD_FREE_BYTES.with(Cell::get) } +/// Hot-cache slot claimed by `OLD_FREE_BYTES`, which +/// `gc_budgeted_due_trigger` reads on every `gc_malloc`. Liveness +/// instrumentation for `gc::tests::trigger_path_tls`. +#[cfg(test)] +pub(crate) fn old_free_bytes_slot_index() -> u32 { + OLD_FREE_BYTES.slot_index() +} + fn old_free_push(user_ptr: usize, total_size: usize) { if user_ptr == 0 || total_size < GC_HEADER_SIZE { return; diff --git a/crates/perry-runtime/src/gc/policy.rs b/crates/perry-runtime/src/gc/policy.rs index 0977d5e70a..ef59bef746 100644 --- a/crates/perry-runtime/src/gc/policy.rs +++ b/crates/perry-runtime/src/gc/policy.rs @@ -5,7 +5,7 @@ pub(super) const GC_FLAG_IN_ALLOC: u8 = 0b01; /// Bit 1 of GC_FLAGS — suppression flag (JSON.parse). pub(super) const GC_FLAG_SUPPRESSED: u8 = 0b10; -thread_local! { +crate::perry_thread_local! { pub(super) static GC_FLAGS: Cell = const { Cell::new(0) }; } @@ -292,7 +292,7 @@ thread_local! { static GC_NURSERY_CAP_TEST_SUPPRESSED: Cell = const { Cell::new(false) }; } -thread_local! { +crate::perry_thread_local! { /// Lower bound for the next GC trigger. Bumped after each /// `gc_collect_inner` based on collection effectiveness (see the /// adaptive logic in `gc_check_trigger`). @@ -401,7 +401,7 @@ pub(super) const GC_MALLOC_COUNT_STEP_INITIAL: usize = 100_000; pub(super) const GC_MALLOC_COUNT_STEP_MAX: usize = 2_000_000; pub(super) const GC_MALLOC_COUNT_STEP_MIN: usize = 10_000; -thread_local! { +crate::perry_thread_local! { /// Per-program adaptive malloc-count step. Mirrors `GC_STEP_BYTES` /// behaviour: doubles when mostly-garbage, halves when mostly-live. pub(super) static GC_MALLOC_COUNT_STEP: std::cell::Cell = @@ -431,7 +431,7 @@ thread_local! { /// the Map/Set side-allocation finalizers) instead of leaking. const GC_EXTERNAL_SIDE_ALLOC_STEP: usize = 16 * 1024 * 1024; -thread_local! { +crate::perry_thread_local! { static GC_EXTERNAL_SIDE_ALLOC_PENDING: std::cell::Cell = const { std::cell::Cell::new(0) }; static GC_EXTERNAL_SIDE_LIVE_BYTES: std::cell::Cell = const { std::cell::Cell::new(0) }; } @@ -942,7 +942,7 @@ impl GcTriggerSnapshot { } } -thread_local! { +crate::perry_thread_local! { pub(super) static GC_DEFERRED_REQUEST: Cell = const { Cell::new(DeferredGcRequest::None) }; pub(super) static GC_OLD_RECLAIM_PENDING: Cell = const { Cell::new(false) }; @@ -2655,7 +2655,7 @@ enum BudgetedGcTrigger { MallocCount, } -thread_local! { +crate::perry_thread_local! { static GC_BUDGETED_CYCLE: RefCell> = const { RefCell::new(None) }; static GC_BUDGETED_CYCLE_ACTIVE: Cell = const { Cell::new(false) }; static GC_BUDGETED_STEP_ACTIVE: Cell = const { Cell::new(false) }; @@ -2690,6 +2690,56 @@ pub(super) fn gc_old_reclaim_debt_bytes(old_in_use: usize, baseline: usize) -> u old_in_use.saturating_sub(trigger) as u64 } +/// The thread-locals [`gc_budgeted_due_trigger`] reads on its fast path, with +/// the hot-cache slot each one claimed. +/// +/// Enumerating them in the module that reads them is what makes the coverage +/// test able to fail: reverting any declaration below to a raw +/// `thread_local!` removes `slot_index` and breaks this function's build at +/// the declaration's own name, rather than leaving a silently slow path — the +/// failure mode `tls_hot.rs` was written to abolish and that this path +/// nonetheless kept for three years. +#[cfg(test)] +pub(crate) fn trigger_path_hot_slot_indices() -> Vec<(&'static str, u32)> { + // Touch each one first: a slot is claimed on first read, not at + // declaration, so an unread declaration reports the unassigned sentinel. + // + // `GC_DEFERRED_REQUEST` is deliberately absent: `defer_gc_request` reads it + // only when `GC_ROOT_LOCK_DEPTH` is non-zero, so on the fast path it is + // never touched and never claims a slot. Listing it made this function's + // own test fail with `index 4294967295` on the first run — which is the + // evidence that the test can fail, and the reason the list is "what the + // fast path reads" rather than "what the module declares". + let _ = gc_budgeted_due_trigger(); + vec![ + ("GC_OLD_RECLAIM_PENDING", GC_OLD_RECLAIM_PENDING.slot_index()), + ( + "GC_LAST_OLD_RECLAIM_IN_USE_BYTES", + GC_LAST_OLD_RECLAIM_IN_USE_BYTES.slot_index(), + ), + ("GC_NEXT_MALLOC_TRIGGER", GC_NEXT_MALLOC_TRIGGER.slot_index()), + ("GC_NEXT_TRIGGER_BYTES", GC_NEXT_TRIGGER_BYTES.slot_index()), + ("GC_TRIGGER_ARMED", GC_TRIGGER_ARMED.slot_index()), + ( + "GC_EXTERNAL_SIDE_LIVE_BYTES", + GC_EXTERNAL_SIDE_LIVE_BYTES.slot_index(), + ), + ("GC_MAJOR_PACING_RETAINING", GC_MAJOR_PACING_RETAINING.slot_index()), + ("GC_FLAGS", GC_FLAGS.slot_index()), + ( + "GC_BUDGETED_CYCLE_ACTIVE", + GC_BUDGETED_CYCLE_ACTIVE.slot_index(), + ), + ("GC_BUDGETED_STEP_ACTIVE", GC_BUDGETED_STEP_ACTIVE.slot_index()), + ( + "OLD_GEN_IN_USE_BYTES", + crate::arena::old_gen_in_use_bytes_slot_index(), + ), + ("OLD_FREE_BYTES", super::old_free_bytes_slot_index()), + ("MALLOC_STATE", super::malloc_state_slot_index()), + ] +} + fn gc_budgeted_due_trigger() -> Option { let old_pending = GC_OLD_RECLAIM_PENDING.with(Cell::get); // #6010: external Map/Set side-buffer bytes escalate to OldReclaim too. diff --git a/crates/perry-runtime/src/gc/tenuring.rs b/crates/perry-runtime/src/gc/tenuring.rs index 4fd221261f..1abb9dd99c 100644 --- a/crates/perry-runtime/src/gc/tenuring.rs +++ b/crates/perry-runtime/src/gc/tenuring.rs @@ -152,7 +152,7 @@ const RAISE_DEBOUNCE_CYCLES: u8 = 2; /// adaptive threshold has eliminated the re-copying). const NURSERY_CAP_SCALE_MAX: u8 = 4; -thread_local! { +crate::perry_thread_local! { static TENURING_SURVIVALS: Cell = const { Cell::new(GC_TENURING_SURVIVALS_MAX) }; static RAISE_STREAK: Cell = const { Cell::new(0) }; /// Survival-rate lock: promote-on-first-copy until influx goes quiet. @@ -354,7 +354,7 @@ pub(super) fn note_surviving_object_census(moved_bytes: usize, moved_objects: us return; } OBJECT_CENSUS_SEEDED.with(|seeded| seeded.set(true)); - let previous = MEAN_SURVIVING_OBJECT_BYTES.replace(mean); + let previous = MEAN_SURVIVING_OBJECT_BYTES.with(|c| c.replace(mean)); if previous != mean && crate::gc::gc_diag_enabled() { eprintln!( "[gc-tenuring] nursery cap object denomination: mean_surviving_object_bytes {} -> {} \ @@ -417,7 +417,7 @@ pub(super) fn maybe_seed_object_census_from_allocation(from_space_in_use_bytes: if mean == 0 { return; } - let previous = MEAN_SURVIVING_OBJECT_BYTES.replace(mean); + let previous = MEAN_SURVIVING_OBJECT_BYTES.with(|c| c.replace(mean)); if crate::gc::gc_diag_enabled() { eprintln!( "[gc-tenuring] nursery cap object denomination: allocation census seeded \ diff --git a/crates/perry-runtime/src/gc/tests/mod.rs b/crates/perry-runtime/src/gc/tests/mod.rs index ac6d08f01e..959a63a087 100644 --- a/crates/perry-runtime/src/gc/tests/mod.rs +++ b/crates/perry-runtime/src/gc/tests/mod.rs @@ -58,6 +58,8 @@ mod smoke; mod step_bounds; pub(super) mod support; mod teardown; +mod tls_fill_reentrancy; +mod trigger_path_tls; mod telemetry_verifier; mod temp_roots; mod triggers; diff --git a/crates/perry-runtime/src/gc/tests/tls_fill_reentrancy.rs b/crates/perry-runtime/src/gc/tests/tls_fill_reentrancy.rs new file mode 100644 index 0000000000..da66fb1113 --- /dev/null +++ b/crates/perry-runtime/src/gc/tests/tls_fill_reentrancy.rs @@ -0,0 +1,43 @@ +//! A thread whose FIRST act is resolving the hot-TLS cache must survive it. +//! +//! `tls_hot::fill`'s first provider is `arena_hot_addr()`, which initializes +//! `ARENA` — and `Arena::new` reads `ARENA_TOTAL_BYTES` and `BLOCK_POOL`. If +//! either of those were declared with `crate::perry_thread_local!`, the read +//! would go `HotKey::get -> hot() -> hot_uncached -> hot_via_tls`, find +//! `temp_roots` still null (`fill` writes it LAST, deliberately, so a +//! re-entrant reader cannot mistake a half-filled cache for a ready one), call +//! `fill` again, and re-run `ARENA`'s initializer — without bound. +//! +//! That failure is a stack overflow at thread start, not a slow path, and it +//! only appears on a thread where the first `hot()` precedes the first arena +//! touch. This test is that thread. It is the standing guard on the rule +//! recorded at those three declarations in `arena/block.rs`: a thread-local +//! read from inside the dynamic extent of a `fill` provider cannot use the +//! macro. +//! +//! Sabotage-proved: moving `ARENA_TOTAL_BYTES` alone into the +//! `crate::perry_thread_local!` block below it makes this test abort with +//! `thread '' has overflowed its stack / fatal runtime error: stack +//! overflow` (release, 2026-09-05). It is not a hypothetical. +#[test] +fn a_fresh_thread_may_resolve_the_hot_cache_before_touching_the_arena() { + // A small stack so an unbounded `fill` recursion aborts here rather than + // running long enough to look like a hang. + std::thread::Builder::new() + .stack_size(1 << 20) + .spawn(|| { + // `published_slots` reaches `hot()` and nothing else, so this is + // the first arena-touching call on the thread. + let _ = crate::tls_hot::published_slots(); + // And the arena still works afterwards: `fill` ran to completion + // rather than being abandoned mid-way by a recursion guard. + assert!( + crate::arena::arena_total_bytes() > 0, + "the arena reported no reserved bytes after `fill`, so \ + `ARENA`'s initializer did not complete on this thread" + ); + }) + .unwrap() + .join() + .expect("resolving the hot TLS cache first on a fresh thread panicked"); +} diff --git a/crates/perry-runtime/src/gc/tests/trigger_path_tls.rs b/crates/perry-runtime/src/gc/tests/trigger_path_tls.rs new file mode 100644 index 0000000000..e5791c4e8e --- /dev/null +++ b/crates/perry-runtime/src/gc/tests/trigger_path_tls.rs @@ -0,0 +1,67 @@ +//! The `gc_check_trigger` fast path must resolve its thread-locals through the +//! hot cache, not through `_tlv_get_addr`. +//! +//! `gc_check_trigger` runs on **every** `gc_malloc`. Measured with `sample` on +//! the compiled claude-code TUI streaming a 3300-char reply (14,580 main-thread +//! samples), `_tlv_get_addr` was 380 main-thread leaf samples and **85 of them +//! sat under `gc_budgeted_due_trigger < gc_check_trigger < gc_malloc`** — the +//! trigger predicate resolving `GC_OLD_RECLAIM_PENDING`, +//! `GC_LAST_OLD_RECLAIM_IN_USE_BYTES`, `GC_NEXT_MALLOC_TRIGGER`, +//! `GC_TRIGGER_ARMED`, `GC_NEXT_TRIGGER_BYTES`, `GC_EXTERNAL_SIDE_LIVE_BYTES`, +//! `OLD_GEN_IN_USE_BYTES`, `ARENA_TOTAL_BYTES`, `OLD_FREE_BYTES`, +//! `MALLOC_STATE` and the survivor cells one out-of-line call at a time. +//! +//! `scripts/check_thread_locals.py` cannot express this: it ratchets on the +//! number of raw `thread_local!` *blocks* per file, so `gc/policy.rs` read as +//! "6" while declaring 28 cold thread-locals, and the block count does not move +//! when a declaration is converted alongside a split (`arena/block.rs` went +//! from 11 cold declarations to 0 with its recorded count unchanged at 2). The +//! declaration-denominated ratchet added in the same change closes the *policy* +//! hole; this test is the *runtime* half — it asserts the mechanism is live on +//! the path the profile named. + +use crate::tls_hot::{published_slots, HOT_SLOT_CAPACITY}; + +/// Every trigger-path declaration named above must own a hot slot, and +/// driving `gc_check_trigger` must actually populate slots on the calling +/// thread. +/// +/// Two assertions, because they fail to different sabotage. Reverting one +/// declaration to a raw `thread_local!` removes `slot_index` and breaks the +/// build at the named line; making the slot allocator hand out its overflow +/// sentinel keeps the build green and trips the bound check below. +#[test] +fn gc_check_trigger_resolves_its_thread_locals_through_the_hot_cache() { + // A fresh thread so the "how many slots did this path publish" reading is + // this path's, not the harness's. + std::thread::spawn(|| { + let before = published_slots(); + crate::gc::gc_check_trigger(); + let after = published_slots(); + + for (name, idx) in crate::gc::policy::trigger_path_hot_slot_indices() { + assert!( + (idx as usize) < HOT_SLOT_CAPACITY, + "{name} did not claim a hot TLS slot (index {idx}, capacity \ + {HOT_SLOT_CAPACITY}); it is paying `_tlv_get_addr` per read on \ + every `gc_malloc`", + ); + } + + // The predicate reads at least the ten declarations listed in the + // module docs. A lower bound, deliberately: `gc_check_trigger` reaches + // other hot declarations too, and this must not fail when an unrelated + // one is added or removed. It DOES fail if the trigger path stops + // going through `HotKey` at all, which is the regression #7469 has + // already suffered three times. + let published = after - before; + assert!( + published >= 11, + "gc_check_trigger published only {published} hot TLS slots on a \ + fresh thread ({before} -> {after}); the trigger path's \ + thread-locals are resolving through `_tlv_get_addr` again", + ); + }) + .join() + .expect("trigger-path TLS probe thread panicked"); +} diff --git a/crates/perry-runtime/src/gc/trace.rs b/crates/perry-runtime/src/gc/trace.rs index fb350daa45..0232cc6a60 100644 --- a/crates/perry-runtime/src/gc/trace.rs +++ b/crates/perry-runtime/src/gc/trace.rs @@ -1,6 +1,6 @@ use super::*; -thread_local! { +crate::perry_thread_local! { /// Set by test-only helpers that wipe page metadata for isolation /// (`old_arena_page_index_clear_for_tests`): real objects become /// unclassifiable in that synthetic state, so the differential verifier @@ -85,7 +85,7 @@ pub(super) fn classifier_verify_enabled() -> bool { *CACHED.get_or_init(|| super::env_flag_enabled("PERRY_GC_VERIFY_CLASSIFIER")) } -thread_local! { +crate::perry_thread_local! { pub(super) static MARK_SEEDS: std::cell::UnsafeCell> = const { std::cell::UnsafeCell::new(Vec::new()) }; } @@ -922,7 +922,7 @@ pub(super) fn trace_marked_objects(valid_ptrs: &ValidPointerSet) { /// old-block neighbors as new block-persist candidates. pub(super) const BLOCK_PERSIST_WINDOW: usize = 5; -thread_local! { +crate::perry_thread_local! { /// Objects this thread's block-persistence pass has FORCE-MARKED since /// process start — i.e. kept alive for no reason other than sharing a /// block with something reachable. diff --git a/scripts/check_thread_locals.py b/scripts/check_thread_locals.py index b2db1a1481..bff22628f2 100755 --- a/scripts/check_thread_locals.py +++ b/scripts/check_thread_locals.py @@ -211,11 +211,23 @@ def cfg_test_module_files(root: Path, crates: list[str]) -> set[str]: return test_only -def shipping_raw_blocks(src: str) -> int: - """Raw `thread_local!` blocks that survive into a non-test build. +def shipping_raw_declarations(src: str) -> int: + """Raw `thread_local!` DECLARATIONS that survive into a non-test build. Skips a block carrying `#[cfg(test)]` directly above it, and any block inside an inline `#[cfg(test)] mod … { … }`. + + Declarations, not blocks. `_tlv_get_addr` is paid per *declaration* read, + and `thread_local! { … }` holds any number of them, so a block-denominated + ratchet cannot see the thing it exists to bound. Measured on this + repository at `d36a1af0c`: the 122 recorded cold BLOCKS were **339 cold + declarations**, `gc/policy.rs` alone recorded as 6 while declaring 28 — + and a `static` added to an already-recorded block passed the gate in + silence, which is how the allocation path re-accumulated the cost #7469 + removed. Counting declarations also makes the two halves of the report + commensurable: the hot side was already counted in declarations, so + "318 hot / 122 cold" was comparing unlike units and read as a 2.6:1 + majority for the mechanism when cold was in fact the larger number. """ gated_spans = [brace_span(src, m.start()) for m in CFG_TEST_INLINE_MOD_RE.finditer(src)] count = 0 @@ -226,12 +238,13 @@ def shipping_raw_blocks(src: str) -> int: line = preceding[preceding.rfind("\n") + 1 :].strip() if line == "#[cfg(test)]": continue - count += 1 + open_at, close_at = brace_span(src, m.start()) + count += len(DECL_RE.findall(src[open_at + 1 : close_at])) return count def scan(root: Path, crates: list[str]) -> tuple[dict[str, int], int]: - """Raw `thread_local!` blocks per file, and total hot declarations.""" + """Raw `thread_local!` declarations per file, and total hot declarations.""" raw: dict[str, int] = {} hot_declarations = 0 test_only_files = cfg_test_module_files(root, crates) @@ -249,7 +262,7 @@ def scan(root: Path, crates: list[str]) -> tuple[dict[str, int], int]: ) if rel in EXCLUDED or rel in test_only_files: continue - count = shipping_raw_blocks(src) + count = shipping_raw_declarations(src) if count: raw[rel] = count return raw, hot_declarations @@ -271,7 +284,7 @@ def verify(root: Path, crates: list[str], allowlist_path: Path) -> list[str]: for rel, count in sorted(raw.items()): if rel not in recorded: problems.append( - f"{rel}: {count} raw `thread_local!` block(s), none allowed.\n" + f"{rel}: {count} raw `thread_local!` declaration(s), none allowed.\n" f" Use `crate::perry_thread_local!` — same syntax, same " f"`.with()` at every call site, and the address lands in this " f"thread's hot cache instead of costing a `_tlv_get_addr` call " @@ -281,7 +294,7 @@ def verify(root: Path, crates: list[str], allowlist_path: Path) -> list[str]: elif recorded[rel] != count: direction = "gained" if count > recorded[rel] else "lost" problems.append( - f"{rel}: {direction} raw `thread_local!` blocks " + f"{rel}: {direction} raw `thread_local!` declarations " f"({recorded[rel]} recorded, {count} found). " f"Convert it, or run --update to re-record." ) @@ -290,7 +303,7 @@ def verify(root: Path, crates: list[str], allowlist_path: Path) -> list[str]: if rel not in raw: problems.append( f"{rel}: recorded as having {recorded[rel]} cold " - f"`thread_local!` block(s), but has none. A stale entry is one " + f"`thread_local!` declaration(s), but has none. A stale entry is one " f"nobody has to justify — delete it with --update." ) @@ -311,9 +324,11 @@ def write_allowlist(root: Path, crates: list[str], allowlist_path: Path) -> None json.dumps( { "_comment": ( - "Files still declaring raw `thread_local!`. Every entry is a " - "declaration that pays `_tlv_get_addr` on Darwin; the count is " - "a ratchet, so adding one to an already-listed file fails too. " + "Files still declaring raw `thread_local!`. The count is the " + "number of DECLARATIONS that survive into a shipping build — " + "each one pays `_tlv_get_addr` per read on Darwin — and it is a " + "ratchet, so adding a `static` to an already-listed file fails " + "whether or not it opens a new block. " "New code should use `crate::perry_thread_local!` — see " "crates/perry-runtime/src/tls_hot.rs. Regenerate with " "scripts/check_thread_locals.py --update." @@ -364,13 +379,30 @@ def self_test() -> int: failures.append("a new raw `thread_local!` in an unlisted file passed") (src_dir / "new.rs").unlink() - # 2. A second raw declaration in an already-listed file must fail. + # 2. A second raw declaration in an already-listed file must fail — + # in a NEW block… write_source(src_dir / "cold.rs", "thread_local! { static A: u8 = const { 0 }; }\n" "thread_local! { static D: u8 = const { 0 }; }\n" ) if not verify(root, CRATES, allowlist): - failures.append("a raw `thread_local!` added to a listed file passed") + failures.append("a raw `thread_local!` block added to a listed file passed") + + # 2b. …and, the half a block-denominated ratchet could not see, inside + # the block that is ALREADY recorded. This is how the allocation + # path re-accumulated the cost #7469 removed: `gc/policy.rs` + # declared 28 cold thread-locals while the allowlist read "6", and + # every one after the first was free to add. + write_source(src_dir / "cold.rs", + "thread_local! {\n" + " static A: u8 = const { 0 };\n" + " static D: u8 = const { 0 };\n" + "}\n" + ) + if not verify(root, CRATES, allowlist): + failures.append( + "a raw `static` added to a listed file's EXISTING block passed" + ) # 3. A stale entry must fail. write_source(src_dir / "cold.rs", @@ -429,7 +461,7 @@ def self_test() -> int: print(f"SELF-TEST FAILED: {f}", file=sys.stderr) if failures: return 1 - print("self-test: the checker can fail in all six directions") + print("self-test: the checker can fail in all seven directions") return 0 @@ -455,7 +487,7 @@ def main() -> int: raw, hot_declarations = scan(REPO, CRATES) print( f"thread-local policy OK: {hot_declarations} hot declarations, " - f"{sum(raw.values())} raw blocks in {len(raw)} recorded cold files, " + f"{sum(raw.values())} cold declarations in {len(raw)} recorded files, " f"capacity {hot_slot_capacity(REPO)}" ) return 0 diff --git a/scripts/gc_runtime_root_holders.json b/scripts/gc_runtime_root_holders.json index b52c8629f5..e3b4d1fd22 100644 --- a/scripts/gc_runtime_root_holders.json +++ b/scripts/gc_runtime_root_holders.json @@ -270,7 +270,7 @@ "file": "crates/perry-runtime/src/gc/census.rs", "name": "PASS1_MARKED", "verdict": "non_moving_snapshot", - "why": "Real GC header addresses, deliberately untraced so the diagnostic does not keep its observed objects alive. Populated only at the end of mark propagation of a synchronous full cycle; consumed at sweep entry in the same run_to_completion invocation. The intervening full-cycle phases do not relocate or run JS callbacks. The Vec is used for membership comparisons and dropped with the census before sweep. Budgeted and minor cycles skip both boundaries. Pin re-audited 2026-09-05 after #9760 touched `gc/mod.rs`: that change is `mod heap_stats;` plus a `pub(crate) use` re-export and alters no mark/sweep control flow. `heap_stats()` is reached only from `js_bun_jsc_heap_stats` (the JS-facing `bun:jsc.heapStats()`), i.e. from mutator code, never inside a cycle, and its own module contract forbids allocation or collection during its walk. The mark-complete \u2192 sweep-entry window is unchanged. Re-audited 2026-09-05 (train125) after #9769 and #9771 touched pinned files. #9769 adds one `reg_scanner!` registration to `gc/mod.rs`; #9771 adds a feature-gated `alloc_census_init()` there and a feature-gated Rust-heap dump inside `take_census`. `alloc-census` is not in the default feature set, and decisively: `census_take_if_armed_at_full_sweep_start` does `PASS1_MARKED.with(|p| p.borrow_mut().take())` BEFORE calling `take_census`, so the snapshot has already left the thread-local by the time #9771's code runs \u2014 it cannot affect the window. Neither change alters mark/sweep control flow. Re-audited 2026-09-05 (train126) after #9755 restructured `gc/cycle.rs`. Its hunks are all root-scan machinery (`RootScanSubphase`, `RootScanCycleState`, the mutable-scanner iteration state), which runs BEFORE mark propagation completes; `gc/mod.rs` gains only a `mod young_log;` declaration. The bracketing is unchanged \u2014 `census_pass1_if_armed` is still inside `step_mark_propagation` and `census_take_if_armed_at_full_sweep_start` inside `step_sweep` \u2014 and a synchronous full mark-sweep still moves nothing between them.", + "why": "Real GC header addresses, deliberately untraced so the diagnostic does not keep its observed objects alive. Populated only at the end of mark propagation of a synchronous full cycle; consumed at sweep entry in the same run_to_completion invocation. The intervening full-cycle phases do not relocate or run JS callbacks. The Vec is used for membership comparisons and dropped with the census before sweep. Budgeted and minor cycles skip both boundaries. Pin re-audited 2026-09-05 after #9760 touched `gc/mod.rs`: that change is `mod heap_stats;` plus a `pub(crate) use` re-export and alters no mark/sweep control flow. `heap_stats()` is reached only from `js_bun_jsc_heap_stats` (the JS-facing `bun:jsc.heapStats()`), i.e. from mutator code, never inside a cycle, and its own module contract forbids allocation or collection during its walk. The mark-complete \u2192 sweep-entry window is unchanged. Re-audited 2026-09-05 (train125) after #9769 and #9771 touched pinned files. #9769 adds one `reg_scanner!` registration to `gc/mod.rs`; #9771 adds a feature-gated `alloc_census_init()` there and a feature-gated Rust-heap dump inside `take_census`. `alloc-census` is not in the default feature set, and decisively: `census_take_if_armed_at_full_sweep_start` does `PASS1_MARKED.with(|p| p.borrow_mut().take())` BEFORE calling `take_census`, so the snapshot has already left the thread-local by the time #9771's code runs \u2014 it cannot affect the window. Neither change alters mark/sweep control flow. Re-audited 2026-09-05 (train126) after #9755 restructured `gc/cycle.rs`. Its hunks are all root-scan machinery (`RootScanSubphase`, `RootScanCycleState`, the mutable-scanner iteration state), which runs BEFORE mark propagation completes; `gc/mod.rs` gains only a `mod young_log;` declaration. The bracketing is unchanged \u2014 `census_pass1_if_armed` is still inside `step_mark_propagation` and `census_take_if_armed_at_full_sweep_start` inside `step_sweep` \u2014 and a synchronous full mark-sweep still moves nothing between them. Re-audited 2026-09-05 after #9830 touched `gc/policy.rs`. That change is (a) six `thread_local! {` blocks rewritten as `crate::perry_thread_local! {` and (b) one `#[cfg(test)]` accessor listing the trigger path's hot-slot indices. The macro keeps the same storage, the same `.with()` at every read and write, and the same destructor registration (the teardown guard exists exactly when `needs_drop` holds, which is what `std::thread_local!` already decided); no value, predicate or branch in the file changes, so no mark or sweep control flow does. The one new behaviour is on a declaration's FIRST read: `HotKey::resolve_and_cache` takes a mutex and allocates a key through the GLOBAL allocator. Even if a first read landed inside this window it would be sound \u2014 the window's contract is that nothing relocates and no JS callback runs, and a mimalloc allocation does neither. `census_pass1_if_armed` is still inside `step_mark_propagation` and `census_take_if_armed_at_full_sweep_start` inside `step_sweep`; the bracketing is untouched.", "window": { "start": { "file": "crates/perry-runtime/src/gc/census.rs", @@ -288,7 +288,7 @@ "crates/perry-runtime/src/gc/census.rs": "8050a9d1ca15f783195ccfa5963089b60bc4a6c31d9ced5755537623e70e7e3c", "crates/perry-runtime/src/gc/cycle.rs": "763d552271b8e983a796b4e9648cd8ee984a0602b2b56aeefdb8713c0049c31f", "crates/perry-runtime/src/gc/mod.rs": "085c3dcde34a172aa2b96ee4500658abae77cd34ee7f0e2dfeee06ae5774a414", - "crates/perry-runtime/src/gc/policy.rs": "319ed42f1a985c88f6362657a08518077283fe5216d6055fc82343b34dec50f9", + "crates/perry-runtime/src/gc/policy.rs": "92dbf50a9838a1b040dfb369b88edf1a8bc508c82240ae053d0261a07ed070e4", "crates/perry-runtime/src/gc/progress.rs": "a5ad3971bbe4047229ca57325234780daa85921dbc778e1c08dff4ad07ccfb96" } } diff --git a/scripts/thread_local_cold_allowlist.json b/scripts/thread_local_cold_allowlist.json index 4744f170ac..9686b6e210 100644 --- a/scripts/thread_local_cold_allowlist.json +++ b/scripts/thread_local_cold_allowlist.json @@ -1,83 +1,79 @@ { - "_comment": "Files still declaring raw `thread_local!`. Every entry is a declaration that pays `_tlv_get_addr` on Darwin; the count is a ratchet, so adding one to an already-listed file fails too. New code should use `crate::perry_thread_local!` \u2014 see crates/perry-runtime/src/tls_hot.rs. Regenerate with scripts/check_thread_locals.py --update.", - "_hot_declarations": 287, + "_comment": "Files still declaring raw `thread_local!`. The count is the number of DECLARATIONS that survive into a shipping build \u2014 each one pays `_tlv_get_addr` per read on Darwin \u2014 and it is a ratchet, so adding a `static` to an already-listed file fails whether or not it opens a new block. New code should use `crate::perry_thread_local!` \u2014 see crates/perry-runtime/src/tls_hot.rs. Regenerate with scripts/check_thread_locals.py --update.", + "_hot_declarations": 385, "files": { "crates/perry-runtime/src/agent.rs": 1, - "crates/perry-runtime/src/arena/block.rs": 2, + "crates/perry-runtime/src/arena/block.rs": 5, "crates/perry-runtime/src/arena/page_meta.rs": 2, - "crates/perry-runtime/src/async_context.rs": 2, - "crates/perry-runtime/src/async_hooks.rs": 3, - "crates/perry-runtime/src/builtins/console.rs": 2, - "crates/perry-runtime/src/builtins/formatting.rs": 5, + "crates/perry-runtime/src/async_context.rs": 3, + "crates/perry-runtime/src/async_hooks.rs": 7, + "crates/perry-runtime/src/builtins/console.rs": 4, + "crates/perry-runtime/src/builtins/formatting.rs": 9, "crates/perry-runtime/src/builtins/formatting/boxed_primitives.rs": 1, - "crates/perry-runtime/src/builtins/globals.rs": 2, + "crates/perry-runtime/src/builtins/globals.rs": 4, "crates/perry-runtime/src/bun_ffi/types.rs": 1, "crates/perry-runtime/src/child_process/reactor.rs": 1, - "crates/perry-runtime/src/child_process/v8_serde.rs": 1, + "crates/perry-runtime/src/child_process/v8_serde.rs": 2, "crates/perry-runtime/src/closure/dispatch/errors.rs": 1, - "crates/perry-runtime/src/cluster.rs": 2, + "crates/perry-runtime/src/cluster.rs": 3, "crates/perry-runtime/src/dyn_eval/bridge.rs": 1, "crates/perry-runtime/src/dyn_eval/env.rs": 1, - "crates/perry-runtime/src/dyn_eval/mod.rs": 1, + "crates/perry-runtime/src/dyn_eval/mod.rs": 6, "crates/perry-runtime/src/eh.rs": 1, "crates/perry-runtime/src/eh_walker.rs": 1, - "crates/perry-runtime/src/error.rs": 2, + "crates/perry-runtime/src/error.rs": 3, "crates/perry-runtime/src/event_pump.rs": 1, - "crates/perry-runtime/src/fs/dir_glob_watch/watch.rs": 1, - "crates/perry-runtime/src/fs/filehandle.rs": 1, - "crates/perry-runtime/src/fs/mod.rs": 1, - "crates/perry-runtime/src/fs/stream.rs": 1, - "crates/perry-runtime/src/gc/barrier/mod.rs": 2, - "crates/perry-runtime/src/gc/barrier_arming.rs": 1, + "crates/perry-runtime/src/fs/dir_glob_watch/watch.rs": 7, + "crates/perry-runtime/src/fs/filehandle.rs": 6, + "crates/perry-runtime/src/fs/mod.rs": 6, + "crates/perry-runtime/src/fs/stream.rs": 4, + "crates/perry-runtime/src/gc/barrier/mod.rs": 3, + "crates/perry-runtime/src/gc/barrier_arming.rs": 3, "crates/perry-runtime/src/gc/cycle_malloc_trim.rs": 1, "crates/perry-runtime/src/gc/fromspace_scan.rs": 1, "crates/perry-runtime/src/gc/layout.rs": 1, - "crates/perry-runtime/src/gc/layout_tables.rs": 1, + "crates/perry-runtime/src/gc/layout_tables.rs": 3, "crates/perry-runtime/src/gc/malloc.rs": 2, - "crates/perry-runtime/src/gc/mod.rs": 2, - "crates/perry-runtime/src/gc/old_free.rs": 1, - "crates/perry-runtime/src/gc/policy.rs": 6, - "crates/perry-runtime/src/gc/promote_in_place.rs": 1, - "crates/perry-runtime/src/gc/roots/runtime_handles.rs": 1, + "crates/perry-runtime/src/gc/mod.rs": 3, + "crates/perry-runtime/src/gc/promote_in_place.rs": 11, + "crates/perry-runtime/src/gc/roots/runtime_handles.rs": 2, "crates/perry-runtime/src/gc/roots/scan_mode.rs": 1, "crates/perry-runtime/src/gc/roots/shadow_stack.rs": 2, "crates/perry-runtime/src/gc/roots/temp_roots.rs": 1, - "crates/perry-runtime/src/gc/scan_fallback.rs": 1, + "crates/perry-runtime/src/gc/scan_fallback.rs": 2, "crates/perry-runtime/src/gc/shape_install.rs": 1, - "crates/perry-runtime/src/gc/telemetry.rs": 2, - "crates/perry-runtime/src/gc/tenuring.rs": 1, - "crates/perry-runtime/src/gc/trace.rs": 3, + "crates/perry-runtime/src/gc/telemetry.rs": 3, "crates/perry-runtime/src/intl/number_format.rs": 1, "crates/perry-runtime/src/iter_result.rs": 1, - "crates/perry-runtime/src/json/mod.rs": 1, + "crates/perry-runtime/src/json/mod.rs": 12, "crates/perry-runtime/src/json/raw_json.rs": 1, "crates/perry-runtime/src/json_tape.rs": 2, - "crates/perry-runtime/src/json_tape_store.rs": 1, + "crates/perry-runtime/src/json_tape_store.rs": 3, "crates/perry-runtime/src/media_playback.rs": 1, - "crates/perry-runtime/src/native_arena.rs": 1, + "crates/perry-runtime/src/native_arena.rs": 3, "crates/perry-runtime/src/node_http2_constants.rs": 1, - "crates/perry-runtime/src/node_inspector.rs": 1, + "crates/perry-runtime/src/node_inspector.rs": 2, "crates/perry-runtime/src/node_repl.rs": 1, - "crates/perry-runtime/src/node_stream_constructors.rs": 2, - "crates/perry-runtime/src/node_stream_tests.rs": 1, - "crates/perry-runtime/src/node_submodules/blob.rs": 1, - "crates/perry-runtime/src/node_submodules/diagnostics.rs": 3, - "crates/perry-runtime/src/node_submodules/diagnostics_tail.rs": 1, - "crates/perry-runtime/src/node_submodules/mod.rs": 1, - "crates/perry-runtime/src/node_submodules/test.rs": 1, - "crates/perry-runtime/src/node_submodules/test_once_unit_tests.rs": 1, + "crates/perry-runtime/src/node_stream_constructors.rs": 3, + "crates/perry-runtime/src/node_stream_tests.rs": 23, + "crates/perry-runtime/src/node_submodules/blob.rs": 4, + "crates/perry-runtime/src/node_submodules/diagnostics.rs": 9, + "crates/perry-runtime/src/node_submodules/diagnostics_tail.rs": 2, + "crates/perry-runtime/src/node_submodules/mod.rs": 4, + "crates/perry-runtime/src/node_submodules/test.rs": 11, + "crates/perry-runtime/src/node_submodules/test_once_unit_tests.rs": 2, "crates/perry-runtime/src/node_submodules/test_property.rs": 1, - "crates/perry-runtime/src/node_submodules/trace_events.rs": 1, - "crates/perry-runtime/src/object/native_module/callable_exports.rs": 2, + "crates/perry-runtime/src/node_submodules/trace_events.rs": 8, + "crates/perry-runtime/src/object/native_module/callable_exports.rs": 3, "crates/perry-runtime/src/object/spill.rs": 1, "crates/perry-runtime/src/os/os_process_emitter.rs": 1, - "crates/perry-runtime/src/os_process_streams.rs": 1, - "crates/perry-runtime/src/perf_hooks.rs": 2, - "crates/perry-runtime/src/process.rs": 2, - "crates/perry-runtime/src/process/env_misc.rs": 3, + "crates/perry-runtime/src/os_process_streams.rs": 3, + "crates/perry-runtime/src/perf_hooks.rs": 11, + "crates/perry-runtime/src/process.rs": 12, + "crates/perry-runtime/src/process/env_misc.rs": 4, "crates/perry-runtime/src/process/permission.rs": 1, "crates/perry-runtime/src/process/report.rs": 1, - "crates/perry-runtime/src/proxy.rs": 1, + "crates/perry-runtime/src/proxy.rs": 5, "crates/perry-runtime/src/pty/reactor.rs": 1, "crates/perry-runtime/src/static_plugins.rs": 1, "crates/perry-runtime/src/timer.rs": 1, @@ -87,7 +83,7 @@ "crates/perry-runtime/src/util_promisify.rs": 1, "crates/perry-runtime/src/v8.rs": 2, "crates/perry-runtime/src/wasi.rs": 1, - "crates/perry-runtime/src/weakref.rs": 1, - "crates/perry-runtime/src/web_storage.rs": 1 + "crates/perry-runtime/src/weakref.rs": 2, + "crates/perry-runtime/src/web_storage.rs": 2 } } From 4a58e853e5ff8cd8f4b679a8d7c8339f0af0a251 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 5 Sep 2026 22:39:00 +0200 Subject: [PATCH 2/2] docs(changelog): rename the fragment to its PR number Claude-Session: https://claude.ai/code/session_014UZWia6L37DpA93VLtNK9m --- ...gc-trigger-path-hot-tls.md => 9827-gc-trigger-path-hot-tls.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename changelog.d/{9830-gc-trigger-path-hot-tls.md => 9827-gc-trigger-path-hot-tls.md} (100%) diff --git a/changelog.d/9830-gc-trigger-path-hot-tls.md b/changelog.d/9827-gc-trigger-path-hot-tls.md similarity index 100% rename from changelog.d/9830-gc-trigger-path-hot-tls.md rename to changelog.d/9827-gc-trigger-path-hot-tls.md