From 0d92005429ca0388ff6206eba88bfdf287eeac1b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sun, 6 Sep 2026 04:41:02 +0000 Subject: [PATCH 1/3] fix(gc): price the tiny-parse pressure guard by the productivity backoff Issue #9831 measured the ArenaBytes arm firing 51 times in one 66-delta claude-code reply, each collection freeing a median 131 KB, while the adaptive step sat saturated at 1 GiB. The issue located the discarded backoff in the arm's own re-arm arithmetic; correcting that (the issue's refuted branch) bought -10.8 % CPU for +22 % settled footprint and was rightly rejected. The arm's re-arm is not what re-fires it. Between two consecutive firings the arena grows a few hundred KB, against a trigger armed 16 MB (and below the ceiling, up to 128 MB) above the post-collection total. What pulls the trigger back down is the tiny-parse pressure guard: after every `JSON.parse` that grew the arena by <= 1 MB, `gc_bump_malloc_trigger` (and `gc_schedule_parse_boundary_collection_ if_pressure`, and the boundary collector they arm) tests the absolute `arena_in_use_bytes() >= 48 MB` and, if so, sets the trigger to "now". That threshold is a quantity no collection can lower below the live set, so on a program whose live set never drops under it every small parse -- one per SSE delta -- forced a minor at the next safepoint. The step those minors doubled was consulted by nothing. The guard now also requires the arena to have grown, since the last collection of any kind ended, by a headroom priced from the step: the step rescaled so that its power-on value (128 MB, the ceiling) buys the 16 MB floor, and each doubling the arm's ceiling clamp discards buys the guard one more doubling, bounded by the same ceiling. A productive collection halves the step and the guard keeps the cadence it always had; an unproductive one earns it room. The boundary collector re-prices a pending request so a collection that already satisfied it is not followed by a second one. Measured on the compiled claude-code TUI (cli_2.1.112.js, Linux, same perry binary, runtime-only A/B, 7 interleaved rounds, 3300-char streamed reply, chunk 50): turn CPU base 30.2-41.5 s (mean 35.1) fix 27.8-29.2 s (mean 28.6) post-turn RSS base 754-1057 MB (mean 803) fix 733-855 MB (mean 786) post-idle RSS base 527-1073 MB (mean 736) fix 517-843 MB (mean 722) peak RSS 1964-2062 MB both arms The fix wins CPU in every pair (-8 % to -30 %); footprint is flat within the base's own spread. The base arm is bimodal in both, which is what an absolute in-use threshold does. PERRY_GC_DIAG on one reply: copying minors 104 -> 84 (ArenaBytes 41 -> 13), old-gen fulls 19 -> 7, and the guard forced exactly one collection, after a genuine 16 MB of growth (`[gc-tiny-parse]` is the new witness line). test_memory_json_churn -- the guard's motivating shape -- is byte-identical in output and RSS in all four GC modes; 48/48 test_gap_gc_* and 8/8 test_gap_json_* pass. The arm's own arithmetic is left as it was and now says why. Claude-Session: https://claude.ai/code/session_015kqVkH6rHzfvXskGAj3tRv --- crates/perry-runtime/src/gc/policy.rs | 136 ++++++++++- crates/perry-runtime/src/gc/tests/mod.rs | 1 + .../src/gc/tests/tiny_parse_pressure.rs | 219 ++++++++++++++++++ scripts/gc_runtime_root_holders.json | 10 +- 4 files changed, 357 insertions(+), 9 deletions(-) create mode 100644 crates/perry-runtime/src/gc/tests/tiny_parse_pressure.rs diff --git a/crates/perry-runtime/src/gc/policy.rs b/crates/perry-runtime/src/gc/policy.rs index 0977d5e70a..830c6cc70c 100644 --- a/crates/perry-runtime/src/gc/policy.rs +++ b/crates/perry-runtime/src/gc/policy.rs @@ -367,6 +367,92 @@ pub(super) fn gc_suppressed_parse_is_tiny(parse_growth: usize) -> bool { parse_growth <= GC_SUPPRESSED_TINY_PARSE_BYTES } +/// #9831: how much the arena must have grown since the last collection ended +/// before tiny-parse churn may force another one — priced by the part of the +/// productivity backoff the `ArenaBytes` arm computes and then discards. +/// +/// `GC_STEP_BYTES` powers on at `GC_THRESHOLD_INITIAL_BYTES`, which equals the +/// trigger ceiling, and doubles on every collection that frees almost nothing. +/// The arm's own re-arm (`gc_finish_arena_trigger_collection`) clamps +/// `new_total + step` at the ceiling, so every doubling past the initial step +/// is backoff the arm has computed, stored, and cannot express. This rescales +/// the step so that its power-on value buys exactly the headroom floor, and +/// each discarded doubling buys the guard one more doubling — up to the same +/// ceiling, so a run of unproductive collections can never let the guard wait +/// longer than the arm itself would. A productive collection halves the step +/// toward its 16 MB floor, which maps below the headroom floor and clamps to +/// it: a churn loop whose collections pay keeps today's cadence. +pub(super) fn tiny_parse_pressure_headroom_bytes(step: usize) -> usize { + let floor = gc_trigger_headroom_floor_bytes(); + let ceiling = gc_trigger_absolute_ceiling_bytes(); + // 64-bit intermediate on purpose: `step` reaches 1 GiB and `floor` 16 MiB, + // whose product does not fit a 32-bit `usize` (watchOS/visionOS are ILP32; + // see `influx_driven_nursery_cap_bytes` for the same trap). + let scaled = (step as u64).saturating_mul(floor as u64) / (GC_THRESHOLD_INITIAL_BYTES as u64); + let scaled = scaled.min(usize::MAX as u64) as usize; + scaled.max(floor).min(ceiling.max(floor)) +} + +/// #9831: whether tiny-parse churn has earned a forced collection. +/// +/// The guard used to be `in_use >= in_use_trigger` alone — an absolute +/// threshold on a quantity no collection can lower below the live set. On a +/// program whose live set sits above it permanently (the compiled claude-code +/// TUI holds 59–297 MB of arena through one streamed reply) that made every +/// tiny `JSON.parse` the SSE stream consists of force a collection at the next +/// safepoint: 51 minors in one 66-delta reply, each freeing a median 131 KB, +/// while the adaptive step sat at its 1 GiB maximum saying "back off" and +/// nothing consulted it. The growth clause is what that step is for. +/// +/// `base` is `arena_in_use_bytes()` as the last collection ended +/// (`GC_TINY_PARSE_PRESSURE_BASE_BYTES`); `in_use` is the same reading now. +pub(super) fn tiny_parse_pressure_due_with( + in_use: usize, + in_use_trigger: usize, + base: usize, + step: usize, +) -> bool { + in_use >= in_use_trigger + && in_use >= base.saturating_add(tiny_parse_pressure_headroom_bytes(step)) +} + +/// The live [`tiny_parse_pressure_due_with`]: current base and step. +pub(super) fn tiny_parse_pressure_due(in_use: usize, in_use_trigger: usize) -> bool { + let base = GC_TINY_PARSE_PRESSURE_BASE_BYTES.with(Cell::get); + let step = GC_STEP_BYTES.with(Cell::get); + tiny_parse_pressure_due_with(in_use, in_use_trigger, base, step) +} + +/// The in-use reading the tiny-parse guard compares against in this collector +/// mode: the generational collector's guard sits higher than the full +/// mark-sweep one because a minor is the cheaper collection to force. +fn tiny_parse_in_use_trigger_for_mode() -> usize { + if gen_gc_enabled() { + gc_tiny_parse_in_use_trigger_dyn_bytes() + } else { + gc_tiny_parse_full_gc_in_use_trigger_dyn_bytes() + } +} + +/// `PERRY_GC_DIAG=1` witness that the guard is the arm that forced a +/// collection (CLAUDE.md: a gate must assert its subject was live). One line +/// per forced collection, tagged with which parse boundary asked for it. +fn diag_tiny_parse_forced_collection(site: &str, in_use: usize) { + if !crate::gc::gc_diag_enabled() { + return; + } + let base = GC_TINY_PARSE_PRESSURE_BASE_BYTES.with(Cell::get); + let step = GC_STEP_BYTES.with(Cell::get); + eprintln!( + "[gc-tiny-parse] forced collection site={} in_use={} base={} headroom={} step={}", + site, + in_use, + base, + tiny_parse_pressure_headroom_bytes(step), + step + ); +} + pub(super) fn gc_bump_arena_trigger_target( bytes_now: usize, step: usize, @@ -980,6 +1066,12 @@ thread_local! { /// reading and still escalate. Nursery garbage is not. pub(super) static GC_LAST_COLLECTION_POST_IN_USE_BYTES: Cell = const { Cell::new(0) }; + /// #9831: `arena_in_use_bytes()` as the most recent collection of ANY kind + /// ended — the base the tiny-parse pressure guard measures growth from. + /// The bump-offset reading rather than the live census above, because the + /// guard compares against `arena_in_use_bytes()` at every parse boundary, + /// and mixing the two would count every swept hole as growth. + pub(super) static GC_TINY_PARSE_PRESSURE_BASE_BYTES: Cell = const { Cell::new(0) }; /// Yield-adaptive backoff for major-GC pacing (#7726). /// /// `arena_growth_full_escalation_due` escalates a minor to a full once the @@ -1347,12 +1439,12 @@ pub fn gc_bump_malloc_trigger() { let is_tiny_parse = gc_bump_malloc_trigger_with_snapshot(current, bytes_now); if is_tiny_parse { let use_gen_gc = gen_gc_enabled(); - let in_use_trigger = if use_gen_gc { - gc_tiny_parse_in_use_trigger_dyn_bytes() - } else { - gc_tiny_parse_full_gc_in_use_trigger_dyn_bytes() - }; - if crate::arena::arena_in_use_bytes() < in_use_trigger { + // #9831: an absolute in-use guard alone forced a collection after + // EVERY tiny parse on a program whose live set never drops below it. + // The guard now also requires the arena to have grown past the + // productivity-priced headroom since the last collection ended. + let in_use = crate::arena::arena_in_use_bytes(); + if !tiny_parse_pressure_due(in_use, tiny_parse_in_use_trigger_for_mode()) { return; } if use_gen_gc { @@ -1361,6 +1453,7 @@ pub fn gc_bump_malloc_trigger() { return; } GC_SUPPRESSED_TINY_PARSE_COLLECTION_PENDING.with(|pending| pending.set(true)); + diag_tiny_parse_forced_collection("parse_end", in_use); GC_NEXT_TRIGGER_BYTES.with(|trigger| { if trigger.get() > bytes_now { trigger.set(bytes_now); @@ -1396,6 +1489,15 @@ pub fn gc_collect_pending_suppressed_parse() { GC_SUPPRESSED_TINY_PARSE_COLLECTION_PENDING.with(|pending| pending.set(true)); return; } + // #9831: the request was priced when it was made; a collection of any kind + // since then has moved the base, and re-pricing here is what keeps the + // boundary collection from stacking a second minor on top of it. A request + // nothing has satisfied is still due and still collects. + let in_use = crate::arena::arena_in_use_bytes(); + if !tiny_parse_pressure_due(in_use, tiny_parse_in_use_trigger_for_mode()) { + return; + } + diag_tiny_parse_forced_collection("parse_boundary", in_use); let total = crate::arena::arena_total_bytes(); GC_NEXT_TRIGGER_BYTES.with(|trigger| { @@ -1419,7 +1521,12 @@ pub fn gc_schedule_parse_boundary_collection_if_pressure() { if !gen_gc_enabled() { return; } - if crate::arena::arena_in_use_bytes() < gc_tiny_parse_in_use_trigger_dyn_bytes() { + // #9831: priced the same way as the post-parse guard above — see + // `tiny_parse_pressure_due_with`. + if !tiny_parse_pressure_due( + crate::arena::arena_in_use_bytes(), + gc_tiny_parse_in_use_trigger_dyn_bytes(), + ) { return; } GC_SUPPRESSED_TINY_PARSE_COLLECTION_PENDING.with(|pending| pending.set(true)); @@ -1911,6 +2018,8 @@ pub(super) fn pacing_arena_in_use_bytes() -> usize { pub(super) fn note_collection_finished_arena_occupancy(full: bool) { let bytes = pacing_arena_in_use_bytes(); GC_LAST_COLLECTION_POST_IN_USE_BYTES.with(|cell| cell.set(bytes)); + // #9831: the same moment, in the units the tiny-parse guard reads. + GC_TINY_PARSE_PRESSURE_BASE_BYTES.with(|cell| cell.set(crate::arena::arena_in_use_bytes())); super::arena_right_size::note_collection_finished(bytes, full); } @@ -2199,6 +2308,19 @@ fn gc_finish_arena_trigger_collection(pre_in_use: usize, outcome: GcCollectOutco // `new_total` so a workload whose post-GC live set already // approaches the ceiling doesn't thrash on every fresh // allocation. + // + // #9831: above `ceiling - floor` this arithmetic re-arms at `new_total + + // floor` whatever `step` says — the productivity backoff the branch above + // just computed is not visible to it. That is deliberate, and measured: + // pricing the arm's own headroom by the step bought -10.8 % turn CPU on + // the compiled claude-code TUI and cost +22 % settled footprint — a + // CPU-for-footprint trade this project does not accept (the issue's own + // refuted branch). The step is instead consumed where it was + // actually being discarded — the tiny-parse pressure guard + // (`tiny_parse_pressure_due_with`), which pulled this trigger down to + // "now" after every small `JSON.parse` on a heap above its in-use + // threshold and so re-fired this arm 51 times in one reply regardless of + // what this function armed. let stepped = new_total.saturating_add(step); let capped = stepped.min(gc_trigger_absolute_ceiling_bytes()); let floor = new_total.saturating_add(gc_trigger_headroom_floor_bytes()); diff --git a/crates/perry-runtime/src/gc/tests/mod.rs b/crates/perry-runtime/src/gc/tests/mod.rs index ac6d08f01e..1dd69f6e99 100644 --- a/crates/perry-runtime/src/gc/tests/mod.rs +++ b/crates/perry-runtime/src/gc/tests/mod.rs @@ -60,6 +60,7 @@ pub(super) mod support; mod teardown; mod telemetry_verifier; mod temp_roots; +mod tiny_parse_pressure; mod triggers; mod typed_layout_intact_residual; mod u8_inline_cache; diff --git a/crates/perry-runtime/src/gc/tests/tiny_parse_pressure.rs b/crates/perry-runtime/src/gc/tests/tiny_parse_pressure.rs new file mode 100644 index 0000000000..fd4be0507d --- /dev/null +++ b/crates/perry-runtime/src/gc/tests/tiny_parse_pressure.rs @@ -0,0 +1,219 @@ +//! #9831: the tiny-parse pressure guard prices the collections it forces. +//! +//! The guard used to be an absolute `arena_in_use_bytes() >= 48 MB` test, so +//! on a program whose live set never drops below that it forced a collection +//! after EVERY tiny `JSON.parse` — the adaptive step's backoff was computed +//! by each of those collections and consulted by none of them. These tests +//! pin the pricing: the headroom the guard demands between collections is the +//! part of the step the `ArenaBytes` arm's ceiling clamp discards, and the +//! guard is not due until the arena has grown that much since the last +//! collection ended. +//! +//! Sabotage-proved: restoring the absolute guard (dropping the growth clause +//! from `tiny_parse_pressure_due_with`) fails +//! `the_absolute_guard_alone_is_the_bug_the_growth_clause_exists_for` and +//! `growth_past_the_headroom_is_due`'s boundary half while the rest pass; +//! pricing the headroom at the raw step (`floor.max(step.min(ceiling))`) +//! fails `power_on_step_buys_exactly_the_headroom_floor`. + +use super::super::heap_budget::{ + gc_trigger_absolute_ceiling_bytes, gc_trigger_headroom_floor_bytes, +}; +use super::super::policy::{ + tiny_parse_pressure_due, tiny_parse_pressure_due_with, tiny_parse_pressure_headroom_bytes, + GC_STEP_BYTES, GC_THRESHOLD_INITIAL_BYTES, GC_THRESHOLD_MAX_BYTES, + GC_TINY_PARSE_PRESSURE_BASE_BYTES, +}; + +const MB: usize = 1024 * 1024; + +/// Restores the two live cells the guard reads, so a test that moves them +/// cannot leak its state into the next one (the suite is single-threaded, but +/// the cells outlive the test). +struct LiveCellsGuard { + step: usize, + base: usize, +} + +impl LiveCellsGuard { + fn set(step: usize, base: usize) -> Self { + Self { + step: GC_STEP_BYTES.with(|cell| cell.replace(step)), + base: GC_TINY_PARSE_PRESSURE_BASE_BYTES.with(|cell| cell.replace(base)), + } + } +} + +impl Drop for LiveCellsGuard { + fn drop(&mut self) { + GC_STEP_BYTES.with(|cell| cell.set(self.step)); + GC_TINY_PARSE_PRESSURE_BASE_BYTES.with(|cell| cell.set(self.base)); + } +} + +#[test] +fn power_on_step_buys_exactly_the_headroom_floor() { + // The step powers on at the trigger ceiling. That is not evidence of an + // unproductive collection — nothing has run yet — so the guard keeps the + // cadence it always had: the headroom floor, not the ceiling. + assert_eq!( + tiny_parse_pressure_headroom_bytes(GC_THRESHOLD_INITIAL_BYTES), + gc_trigger_headroom_floor_bytes(), + "a step that has never been priced must buy the floor, not the ceiling" + ); +} + +#[test] +fn a_productive_collection_keeps_the_headroom_floor() { + // Productive collections halve the step toward its own 16 MB floor. Every + // step at or below the power-on value maps to the headroom floor. + let floor = gc_trigger_headroom_floor_bytes(); + for step in [16 * MB, 32 * MB, 64 * MB, GC_THRESHOLD_INITIAL_BYTES / 2] { + assert_eq!( + tiny_parse_pressure_headroom_bytes(step), + floor, + "step {step} is a productive reading and must keep the floor" + ); + } +} + +#[test] +fn each_doubling_the_ceiling_discards_doubles_the_headroom() { + let floor = gc_trigger_headroom_floor_bytes(); + let ceiling = gc_trigger_absolute_ceiling_bytes(); + let mut previous = tiny_parse_pressure_headroom_bytes(GC_THRESHOLD_INITIAL_BYTES); + for doublings in 1..=3u32 { + let step = GC_THRESHOLD_INITIAL_BYTES << doublings; + let headroom = tiny_parse_pressure_headroom_bytes(step); + let expected = (floor << doublings).min(ceiling); + assert_eq!( + headroom, expected, + "{doublings} discarded doubling(s) must buy floor << {doublings}, bounded by the ceiling" + ); + assert!( + headroom >= previous, + "backing off further must never shrink the headroom" + ); + if ceiling >= (floor << doublings) { + assert!( + headroom > previous, + "an unproductive collection must earn more headroom than the reading before it" + ); + } + previous = headroom; + } +} + +#[test] +fn headroom_is_bounded_by_the_absolute_ceiling() { + let ceiling = gc_trigger_absolute_ceiling_bytes(); + let saturated = tiny_parse_pressure_headroom_bytes(GC_THRESHOLD_MAX_BYTES); + assert!( + saturated <= ceiling, + "a saturated step ({saturated}) must not let the guard wait longer than the arm's ceiling ({ceiling})" + ); + // On the unconstrained desktop budget the saturated step (1 GiB, three + // doublings past the 128 MB initial) reaches the 128 MB ceiling exactly; + // under a small `PERRY_GC_HEAP_LIMIT` the ceiling is lower and the clamp + // binds earlier. Either way the saturated reading IS the ceiling. + if ceiling <= gc_trigger_headroom_floor_bytes() << 3 { + assert_eq!(saturated, ceiling); + } +} + +#[test] +fn below_the_in_use_trigger_is_never_due() { + let trigger = 48 * MB; + // Even with zero base and the most productive step, the guard stays off + // below its in-use trigger: small heaps are the regular arms' business. + assert!(!tiny_parse_pressure_due_with( + trigger - 1, + trigger, + 0, + 16 * MB + )); + assert!(!tiny_parse_pressure_due_with(0, trigger, 0, 16 * MB)); +} + +#[test] +fn the_absolute_guard_alone_is_the_bug_the_growth_clause_exists_for() { + // The measured shape: a 60 MB live set above the 48 MB trigger, a tiny + // parse that grew the arena by a few KB since the collection that just + // ran, and a step saturated at its maximum because those collections free + // nothing. The old guard said "collect" here after every parse. + let trigger = 48 * MB; + let base = 60 * MB; + let in_use = base + 4096; + assert!( + !tiny_parse_pressure_due_with(in_use, trigger, base, GC_THRESHOLD_MAX_BYTES), + "a few KB of growth past a collection that freed nothing must not force another" + ); + // Nor with a productive step: 4 KB is below the headroom floor too. + assert!(!tiny_parse_pressure_due_with( + in_use, + trigger, + base, + 16 * MB + )); +} + +#[test] +fn growth_past_the_headroom_is_due() { + let trigger = 48 * MB; + let base = 60 * MB; + for step in [16 * MB, GC_THRESHOLD_INITIAL_BYTES, GC_THRESHOLD_MAX_BYTES] { + let headroom = tiny_parse_pressure_headroom_bytes(step); + let boundary = base + headroom; + assert!( + tiny_parse_pressure_due_with(boundary, trigger, base, step), + "growth of exactly the headroom ({headroom}) at step {step} is due" + ); + assert!( + !tiny_parse_pressure_due_with(boundary - 1, trigger, base, step), + "one byte short of the headroom ({headroom}) at step {step} is not" + ); + } +} + +#[test] +fn the_live_predicate_reads_the_step_and_the_base() { + let trigger = 48 * MB; + let base = 60 * MB; + let floor = gc_trigger_headroom_floor_bytes(); + let ceiling = gc_trigger_absolute_ceiling_bytes(); + + // Saturated step: the guard waits for the ceiling's worth of growth. + let _cells = LiveCellsGuard::set(GC_THRESHOLD_MAX_BYTES, base); + if ceiling > floor { + assert!(!tiny_parse_pressure_due(base + floor, trigger)); + } + assert!(tiny_parse_pressure_due(base + ceiling.max(floor), trigger)); + + // Productive step: the floor is enough again. + GC_STEP_BYTES.with(|cell| cell.set(16 * MB)); + assert!(tiny_parse_pressure_due(base + floor, trigger)); + assert!(!tiny_parse_pressure_due(base + floor - 1, trigger)); +} + +#[test] +fn a_finished_collection_moves_the_base_to_the_post_collection_reading() { + use super::super::js_gc_collect; + // Whatever the base was, a completed collection re-baselines it to the + // arena's post-collection in-use reading — the same reading the guard + // compares against at the next parse boundary. Assert the identity of the + // two readings, not merely that the cell moved: a base recorded in other + // units (the live census) would count every swept hole as growth. + let _cells = LiveCellsGuard::set(GC_STEP_BYTES.with(|cell| cell.get()), usize::MAX); + js_gc_collect(); + let base = GC_TINY_PARSE_PRESSURE_BASE_BYTES.with(|cell| cell.get()); + assert_ne!( + base, + usize::MAX, + "a finished collection must record the base" + ); + assert_eq!( + base, + crate::arena::arena_in_use_bytes(), + "the base must be the post-collection `arena_in_use_bytes()` reading" + ); +} diff --git a/scripts/gc_runtime_root_holders.json b/scripts/gc_runtime_root_holders.json index b52c8629f5..0b2afbf149 100644 --- a/scripts/gc_runtime_root_holders.json +++ b/scripts/gc_runtime_root_holders.json @@ -270,7 +270,7 @@ "file": "crates/perry-runtime/src/gc/census.rs", "name": "PASS1_MARKED", "verdict": "non_moving_snapshot", - "why": "Real GC header addresses, deliberately untraced so the diagnostic does not keep its observed objects alive. Populated only at the end of mark propagation of a synchronous full cycle; consumed at sweep entry in the same run_to_completion invocation. The intervening full-cycle phases do not relocate or run JS callbacks. The Vec is used for membership comparisons and dropped with the census before sweep. Budgeted and minor cycles skip both boundaries. Pin re-audited 2026-09-05 after #9760 touched `gc/mod.rs`: that change is `mod heap_stats;` plus a `pub(crate) use` re-export and alters no mark/sweep control flow. `heap_stats()` is reached only from `js_bun_jsc_heap_stats` (the JS-facing `bun:jsc.heapStats()`), i.e. from mutator code, never inside a cycle, and its own module contract forbids allocation or collection during its walk. The mark-complete \u2192 sweep-entry window is unchanged. Re-audited 2026-09-05 (train125) after #9769 and #9771 touched pinned files. #9769 adds one `reg_scanner!` registration to `gc/mod.rs`; #9771 adds a feature-gated `alloc_census_init()` there and a feature-gated Rust-heap dump inside `take_census`. `alloc-census` is not in the default feature set, and decisively: `census_take_if_armed_at_full_sweep_start` does `PASS1_MARKED.with(|p| p.borrow_mut().take())` BEFORE calling `take_census`, so the snapshot has already left the thread-local by the time #9771's code runs \u2014 it cannot affect the window. Neither change alters mark/sweep control flow. Re-audited 2026-09-05 (train126) after #9755 restructured `gc/cycle.rs`. Its hunks are all root-scan machinery (`RootScanSubphase`, `RootScanCycleState`, the mutable-scanner iteration state), which runs BEFORE mark propagation completes; `gc/mod.rs` gains only a `mod young_log;` declaration. The bracketing is unchanged \u2014 `census_pass1_if_armed` is still inside `step_mark_propagation` and `census_take_if_armed_at_full_sweep_start` inside `step_sweep` \u2014 and a synchronous full mark-sweep still moves nothing between them.", + "why": "Real GC header addresses, deliberately untraced so the diagnostic does not keep its observed objects alive. Populated only at the end of mark propagation of a synchronous full cycle; consumed at sweep entry in the same run_to_completion invocation. The intervening full-cycle phases do not relocate or run JS callbacks. The Vec is used for membership comparisons and dropped with the census before sweep. Budgeted and minor cycles skip both boundaries. Pin re-audited 2026-09-05 after #9760 touched `gc/mod.rs`: that change is `mod heap_stats;` plus a `pub(crate) use` re-export and alters no mark/sweep control flow. `heap_stats()` is reached only from `js_bun_jsc_heap_stats` (the JS-facing `bun:jsc.heapStats()`), i.e. from mutator code, never inside a cycle, and its own module contract forbids allocation or collection during its walk. The mark-complete \u2192 sweep-entry window is unchanged. Re-audited 2026-09-05 (train125) after #9769 and #9771 touched pinned files. #9769 adds one `reg_scanner!` registration to `gc/mod.rs`; #9771 adds a feature-gated `alloc_census_init()` there and a feature-gated Rust-heap dump inside `take_census`. `alloc-census` is not in the default feature set, and decisively: `census_take_if_armed_at_full_sweep_start` does `PASS1_MARKED.with(|p| p.borrow_mut().take())` BEFORE calling `take_census`, so the snapshot has already left the thread-local by the time #9771's code runs \u2014 it cannot affect the window. Neither change alters mark/sweep control flow. Re-audited 2026-09-05 (train126) after #9755 restructured `gc/cycle.rs`. Its hunks are all root-scan machinery (`RootScanSubphase`, `RootScanCycleState`, the mutable-scanner iteration state), which runs BEFORE mark propagation completes; `gc/mod.rs` gains only a `mod young_log;` declaration. The bracketing is unchanged \u2014 `census_pass1_if_armed` is still inside `step_mark_propagation` and `census_take_if_armed_at_full_sweep_start` inside `step_sweep` \u2014 and a synchronous full mark-sweep still moves nothing between them. Re-audited 2026-09-06 after #9831 touched `gc/policy.rs`. Its hunks are (a) the tiny-parse pressure guard's pricing (`tiny_parse_pressure_headroom_bytes`, `tiny_parse_pressure_due*`, a `Cell` byte-count base) consulted from JSON.parse's mutator-side boundaries (`gc_bump_malloc_trigger`, `gc_collect_pending_suppressed_parse`, `gc_schedule_parse_boundary_collection_if_pressure`), none of which is reachable from inside a cycle, and (b) one extra `Cell` store in `note_collection_finished_arena_occupancy`, which runs from `publish_reclaim_outcome` in the Publish subphase \u2014 after `step_sweep` has already consumed the snapshot. Mark/sweep control flow between `census_pass1_if_armed` and `census_take_if_armed_at_full_sweep_start` is untouched.", "window": { "start": { "file": "crates/perry-runtime/src/gc/census.rs", @@ -288,7 +288,7 @@ "crates/perry-runtime/src/gc/census.rs": "8050a9d1ca15f783195ccfa5963089b60bc4a6c31d9ced5755537623e70e7e3c", "crates/perry-runtime/src/gc/cycle.rs": "763d552271b8e983a796b4e9648cd8ee984a0602b2b56aeefdb8713c0049c31f", "crates/perry-runtime/src/gc/mod.rs": "085c3dcde34a172aa2b96ee4500658abae77cd34ee7f0e2dfeee06ae5774a414", - "crates/perry-runtime/src/gc/policy.rs": "319ed42f1a985c88f6362657a08518077283fe5216d6055fc82343b34dec50f9", + "crates/perry-runtime/src/gc/policy.rs": "5626929989c5093109cd9322ad4ff3415bd1585ac33bfb2070b06cc4fa2a0207", "crates/perry-runtime/src/gc/progress.rs": "a5ad3971bbe4047229ca57325234780daa85921dbc778e1c08dff4ad07ccfb96" } } @@ -311,6 +311,12 @@ "verdict": "not_a_gc_pointer", "why": "#9772: releasable block BYTES the last idle selection promised \u2014 a size, not an address. A `Cell` compared against what the collection actually released." }, + { + "file": "crates/perry-runtime/src/gc/policy.rs", + "name": "GC_TINY_PARSE_PRESSURE_BASE_BYTES", + "verdict": "not_a_gc_pointer", + "why": "#9831: `arena_in_use_bytes()` (a sum of block bump offsets) recorded as each collection ends, read back by the tiny-parse pressure guard to price growth since then. A byte COUNT, never an address or a NaN-boxed value: written only from `note_collection_finished_arena_occupancy` and the test seam, read only by `tiny_parse_pressure_due`/`diag_tiny_parse_forced_collection`. Same shape as its frontier siblings `GC_LAST_COLLECTION_POST_IN_USE_BYTES`/`GC_STEP_BYTES`, with the verdict those still owe." + }, { "file": "crates/perry-runtime/src/gc/trace.rs", "name": "FORWARDED_STUB_MEMBERSHIP_RECOVERIES", From 644b9d36247a53150edb2c3b301e2a964ad8b870 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sun, 6 Sep 2026 04:42:35 +0000 Subject: [PATCH 2/3] docs(changelog): add the #9838 fragment Claude-Session: https://claude.ai/code/session_015kqVkH6rHzfvXskGAj3tRv --- .../9838-tiny-parse-pressure-pricing.md | 37 +++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 changelog.d/9838-tiny-parse-pressure-pricing.md diff --git a/changelog.d/9838-tiny-parse-pressure-pricing.md b/changelog.d/9838-tiny-parse-pressure-pricing.md new file mode 100644 index 0000000000..dcd96c9b53 --- /dev/null +++ b/changelog.d/9838-tiny-parse-pressure-pricing.md @@ -0,0 +1,37 @@ +**The tiny-parse pressure guard now prices the collections it forces by the +adaptive step's productivity backoff (#9831).** On the compiled claude-code +TUI a 3300-character streamed reply spent 30–41 s of CPU in the base arm and +27.8–29.2 s with the fix (mean −19 %, every interleaved pair a win), with +post-turn and post-idle RSS flat within the base's own spread and peak RSS +unchanged. + +#9831 measured the `ArenaBytes` arm firing 51 times in one 66-delta reply, +each collection freeing a median 131 KB, while the adaptive step sat +saturated at 1 GiB — and located the discarded backoff in the arm's own +ceiling clamp. That clamp was not what re-fired the arm: between two firings +the arena grew a few hundred KB against a trigger armed 16–128 MB above the +post-collection total. What pulled the trigger down was the tiny-parse +pressure guard, which after every `JSON.parse` growing the arena by ≤ 1 MB +tested the absolute `arena_in_use_bytes() >= 48 MB` and, if it held, set the +trigger to "now". That is a quantity no collection can lower below the live +set, so on a heap that sits above it permanently every small parse (one per +SSE delta) forced a minor whose backoff nothing read — #9589's shape one +trigger over. + +The guard now also requires the arena to have grown, since the last +collection of any kind ended, by a headroom priced from the step: the step +rescaled so its power-on value buys the 16 MB headroom floor and each +doubling the arm's clamp discards buys one more doubling, bounded by the +trigger ceiling. A productive collection keeps today's cadence; an +unproductive one earns room. The parse-boundary collector re-prices a +pending request so a collection that already satisfied it is not followed by +a second. `PERRY_GC_DIAG=1` gains a `[gc-tiny-parse] forced collection …` +witness line. The arm's own arithmetic is unchanged and now documents why +(pricing it directly was measured at −10.8 % CPU for +22 % footprint, the +issue's refuted branch). + +Validation: `test_memory_json_churn.ts` (the guard's motivating shape) is +byte-identical in output and RSS in all four GC modes; 48/48 `test_gap_gc_*` +and 8/8 `test_gap_json_*` pass; nine new `gc::tests::tiny_parse_pressure` +tests pin the pricing and the predicate, sabotage-proved against both the +old absolute guard and a raw-step pricing. From 6685bdd12f55d163559512723d20735011899564 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sun, 6 Sep 2026 05:52:25 +0000 Subject: [PATCH 3/3] docs(changelog): do not open a line of the #9838 fragment with a bare issue reference --- changelog.d/9838-tiny-parse-pressure-pricing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelog.d/9838-tiny-parse-pressure-pricing.md b/changelog.d/9838-tiny-parse-pressure-pricing.md index dcd96c9b53..1c24003c62 100644 --- a/changelog.d/9838-tiny-parse-pressure-pricing.md +++ b/changelog.d/9838-tiny-parse-pressure-pricing.md @@ -5,7 +5,7 @@ TUI a 3300-character streamed reply spent 30–41 s of CPU in the base arm and post-turn and post-idle RSS flat within the base's own spread and peak RSS unchanged. -#9831 measured the `ArenaBytes` arm firing 51 times in one 66-delta reply, +Issue #9831 measured the `ArenaBytes` arm firing 51 times in one 66-delta reply, each collection freeing a median 131 KB, while the adaptive step sat saturated at 1 GiB — and located the discarded backoff in the arm's own ceiling clamp. That clamp was not what re-fired the arm: between two firings