perf(codegen): land #8589, #8593 and #8597 with cache-key union conflicts resolved

changelog.d/8589-root-spill.md

@@ -0,0 +1,3 @@
+### Changed
+
+- Native GC-root spilling (#8583): a function whose estimated statepoint relocation count — `live_root_slots × safepoints` — exceeds `PERRY_ROOT_SPILL_RELOCATIONS` (default 4,000,000) keeps its GC roots in a heap shadow frame instead of native statepoints. `rewrite-statepoints-for-gc` adds one relocation per live root per safepoint, so a minified-bundle entry function (measured: 795 root slots × ~106k safepoints, grown 439k → 6.5M instructions under RS4GC) drove the `-Os` middle-end super-linear and did not finish; the same unit optimizes in ~5s once that one function is spilled. The function is still compiled at the requested optimization level — only its root representation changes — and its roots stay precise: the runtime already scans shadow-frame and stack-map roots in one walk, and the frame pointer is kept so the FP-chain walker steps over the spilled frame. Each spilled function is reported at default verbosity. `PERRY_ROOT_SPILL_RELOCATIONS=0` disables spilling (every function on native statepoints, the previous behavior).

changelog.d/8593-native-unit-workers.md

@@ -0,0 +1,3 @@
+### Changed
+
+- The default number of concurrent native LLVM codegen-unit workers is now CPU-aware on non-Windows: half the machine's logical CPUs, clamped to `[2, 8]`, instead of a hard-coded `2` (#8583). The `2` default (#8017) was chosen for Windows pagefile pressure and applied everywhere; on a large real bundle (the Claude Code `cli.js` lowers to ~84 units) it left most cores idle while dozens of ~7-minute units ran two at a time. With the giant entry function's roots spilled (#8583) no unit carries an unbounded RS4GC fan-out, so per-unit peak RSS is a bounded ~1-2 GiB and the two-worker cap — not memory — was the wall. Windows keeps the conservative `2`. `PERRY_CODEGEN_UNIT_JOBS` still overrides on every platform.

changelog.d/8597-short-heap-string-equality.md

@@ -0,0 +1,5 @@
+Improved equality checks between statically proven strings by comparing heap-string
+lengths and up to three payload bytes inline before falling back to the full runtime
+helper. This targets the short identifiers used by tree-walking interpreters: retired
+instructions fell 2.41% for `interp` and 1.62% for `iso_miss`, with RSS effectively
+unchanged. Generic-key comparisons retain the smaller existing dispatch.

crates/perry-codegen/src/codegen/closure.rs

@@ -612,6 +612,12 @@ pub(super) fn compile_closure(
         // which spans the body.
         let capture_root_slots =
             u32::from(captures_this || enclosing_class.is_some()) + u32::from(captures_new_target);
+        crate::codegen::helpers::maybe_spill_roots_to_shadow_frame(
+            lf,
+            &llvm_name,
+            m.len() + capture_root_slots as usize,
+            body,
+        );
         lf.enable_shadow_frame(m.len() as u32 + capture_root_slots);
         m
     } else {

crates/perry-codegen/src/codegen/function.rs

@@ -554,6 +554,12 @@ pub(super) fn compile_function(
             cross_module.flat_const_arrays.keys().copied().collect();
         let m =
             crate::collectors::collect_pointer_typed_locals(&f.params, &f.body, &flat_const_ids);
+        crate::codegen::helpers::maybe_spill_roots_to_shadow_frame(
+            lf,
+            &llvm_name,
+            m.len(),
+            &f.body,
+        );
         lf.enable_shadow_frame(m.len() as u32);
         m
     } else {

crates/perry-codegen/src/codegen/helpers.rs

@@ -357,6 +357,74 @@ pub(crate) fn inline_hot_small_max_call_sites() -> u32 {
     })
 }
 
+/// #8583: statepoint relocation estimate above which a function keeps its GC
+/// roots in a shadow frame instead of native statepoints.
+///
+/// `rewrite-statepoints-for-gc` adds one relocation per GC value live across
+/// each safepoint, so the optimizer's post-rewrite cost scales with
+/// `live_roots × safepoints`. Past a point that fan-out makes the `-Os`/`-O3`
+/// middle-end super-linear and the compile does not finish (the Claude Code
+/// bundle's 68 MB entry body measured 795 root slots × ~106k safepoints ≈ 8.4e7
+/// and grew 439k → 6.5M instructions under RS4GC; without RS4GC the same unit
+/// optimized at `-Os` in ~5s). Real functions sit orders of magnitude below
+/// this: hundreds of call sites times tens of slots is ~1e4–1e5. The default
+/// is set well under the measured pathological point and well over ordinary
+/// code, and the post-RS4GC instruction-budget assertion (#8583, inprocess.rs)
+/// backstops any function the estimate misses.
+///
+/// `PERRY_ROOT_SPILL_RELOCATIONS=<n>` overrides it; `0` disables spilling
+/// (every function stays on native statepoints, the pre-#8583 behavior).
+const DEFAULT_ROOT_SPILL_RELOCATIONS: usize = 4_000_000;
+
+fn root_spill_relocation_threshold() -> usize {
+    std::env::var("PERRY_ROOT_SPILL_RELOCATIONS")
+        .ok()
+        .and_then(|v| v.trim().parse::<usize>().ok())
+        .unwrap_or(DEFAULT_ROOT_SPILL_RELOCATIONS)
+}
+
+/// The relocation estimate for a function with `slot_count` GC-root slots and
+/// a body containing `safepoint_sites` call-like expressions. Saturating so a
+/// pathological product cannot wrap.
+pub(crate) fn root_relocation_estimate(slot_count: usize, safepoint_sites: usize) -> usize {
+    slot_count.saturating_mul(safepoint_sites)
+}
+
+/// Decide whether `func` should spill its roots to the shadow frame, and if so
+/// mark it (BEFORE its `enable_*_shadow_frame` call) and report it. Only
+/// meaningful under native stack-map roots — the shadow frame is already the
+/// lowering otherwise. Reporting is at default verbosity because #8421 requires
+/// that a change to how a function is compiled is never silent; the message
+/// states that the optimization level is unchanged.
+pub(super) fn maybe_spill_roots_to_shadow_frame(
+    func: &mut crate::function::LlFunction,
+    fn_name: &str,
+    slot_count: usize,
+    body: &[perry_hir::Stmt],
+) {
+    if !native_stack_roots_enabled() {
+        return;
+    }
+    let threshold = root_spill_relocation_threshold();
+    if threshold == 0 {
+        return;
+    }
+    let sites = crate::collectors::count_safepoint_sites(body);
+    let estimate = root_relocation_estimate(slot_count, sites);
+    if estimate <= threshold {
+        return;
+    }
+    func.request_shadow_frame_spill();
+    eprintln!(
+        "perry: `{fn_name}` keeps its {slot_count} GC roots in a shadow frame instead of \
+         statepoints: an estimated {estimate} relocations ({slot_count} roots × {sites} \
+         safepoints) would make rewrite-statepoints-for-gc fan-out super-linear in the \
+         optimizer (> {threshold}). The function is still compiled at the requested \
+         optimization level; only its GC-root representation changes, and its roots stay \
+         precise (#8583). Override with PERRY_ROOT_SPILL_RELOCATIONS."
+    );
+}
+
 pub(super) fn enable_module_init_shadow_frame(
     func: &mut crate::function::LlFunction,
     stmts: &[perry_hir::Stmt],
@@ -368,6 +436,10 @@ pub(super) fn enable_module_init_shadow_frame(
 
     let shadow_slot_map =
         crate::collectors::collect_pointer_typed_locals(&[], stmts, flat_const_ids);
+    // #8583: the module-entry body is the minified-bundle IIFE — the function
+    // that fans out catastrophically under RS4GC. Decide its root lowering
+    // before the frame is built.
+    maybe_spill_roots_to_shadow_frame(func, "main", shadow_slot_map.len(), stmts);
     func.enable_post_init_shadow_frame(shadow_slot_map.len() as u32);
     let shadow_slot_clears_after_stmt =
         crate::collectors::collect_shadow_slot_clear_points(stmts, &shadow_slot_map);

crates/perry-codegen/src/codegen/method.rs

@@ -322,6 +322,12 @@ pub(super) fn compile_method(
             &method.body,
             &flat_const_ids,
         );
+        crate::codegen::helpers::maybe_spill_roots_to_shadow_frame(
+            lf,
+            &llvm_name,
+            m.len() + 1,
+            &method.body,
+        );
         lf.enable_shadow_frame(m.len() as u32 + 1);
         m
     } else {
@@ -1395,6 +1401,12 @@ pub(super) fn compile_static_method(
             cross_module.flat_const_arrays.keys().copied().collect();
         let m =
             crate::collectors::collect_pointer_typed_locals(&f.params, &f.body, &flat_const_ids);
+        crate::codegen::helpers::maybe_spill_roots_to_shadow_frame(
+            lf,
+            &llvm_name,
+            m.len() + 1,
+            &f.body,
+        );
         lf.enable_shadow_frame(m.len() as u32 + 1);
         m
     } else {

crates/perry-codegen/src/collectors/mod.rs

@@ -42,6 +42,7 @@ mod ptr_shape_report;
 mod ptr_shape_returns;
 mod refs;
 mod repsel_benefit;
+mod safepoint_sites;
 mod scalar_method_dispatch;
 mod scalar_methods;
 mod shadow_slots;
@@ -93,6 +94,7 @@ pub(crate) use ptr_shape_returns::collect_exported_return_shapes;
 pub(crate) use refs::{
     collect_let_ids, collect_ref_ids_in_expr, collect_ref_ids_in_stmts, is_clamp_call,
 };
+pub(crate) use safepoint_sites::count_safepoint_sites;
 pub(crate) use scalar_method_dispatch::{
     collect_module_dispatch_facts, mark_unstable_scalar_method_receivers, ModuleDispatchFacts,
 };

crates/perry-codegen/src/collectors/safepoint_sites.rs

@@ -0,0 +1,219 @@
+//! Count the GC safepoints in a function body (#8583).
+//!
+//! `rewrite-statepoints-for-gc` inserts, at every safepoint, one relocation
+//! per GC value live across it — so the optimizer's post-rewrite work grows
+//! with `live_roots × safepoints`. A function whose product is large enough
+//! makes the `-Os`/`-O3` middle-end super-linear: the 68 MB minified entry
+//! body of the Claude Code bundle measured 795 root slots × ~106k safepoints
+//! and grew 439k → 6.5M instructions under RS4GC, and a single `-Os` pass on
+//! the result did not finish in practical time (#8583).
+//! `codegen/helpers::maybe_spill_roots_to_shadow_frame` multiplies this count
+//! by the function's root-slot count and, past a threshold, keeps that
+//! function's roots in a shadow frame instead of statepoints.
+//!
+//! A safepoint is any call-like expression: a call can re-enter the runtime
+//! and collect. The count is an over-approximation biased toward spilling —
+//! a false positive is a shadow frame on a function that would have been fine
+//! (cheap; the shadow lowering is the pre-#7370 default), while a false
+//! negative would let relocation fan-out reach the optimizer. Nested closures
+//! are NOT counted: each compiles to its own `LlFunction` with its own frame,
+//! so its safepoints belong to it (`walk_expr_children` does not descend into
+//! a closure's body, only its parameter defaults).
+
+use perry_hir::{Expr, Stmt};
+
+/// Total call-like expressions reachable from `stmts` without descending into
+/// nested closures.
+pub fn count_safepoint_sites(stmts: &[Stmt]) -> usize {
+    let mut n = 0usize;
+    for s in stmts {
+        count_in_stmt(s, &mut n);
+    }
+    n
+}
+
+/// A call-like expression is a potential safepoint: anything whose lowering
+/// emits a call that can re-enter the runtime. Nodes not listed contribute
+/// nothing themselves but are still recursed into, so adding a new call
+/// variant can only make the estimate more conservative (a possible
+/// under-count that the post-RS4GC instruction-budget assertion backstops),
+/// never wrong in a way that hides a fan-out.
+fn is_safepoint(e: &Expr) -> bool {
+    matches!(
+        e,
+        Expr::Call { .. }
+            | Expr::CallSpread { .. }
+            | Expr::NativeMethodCall { .. }
+            | Expr::StaticMethodCall { .. }
+            | Expr::SuperCall(_)
+            | Expr::SuperCallSpread(_)
+            | Expr::SuperMethodCall { .. }
+            | Expr::SuperMethodCallSpread { .. }
+            | Expr::ObjectSuperMethodCall { .. }
+            | Expr::New { .. }
+            | Expr::NewDynamic { .. }
+            | Expr::NewDynamicSpread { .. }
+            | Expr::Await(_)
+            | Expr::Yield { .. }
+            | Expr::AsyncFirstCall { .. }
+    )
+}
+
+fn count_in_expr(e: &Expr, n: &mut usize) {
+    if is_safepoint(e) {
+        *n += 1;
+    }
+    // Generic recursion into direct sub-expressions. `walk_expr_children` does
+    // not descend into a closure's statement body (only its param defaults),
+    // which is exactly the boundary we want: a nested closure is a separate
+    // frame and its safepoints are not this function's.
+    perry_hir::walker::walk_expr_children(e, &mut |child| count_in_expr(child, n));
+}
+
+fn count_in_stmt(s: &Stmt, n: &mut usize) {
+    match s {
+        Stmt::Let { init: Some(e), .. }
+        | Stmt::Expr(e)
+        | Stmt::Throw(e)
+        | Stmt::Return(Some(e)) => count_in_expr(e, n),
+        Stmt::Let { init: None, .. } | Stmt::Return(None) => {}
+        Stmt::If {
+            condition,
+            then_branch,
+            else_branch,
+        } => {
+            count_in_expr(condition, n);
+            for st in then_branch {
+                count_in_stmt(st, n);
+            }
+            if let Some(else_branch) = else_branch {
+                for st in else_branch {
+                    count_in_stmt(st, n);
+                }
+            }
+        }
+        Stmt::While { condition, body } | Stmt::DoWhile { body, condition } => {
+            count_in_expr(condition, n);
+            for st in body {
+                count_in_stmt(st, n);
+            }
+        }
+        Stmt::For {
+            init,
+            condition,
+            update,
+            body,
+        } => {
+            if let Some(init) = init {
+                count_in_stmt(init, n);
+            }
+            if let Some(condition) = condition {
+                count_in_expr(condition, n);
+            }
+            if let Some(update) = update {
+                count_in_expr(update, n);
+            }
+            for st in body {
+                count_in_stmt(st, n);
+            }
+        }
+        Stmt::Labeled { body, .. } => count_in_stmt(body, n),
+        Stmt::Try {
+            body,
+            catch,
+            finally,
+        } => {
+            for st in body {
+                count_in_stmt(st, n);
+            }
+            if let Some(catch) = catch {
+                for st in &catch.body {
+                    count_in_stmt(st, n);
+                }
+            }
+            if let Some(finally) = finally {
+                for st in finally {
+                    count_in_stmt(st, n);
+                }
+            }
+        }
+        Stmt::Switch {
+            discriminant,
+            cases,
+        } => {
+            count_in_expr(discriminant, n);
+            for c in cases {
+                if let Some(t) = &c.test {
+                    count_in_expr(t, n);
+                }
+                for st in &c.body {
+                    count_in_stmt(st, n);
+                }
+            }
+        }
+        // No expression children.
+        Stmt::Break
+        | Stmt::Continue
+        | Stmt::LabeledBreak(_)
+        | Stmt::LabeledContinue(_)
+        | Stmt::PreallocateBoxes(_)
+        | Stmt::PreallocateTdzBoxes(_)
+        | Stmt::ReleaseBoxes(_) => {}
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::count_safepoint_sites;
+    use perry_hir::types::Type;
+    use perry_hir::{Expr, Stmt};
+
+    fn call(args: Vec<Expr>) -> Expr {
+        Expr::Call {
+            callee: Box::new(Expr::Undefined),
+            args,
+            type_args: vec![],
+            byte_offset: 0,
+        }
+    }
+
+    fn empty_closure(body: Vec<Stmt>) -> Expr {
+        Expr::Closure {
+            func_id: 0,
+            params: vec![],
+            return_type: Type::Any,
+            body,
+            captures: vec![],
+            mutable_captures: vec![],
+            captures_this: false,
+            captures_new_target: false,
+            enclosing_class: None,
+            is_arrow: false,
+            is_async: false,
+            is_generator: false,
+            is_strict: false,
+        }
+    }
+
+    #[test]
+    fn counts_calls_across_control_flow_but_not_into_closures() {
+        let stmts = vec![
+            Stmt::Expr(call(vec![])),
+            Stmt::While {
+                condition: Expr::Bool(true),
+                body: vec![Stmt::Expr(call(vec![]))],
+            },
+            // A call buried in a nested closure body must NOT be counted.
+            Stmt::Expr(empty_closure(vec![Stmt::Expr(call(vec![]))])),
+            Stmt::Return(Some(call(vec![]))),
+        ];
+        assert_eq!(count_safepoint_sites(&stmts), 3);
+    }
+
+    #[test]
+    fn call_arguments_are_themselves_safepoints() {
+        // f(g(), h()) is three calls.
+        let nested = call(vec![call(vec![]), call(vec![])]);
+        assert_eq!(count_safepoint_sites(&[Stmt::Expr(nested)]), 3);
+    }
+}