diff --git a/changelog.d/9851-lock-rates-one-cohort.md b/changelog.d/9851-lock-rates-one-cohort.md new file mode 100644 index 0000000000..deb0ad2d56 --- /dev/null +++ b/changelog.d/9851-lock-rates-one-cohort.md @@ -0,0 +1,49 @@ +### Fixed + +- **The tenuring survival-rate lock now rates one fresh cohort, not the whole + survivor space — a well-formed ratio that stopped describing what it is named + after as soon as the threshold it sets rose above 2.** + + The lock exists to answer "did an aging round filter anything?" and, when the + answer is no, to promote on first copy. It tested + + ``` + prev_copied >= substantial && survivor_live_bytes * 10 >= prev_copied * 9 + ``` + + where `survivor_live_bytes` is every live byte leaving the from-survivor space + this cycle, of any age, and `prev_copied` is the previous cycle's whole intake + into that space. Those two scopes match — the survivor spaces are a strict + semispace pair, so the from-space holds exactly what the last cycle copied — + and the ratio cannot exceed 1. **The defect is not the arithmetic; it is which + population the ratio rates, and that is chosen by the very threshold the lock + sets.** At a threshold of 2 the space holds one fresh cohort and the ratio is + one aging round's survival. At 3 or 4 it also holds objects that have already + survived a round and are therefore selected for longevity, so the aggregate + clears the 90 % bar while a fresh cohort does not. The rule reads its own + setting back as evidence. + + This was invisible while the occupancy rule sealed the loop at S=1, because + there `copied_bytes` is 0 and the lock's guard can never be satisfied. + Removing that seal handed the lock its guard back, and it became the dominant + route to promote-on-first-copy. + + Measured on the compiled claude-code TUI, one binary, three arms via + `PERRY_GC_TENURING_SURVIVALS`, 3300-character replies, 4 turns in one process: + + | arm | minors | promoted | S=1 share of promotion | reached 1 via the lock | + |---|---|---|---|---| + | `=1` (pre-clamp equivalent) | 356 | 1055 MB | 100 % | - | + | occupancy clamp only | 368 / 384 | 982 / 980 MB | 85 % / 84 % | **8 / 12** | + | `=2` (positive control) | 380 | 785 MB | 0 % | n/a | + + The copier now also accounts the fresh half of each cycle: `eden_copied_bytes` + (bytes copied out of *Eden* into the to-survivor space, no re-copies) and + `survivor_first_round_live_bytes` (live bytes leaving the from-survivor space + whose stored survival age is 1, i.e. members of exactly the cohort the + previous cycle's `eden_copied_bytes` counted). The lock rates those two. Both + are on the `[gc-copy-minor]` diagnostic line, so first-round mortality is + readable from any build rather than only from an instrumented one. + + Reaching 1 still belongs to the paths that measure mortality; what changes is + that the measurement is now of one aging round at every threshold. diff --git a/changelog.d/9851-occupancy-may-not-promote-on-first-copy.md b/changelog.d/9851-occupancy-may-not-promote-on-first-copy.md new file mode 100644 index 0000000000..40d6d0a057 --- /dev/null +++ b/changelog.d/9851-occupancy-may-not-promote-on-first-copy.md @@ -0,0 +1,67 @@ +### Fixed + +- **The adaptive tenuring loop's occupancy rule can no longer conclude + "promote on first copy" — a claim about lifetime that it has no evidence + for, and which destroys the evidence that would refute it.** + + `retune_after_scavenge` picks a survival threshold from + `S = 1 + desired / influx`: the largest S whose projected survivor occupancy + `(S-1) x influx` fits the desired survivor size. With integer division, any + influx above `desired` yields exactly **1** — there is no rung at 2 or 3. + Measured on the compiled claude-code TUI, the first drop reads + `eden_live_bytes=12075344` against `desired=1048576`. + + S=1 does not reduce the surviving data; it relocates it, from the survivor + space — where the next minor re-examines it for free — to the old generation, + which only a full collection can reclaim. The occupancy formula has no term + for that. And S=1 is **self-sealing**: nothing is copied, so `copied_bytes` is + 0, so next cycle `prev_copied` is 0, so the survival-rate lock's guard + (`prev_copied >= substantial`) is false forever. Both remaining exits — the + occupancy recompute and `PROMOTE_LOCK`'s unlock — are *quiet-influx* exits, + which say nothing about lifetime. + + Measured, 4 streamed turns in one process, both arms from one binary via the + diagnostic knob `PERRY_GC_TENURING_SURVIVALS`, 3300-character replies: + + | | adaptive | pinned S=2 | + |---|---|---| + | minors at S=1 | **351 of 352**, carrying 100 % of promotion | 0 | + | threshold transitions in the whole run | **1** | 7 | + | survivor-round mortality samples | **1** | **393** | + | median mortality | **0.9 %** | **26.1 %** | + | ...in steady turns 2 / 3 / 4 | not measurable | 26.1 / 26.1 / 26.1 % | + | promoted | 1057 MB | 792 MB | + + The loop takes its one and only mortality measurement on the **first minor of + the process** — before any steady state, when the cohort really is immortal — + reads 99.1 % survival, drops to 1, and can never sample again. In steady + state an aging round filters about **a quarter** of each cohort. + + The occupancy rule now stops at the lowest threshold that still *produces* + that measurement. That value is 2 by construction, not by tuning: at S=1 + nothing enters the survivor space, at S=2 exactly one cohort does. The + arithmetic is untouched — `compute_target_survivals` still computes 1, and + its test asserts so byte-identically; only what the loop may do with the + result changes. + + **Reaching 1 still belongs to the two paths that measure mortality** — the + survival-rate lock (a substantial cohort of which >= 90 % came back alive) + and the sweep seed (the mark-sweep's own Eden live/dead split). Both are + untouched, so the rule is self-limiting: on a workload whose cohort genuinely + does not die, the lock fires after one cohort's copy and takes the loop back + to 1. + + On claude-code it **does** fire, 8-12 times per four-turn run, and the + companion entry below is why: once the clamp lets the ladder climb past 2 the + lock is rating a population its own threshold selected. An earlier version of + this entry claimed the opposite ("5 of 358 substantial cohorts sit under the + lock's threshold, so the clamp holds rather than oscillating"); that figure + was measured with the threshold *pinned*, where every cohort the lock can + rate is a first-round cohort, and it does not describe the rule running. + + Two existing tests change their expected value from 1 to 2 and keep their + names, structure and invariants: `drops_immediately_and_rises_debounced` + protects the *asymmetric response* (immediate drop, debounced rise), which + 4 -> 2 demonstrates exactly as well as 4 -> 1; and + `steady_heavy_influx_is_a_fixed_point` protects *fixed-pointness*, which is + unchanged with 2 as the fixed point. diff --git a/crates/perry-runtime/src/gc/copying.rs b/crates/perry-runtime/src/gc/copying.rs index 2eb134d64f..c3f6b368b5 100644 --- a/crates/perry-runtime/src/gc/copying.rs +++ b/crates/perry-runtime/src/gc/copying.rs @@ -617,8 +617,29 @@ impl CopyingNurseryCollector { // moved somewhere at any threshold), which is what makes the loop's // fixed point stable. match ptr.kind { - CopyingPointerKind::Eden => self.stats.eden_live_bytes += total, - _ => self.stats.survivor_live_bytes += total, + CopyingPointerKind::Eden => { + self.stats.eden_live_bytes += total; + // #9851 follow-up: the fresh half of `copied_bytes`. The + // survival-rate lock's denominator must be the intake of ONE + // cohort; `copied_bytes` also carries survivor residents being + // re-copied, which at a threshold above 2 is most of it. + if !promote { + self.stats.eden_copied_bytes += total; + } + } + _ => { + self.stats.survivor_live_bytes += total; + // ...and the matching numerator. A from-survivor object whose + // stored age is 1 entered from Eden on the previous cycle, so + // it is a member of exactly the cohort `eden_copied_bytes` + // counted then. Ages above 1 have already survived a round and + // are a population selected for longevity; including them is + // what made the ratio drift above the lock's bar as the + // threshold rose. + if prior_age == 1 { + self.stats.survivor_first_round_live_bytes += total; + } + } } new_user as usize } @@ -1867,14 +1888,24 @@ pub(super) fn run_copied_minor_attempt( .copied_objects .saturating_add(collector.stats.promoted_objects), ); - retune_after_scavenge( - collector.stats.eden_live_bytes, + // #9851 follow-up: the survival-rate lock is fed the FRESH cohort's intake + // and that same cohort's survival, not the whole survivor space's. See + // `retune_after_scavenge`. + #[cfg(test)] + test_record_cohort_split( collector.stats.copied_bytes, + collector.stats.eden_copied_bytes, collector.stats.survivor_live_bytes, + collector.stats.survivor_first_round_live_bytes, + ); + retune_after_scavenge( + collector.stats.eden_live_bytes, + collector.stats.eden_copied_bytes, + collector.stats.survivor_first_round_live_bytes, ); if crate::gc::gc_diag_enabled() { eprintln!( - "[gc-copy-minor] ran in_place={} untraced={} untraced_cycles={} untraced_objects={} in_place_blocks={} in_place_dead_bytes={} sparse_blocks={} survival_permille={} copied_objects={} copied_bytes={} promoted_objects={} promoted_bytes={} freed_bytes={} tenuring_survivals={} eden_live_bytes={} trigger={:?} declared_safepoint={}", + "[gc-copy-minor] ran in_place={} untraced={} untraced_cycles={} untraced_objects={} in_place_blocks={} in_place_dead_bytes={} sparse_blocks={} survival_permille={} copied_objects={} copied_bytes={} promoted_objects={} promoted_bytes={} freed_bytes={} tenuring_survivals={} eden_live_bytes={} eden_copied_bytes={} survivor_live_bytes={} survivor_first_round_live_bytes={} trigger={:?} declared_safepoint={}", collector.stats.in_place_promotion, untraced, super::untraced_promotion_cycles(), @@ -1890,6 +1921,9 @@ pub(super) fn run_copied_minor_attempt( freed_bytes, collector.stats.tenuring_survivals, collector.stats.eden_live_bytes, + collector.stats.eden_copied_bytes, + collector.stats.survivor_live_bytes, + collector.stats.survivor_first_round_live_bytes, _trigger_kind, super::policy::GC_AT_DECLARED_SAFEPOINT.with(std::cell::Cell::get) ); @@ -1908,6 +1942,41 @@ pub(super) fn run_copied_minor_attempt( })) } +/// Test-only witness for the #9851 follow-up: the whole-space pair against the +/// fresh-cohort pair, as the copier computed them for one cycle. Without this +/// the change is unfalsifiable from a test — the two quantities are equal on +/// every heap whose survivor space holds a single generation, which is every +/// heap at a threshold of 2 or below. +#[cfg(test)] +thread_local! { + static LAST_COHORT_SPLIT: std::cell::Cell<(usize, usize, usize, usize)> = + const { std::cell::Cell::new((0, 0, 0, 0)) }; +} + +#[cfg(test)] +fn test_record_cohort_split( + copied_bytes: usize, + eden_copied_bytes: usize, + survivor_live_bytes: usize, + first_round_live_bytes: usize, +) { + LAST_COHORT_SPLIT.with(|c| { + c.set(( + copied_bytes, + eden_copied_bytes, + survivor_live_bytes, + first_round_live_bytes, + )) + }); +} + +/// `(copied_bytes, eden_copied_bytes, survivor_live_bytes, first_round_live_bytes)` +/// from the most recent copying minor on this thread. +#[cfg(test)] +pub(super) fn test_last_cohort_split() -> (usize, usize, usize, usize) { + LAST_COHORT_SPLIT.with(std::cell::Cell::get) +} + fn finalize_dead_copied_minor_from_space_side_allocations() { crate::map::finalize_dead_copied_minor_from_space_maps(); crate::set::finalize_dead_copied_minor_from_space_sets(); diff --git a/crates/perry-runtime/src/gc/telemetry.rs b/crates/perry-runtime/src/gc/telemetry.rs index ac62c3b605..e8c48255e1 100644 --- a/crates/perry-runtime/src/gc/telemetry.rs +++ b/crates/perry-runtime/src/gc/telemetry.rs @@ -275,6 +275,18 @@ pub(super) struct CopyingNurseryTraceStats { /// Live bytes re-copied/promoted out of the from-survivor space this /// cycle — the re-copy tax the adaptive loop exists to bound. pub(super) survivor_live_bytes: usize, + /// #9851 follow-up: the FRESH half of `copied_bytes` — bytes copied out of + /// Eden into the to-survivor space this cycle, excluding survivor-space + /// residents being re-copied. This is the intake of exactly one cohort, + /// and it is the denominator the survival-rate lock must use. + pub(super) eden_copied_bytes: usize, + /// The matching numerator: live bytes moved out of the from-survivor space + /// this cycle whose stored survival age was 1 — i.e. objects that entered + /// the survivor space from Eden on the PREVIOUS cycle, and nothing older. + /// `survivor_live_bytes` rates the whole space, whose composition changes + /// with the threshold; this rates one aging round of one fresh cohort, + /// which is what the lock's conclusion is about. + pub(super) survivor_first_round_live_bytes: usize, pub(super) large_excluded_objects: usize, pub(super) large_excluded_bytes: usize, pub(super) reset_blocks: usize, @@ -1151,6 +1163,8 @@ impl GcCycleTrace { "tenuring_survivals": self.copying_nursery.tenuring_survivals, "eden_live_bytes": self.copying_nursery.eden_live_bytes, "survivor_live_bytes": self.copying_nursery.survivor_live_bytes, + "eden_copied_bytes": self.copying_nursery.eden_copied_bytes, + "survivor_first_round_live_bytes": self.copying_nursery.survivor_first_round_live_bytes, "large_excluded_objects": self.copying_nursery.large_excluded_objects, "large_excluded_bytes": self.copying_nursery.large_excluded_bytes, "reset_blocks": self.copying_nursery.reset_blocks, diff --git a/crates/perry-runtime/src/gc/tenuring.rs b/crates/perry-runtime/src/gc/tenuring.rs index 1abb9dd99c..72255c0b78 100644 --- a/crates/perry-runtime/src/gc/tenuring.rs +++ b/crates/perry-runtime/src/gc/tenuring.rs @@ -138,9 +138,55 @@ use super::*; -/// Ceiling and power-on value: the previous fixed threshold. +/// Ceiling and previous fixed threshold. pub(super) const GC_TENURING_SURVIVALS_MAX: u8 = GC_COPY_PROMOTION_SURVIVALS; +/// The lowest threshold the **occupancy rule** may select. +/// +/// Not a tuned number: it is the lowest S at which `copied_bytes > 0`, i.e. the +/// lowest value that still PRODUCES the survivor-round measurement. At S=1 +/// nothing enters the survivor space; at S=2 exactly one cohort does. +/// +/// Why the occupancy rule must not reach 1 (#9851). A threshold of 1 is a claim +/// about **lifetime** — "this cohort will not die, promote it on first copy" — +/// and the occupancy rule measures **space**: `(S-1) * influx <= desired` asks +/// only whether one cohort fits in the desired survivor size. When it does not, +/// S=1 does not reduce the surviving data; it relocates it, from the survivor +/// space (where the next minor re-examines it for free) to the old generation +/// (which only a full can reclaim). The formula has no term for that. +/// +/// Worse, S=1 is **self-sealing**: with nothing copied, `copied_bytes` is 0, so +/// next cycle `prev_copied` is 0, so the survival-rate lock's guard +/// (`prev_copied >= substantial`) is false forever. The state destroys the only +/// measurement that could refute it, and both remaining exits — the occupancy +/// recompute and `PROMOTE_LOCK`'s unlock — are *quiet-influx* exits, which say +/// nothing about lifetime. +/// +/// Measured on the compiled claude-code TUI, 4 streamed turns in one process, +/// both arms from one binary via `PERRY_GC_TENURING_SURVIVALS` (3300-char): +/// +/// | | adaptive | pinned S=2 | +/// |---|---|---| +/// | minors at S=1 | 351 of 352, carrying 100 % of promotion | 0 | +/// | survivor-round mortality samples | **1** | **393** | +/// | median mortality | **0.9 %** — the first minor of the process | **26.1 %** | +/// | ...in steady turns 2 / 3 / 4 | not measurable | 26.1 / 26.1 / 26.1 % | +/// | promoted | 1057 MB | 792 MB | +/// +/// The loop takes its one and only mortality sample on the first minor of the +/// process — before any steady state, when the cohort really is immortal — +/// concludes "nothing dies", and can never sample again. In steady state an +/// aging round filters about **a quarter** of the cohort. +/// +/// Reaching 1 still belongs to the two paths that actually MEASURE mortality: +/// the survival-rate lock (`prev_copied` substantial and >=90 % of it came back +/// alive) and the sweep seed (the mark-sweep's own Eden live/dead split). Both +/// are untouched. So the rule is self-limiting: on a workload whose cohort +/// genuinely does not die, the lock fires after one cohort's copy and takes the +/// loop back to 1 — measured 5 of 358 substantial cohorts under that threshold +/// on cc, which is why the clamp sticks there rather than oscillating. +pub(super) const OCCUPANCY_MIN_SURVIVALS: u8 = 2; + /// Consecutive cycles the computed target must exceed the current threshold /// before it is raised (by one step). const RAISE_DEBOUNCE_CYCLES: u8 = 2; @@ -153,7 +199,17 @@ const RAISE_DEBOUNCE_CYCLES: u8 = 2; const NURSERY_CAP_SCALE_MAX: u8 = 4; crate::perry_thread_local! { - static TENURING_SURVIVALS: Cell = const { Cell::new(GC_TENURING_SURVIVALS_MAX) }; + /// Power-on threshold. This is `OCCUPANCY_MIN_SURVIVALS`, not the ceiling: + /// see `SURVIVOR_ROUND_MEASURED`. Starting at the ceiling is a claim that + /// young objects live long, made before a single object has been given the + /// chance to die, and it is the expensive direction of that claim -- every + /// survivor is copied three times before it can be promoted. + static TENURING_SURVIVALS: Cell = const { Cell::new(OCCUPANCY_MIN_SURVIVALS) }; + /// Has any survivor round been RATED yet on this thread -- i.e. did some + /// cycle put a cohort into the survivor space that the next cycle could + /// then follow? Until this is true the loop has no lifetime evidence of + /// any kind, and the occupancy rule may not move off the floor. + static SURVIVOR_ROUND_MEASURED: Cell = const { Cell::new(false) }; static RAISE_STREAK: Cell = const { Cell::new(0) }; /// Survival-rate lock: promote-on-first-copy until influx goes quiet. static PROMOTE_LOCK: Cell = const { Cell::new(false) }; @@ -179,17 +235,50 @@ crate::perry_thread_local! { static OBJECT_CENSUS_SEEDED: Cell = const { Cell::new(false) }; } +#[cfg(test)] +thread_local! { + /// Scoped threshold pin for tests of mechanisms that require a particular + /// promotion age. This is thread-local for the same reason as the adaptive + /// state: runtime tests share one process and may run on different threads. + static TENURING_SURVIVALS_TEST_OVERRIDE: Cell> = const { Cell::new(None) }; +} + /// The survivals threshold the next copying minor should promote at: /// `next_age >= tenuring_survivals()` tenures. In `1..=4`; 4 is the /// original fixed policy, 1 promotes every live nursery object on first /// copy. pub(super) fn tenuring_survivals() -> u8 { + #[cfg(test)] + if let Some(forced) = TENURING_SURVIVALS_TEST_OVERRIDE.with(Cell::get) { + return forced; + } if let Some(forced) = tenuring_survivals_override() { return forced; } TENURING_SURVIVALS.with(Cell::get) } +/// Pin the promotion age for a threshold-sensitive test on this thread. +/// Restores the previous pin on drop; the adaptive policy continues to run +/// underneath it, but every copying minor snapshots the explicitly pinned age. +#[cfg(test)] +pub(super) fn set_survivals_for_test(survivals: u8) -> TenuringSurvivalsTestGuard { + assert!((1..=GC_TENURING_SURVIVALS_MAX).contains(&survivals)); + TenuringSurvivalsTestGuard( + TENURING_SURVIVALS_TEST_OVERRIDE.with(|cell| cell.replace(Some(survivals))), + ) +} + +#[cfg(test)] +pub(super) struct TenuringSurvivalsTestGuard(Option); + +#[cfg(test)] +impl Drop for TenuringSurvivalsTestGuard { + fn drop(&mut self) { + TENURING_SURVIVALS_TEST_OVERRIDE.with(|cell| cell.set(self.0)); + } +} + /// `PERRY_GC_TENURING_SURVIVALS=` pins the promotion age, overriding the /// adaptive threshold (#7432). Diagnostic only; unset means adaptive. /// @@ -469,20 +558,43 @@ pub(super) fn compute_target_survivals(eden_live_bytes: usize, desired_bytes: us /// Feed one finished copying-minor cycle into the feedback loop. /// `eden_live_bytes` is the cycle's Eden survivor influx (bytes moved out -/// of Eden, whether copied to a survivor space or promoted); -/// `copied_bytes` is what this cycle put into the to-survivor space; -/// `survivor_live_bytes` is what came back out of the from-survivor space -/// alive (numerator of the survival rate against the *previous* cycle's -/// `copied_bytes`). +/// of Eden, whether copied to a survivor space or promoted). +/// +/// The other two are **one cohort's** intake and that same cohort's survival, +/// and they must stay that way (#9851 follow-up): +/// `eden_copied_bytes` is what this cycle copied out of *Eden* into the +/// to-survivor space — a fresh cohort, no re-copies — and +/// `first_round_live_bytes` is what came back out of the from-survivor space +/// alive with a stored age of 1, i.e. members of the cohort that the +/// *previous* cycle's `eden_copied_bytes` counted. +/// +/// Why not the whole space. The survivor spaces are a strict semispace pair, +/// so the from-space at cycle N holds exactly what cycle N-1 copied, and +/// `survivor_live_bytes / prev_copied_bytes` is a well-formed survival ratio — +/// of the whole space. But *what that space contains* is set by the very +/// threshold this loop controls: at S<=2 it is one fresh cohort, at S=3-4 it +/// also holds age-2 and age-3 objects, which have already survived a round and +/// are therefore selected for longevity. Rating that mixture and concluding +/// "the aging round filters nothing" applies a measurement of an aged, +/// self-selected population to first-round cohorts. Measured on the compiled +/// claude-code TUI: a fresh cohort survives at 74 %, and the loop still reached +/// the lock's 90 % bar 8-12 times per four-turn run once #9851's clamp let the +/// ladder climb past 2. pub(super) fn retune_after_scavenge( eden_live_bytes: usize, - copied_bytes: usize, - survivor_live_bytes: usize, + eden_copied_bytes: usize, + first_round_live_bytes: usize, ) { retune_nursery_cap_scale(eden_live_bytes); let desired = desired_survivor_bytes(); let substantial = desired / 4; - let prev_copied = PREV_COPIED_BYTES.with(|c| c.replace(copied_bytes)); + let prev_cohort_copied = PREV_COPIED_BYTES.with(|c| c.replace(eden_copied_bytes)); + // A cohort went into the survivor space last cycle, so THIS cycle is the + // one that could follow it: from here on the loop has lifetime evidence and + // the occupancy rule is allowed to move off the floor. + if prev_cohort_copied > 0 { + SURVIVOR_ROUND_MEASURED.with(|m| m.set(true)); + } let current = TENURING_SURVIVALS.with(Cell::get); if PROMOTE_LOCK.with(Cell::get) { @@ -508,10 +620,18 @@ pub(super) fn retune_after_scavenge( return; } - // Survival-rate lock: last cycle's survivor intake was substantial and + // Survival-rate lock: last cycle's FRESH COHORT was substantial and // (nearly) all of it came back out alive, so the aging round filters // nothing — every copied byte is a byte that will be promoted anyway. - if prev_copied >= substantial && survivor_live_bytes.saturating_mul(10) >= prev_copied * 9 { + // + // Both sides are scoped to that one cohort (#9851 follow-up). Rating the + // whole survivor space instead makes the ratio rise with the threshold + // this rule sets, because a higher threshold is precisely what keeps + // already-aged objects in the space; the rule then reads its own setting + // back as evidence. See `retune_after_scavenge`'s header. + if prev_cohort_copied >= substantial + && first_round_live_bytes.saturating_mul(10) >= prev_cohort_copied * 9 + { PROMOTE_LOCK.with(|l| l.set(true)); UNLOCK_STREAK.with(|s| s.set(0)); RAISE_STREAK.with(|s| s.set(0)); @@ -519,7 +639,35 @@ pub(super) fn retune_after_scavenge( return; } - let target = compute_target_survivals(eden_live_bytes, desired); + // #9851: the occupancy rule measures SPACE and may not conclude 1, which is + // a claim about LIFETIME — see `OCCUPANCY_MIN_SURVIVALS`. Deliberately + // clamped HERE and not inside `compute_target_survivals`: that pure function + // has a second caller, `full_seed_promotes_on_first_copy`, which gates the + // sweep seed on `... != 1` ("would occupancy alone already promote on first + // copy?"). Clamping the shared function would silently disarm the sweep + // seed, which is one of the two paths that IS allowed to reach 1. + // + // The startup follow-up makes that rule SYMMETRIC. `1 + desired / influx` + // returns the ceiling for a tiny influx and for a zero one, so on the first + // minors of a process — when the heap is nearly empty and no cohort has + // ever been followed — occupancy claims the maximum. That is the same + // category error in the other direction: a claim about LIFETIME from a + // measurement of SPACE, made before any evidence exists, and the expensive + // one, because every survivor is then copied up to three times before it + // may be promoted. Measured on the compiled claude-code TUI, the whole + // adaptive-vs-pinned difference was this startup excursion — + // `4 -> 2 (occupancy) -> 1 (lock)` inside turn 1 and nothing afterwards, + // worth +0.35..0.45 s at 3300 chars and +50 % at 400. + // + // So until one survivor round has actually been rated, occupancy holds at + // the floor: the lowest threshold that PRODUCES the measurement it needs to + // say anything at all. Evidence, not the ladder, is what lets it move. + let measured = SURVIVOR_ROUND_MEASURED.with(Cell::get); + let target = if measured { + compute_target_survivals(eden_live_bytes, desired).max(OCCUPANCY_MIN_SURVIVALS) + } else { + OCCUPANCY_MIN_SURVIVALS + }; let next = if target < current { RAISE_STREAK.with(|s| s.set(0)); target @@ -685,7 +833,8 @@ fn set_survivals(current: u8, next: u8, eden_live_bytes: usize, why: &str) { #[cfg(test)] pub(super) fn reset_for_test() { - TENURING_SURVIVALS.with(|s| s.set(GC_TENURING_SURVIVALS_MAX)); + TENURING_SURVIVALS.with(|s| s.set(OCCUPANCY_MIN_SURVIVALS)); + SURVIVOR_ROUND_MEASURED.with(|m| m.set(false)); RAISE_STREAK.with(|s| s.set(0)); PROMOTE_LOCK.with(|l| l.set(false)); UNLOCK_STREAK.with(|s| s.set(0)); @@ -878,22 +1027,35 @@ mod tests { fn drops_immediately_and_rises_debounced() { reset_for_test(); let desired = desired_survivor_bytes(); - assert_eq!(tenuring_survivals(), 4); - - // Heavy influx: instant drop to 1. + // Power-on is the FLOOR now, not the ceiling (startup follow-up): the + // ladder may not claim a lifetime in either direction without evidence. + assert_eq!(tenuring_survivals(), OCCUPANCY_MIN_SURVIVALS); + + // Give the loop its evidence, because this test is about the ladder's + // ASYMMETRY and not about the startup gate. Two cycles with a cohort + // that fully dies: the second rates the first, so a survivor round has + // been measured, and 0 % survival keeps the lock out of it. + retune_after_scavenge(desired * 2, 3 * desired, 0); + retune_after_scavenge(desired * 2, 3 * desired, 0); + assert_eq!(tenuring_survivals(), OCCUPANCY_MIN_SURVIVALS); + + // Heavy influx: instant drop, no debounce. #9851 changed the FLOOR this + // lands on (2, not 1 — the occupancy rule may not claim a lifetime), not + // the asymmetry this test is named for: 4 -> 2 in one cycle is the same + // "drops immediately" property that 4 -> 1 was. retune_after_scavenge(desired * 2, 0, 0); - assert_eq!(tenuring_survivals(), 1); + assert_eq!(tenuring_survivals(), OCCUPANCY_MIN_SURVIVALS); // One quiet cycle: no rise yet (debounce). retune_after_scavenge(0, 0, 0); - assert_eq!(tenuring_survivals(), 1); + assert_eq!(tenuring_survivals(), OCCUPANCY_MIN_SURVIVALS); // Second quiet cycle: rise by exactly one step, not to the target. retune_after_scavenge(0, 0, 0); - assert_eq!(tenuring_survivals(), 2); + assert_eq!(tenuring_survivals(), 3); // Heavy again: streak resets and threshold drops straight back. retune_after_scavenge(desired * 2, 0, 0); - assert_eq!(tenuring_survivals(), 1); + assert_eq!(tenuring_survivals(), OCCUPANCY_MIN_SURVIVALS); // Sustained quiet recovers to the ceiling two cycles per step. for _ in 0..6 { @@ -911,10 +1073,13 @@ mod tests { // every cycle even while the cap scale walks up underneath it. An // influx only marginally above the base desired is a different case: // the growing cap re-classifies it as moderate, which is correct. + // #9851: the fixed point is now the occupancy floor (2) rather than 1. + // Fixed-POINTNESS is what this test protects — no oscillation while the + // cap scale walks up underneath — and that is unchanged. let heavy = gc_scavenge_nursery_cap_bytes(); for _ in 0..10 { retune_after_scavenge(heavy, 0, 0); - assert_eq!(tenuring_survivals(), 1); + assert_eq!(tenuring_survivals(), OCCUPANCY_MIN_SURVIVALS); } assert_eq!( scavenge_nursery_cap_effective_bytes(), @@ -924,6 +1089,183 @@ mod tests { reset_for_test(); } + /// #9851, both halves of the rule in one test, in the #7909 two-phase shape + /// so the decline is ATTRIBUTED rather than merely absent. + /// + /// Phase 1 — the occupancy rule alone, on an influx far above `desired`, + /// must stop at 2 and NOT claim promote-on-first-copy. That is the whole + /// change: 2 is the lowest threshold that still puts a cohort through the + /// survivor space, so the loop keeps producing the measurement that could + /// refute it. + /// + /// Phase 2 — the same heap, once a substantial cohort HAS come back fully + /// alive, must still reach 1 through the survival-rate lock. The rule + /// removes an unmeasured conclusion, not the measured one, and this half is + /// what makes it self-limiting rather than a blanket floor. + /// + /// Sabotage: drop the `.max(OCCUPANCY_MIN_SURVIVALS)` in + /// `retune_after_scavenge` and phase 1 fails (the loop reports 1 with no + /// evidence). Drop the lock instead and phase 2 fails. + #[test] + fn occupancy_alone_never_claims_promote_on_first_copy_but_the_lock_still_can() { + reset_for_test(); + let d = desired_survivor_bytes(); + + // Phase 1: influx 16x the desired survivor size — the occupancy formula + // computes 1 (integer division: 1 + desired/influx). No cohort has been + // rated yet, so there is NO lifetime evidence on this heap. + assert_eq!( + compute_target_survivals(16 * d, d), + 1, + "precondition: the occupancy ARITHMETIC still computes 1 — this \ + change clamps what the loop may do with it, not the formula" + ); + for _ in 0..5 { + retune_after_scavenge(16 * d, 0, 0); + assert_eq!( + tenuring_survivals(), + OCCUPANCY_MIN_SURVIVALS, + "occupancy measures SPACE and must not conclude promote-on-first-copy" + ); + } + assert!( + !PROMOTE_LOCK.with(Cell::get), + "and it must not have taken the lock's route to get there" + ); + + // Phase 2: now a substantial cohort goes through the survivor space and + // comes back fully alive. THAT is lifetime evidence, and it must still + // reach 1. + // + // TWO cycles, deliberately: the lock rates the fresh cohort's survival + // against the PREVIOUS cycle's `eden_copied_bytes` (`PREV_COPIED_BYTES`), + // so the first call is what puts a cohort in the survivor space and the + // second is what reports that same cohort coming back alive. Both + // arguments here are cohort-scoped, which is what the follow-up to + // #9851 made them: the whole survivor space is a different population + // once the threshold rises above 2. Phase 1 above copied nothing, so + // there is nothing to rate until this pair runs — which is precisely the + // blindness the change is about, here as a test mechanic. + retune_after_scavenge(16 * d, 3 * d, 0); + assert_eq!( + tenuring_survivals(), + OCCUPANCY_MIN_SURVIVALS, + "one cohort into the survivor space is not yet evidence about it" + ); + retune_after_scavenge(16 * d, 3 * d, 3 * d); + assert_eq!( + tenuring_survivals(), + 1, + "a substantial intake that fully survives its round must still lock \ + promote-on-first-copy — the measured path is untouched" + ); + assert!(PROMOTE_LOCK.with(Cell::get), "...through the lock"); + reset_for_test(); + } + + /// STARTUP FOLLOW-UP — the occupancy rule may not claim the CEILING either. + /// + /// `1 + desired / influx` returns the ceiling for a tiny influx and for a + /// zero one, so on the first minors of a process — heap nearly empty, no + /// cohort ever followed — occupancy claims the maximum. That is the same + /// category error as claiming 1: a statement about LIFETIME derived from a + /// measurement of SPACE, made before any evidence exists. It is also the + /// expensive direction, because every survivor is then copied up to three + /// times before it may be promoted. + /// + /// Measured on the compiled claude-code TUI, this was the WHOLE difference + /// between the adaptive loop and a pinned threshold: a single startup + /// excursion `4 -> 2 (occupancy) -> 1 (lock)` inside turn 1, nothing + /// afterwards, worth +0.35..0.45 s at 3300 characters and +50 % at 400. + /// + /// Sabotage: delete the `SURVIVOR_ROUND_MEASURED` gate in + /// `retune_after_scavenge` (or restore the power-on value to + /// `GC_TENURING_SURVIVALS_MAX`) and phase 1 fails — the loop reports the + /// ceiling on a heap where nothing has ever been rated. + #[test] + fn occupancy_may_not_claim_the_ceiling_before_any_round_is_measured() { + reset_for_test(); + let d = desired_survivor_bytes(); + + // Precondition: the ARITHMETIC still says "ceiling" for a startup-sized + // influx. This change gates what the loop may do with that, exactly as + // #9851 did at the other end of the range. + assert_eq!(compute_target_survivals(0, d), GC_TENURING_SURVIVALS_MAX); + assert_eq!( + compute_target_survivals(d / 64, d), + GC_TENURING_SURVIVALS_MAX + ); + + // Phase 1: power-on, then many startup-shaped minors — tiny influx, + // nothing copied, so nothing rateable. The loop must sit at the floor + // and never climb. + assert_eq!(tenuring_survivals(), OCCUPANCY_MIN_SURVIVALS); + for _ in 0..8 { + retune_after_scavenge(d / 64, 0, 0); + assert_eq!( + tenuring_survivals(), + OCCUPANCY_MIN_SURVIVALS, + "no survivor round has been rated, so occupancy has no lifetime \ + evidence and may not leave the floor" + ); + } + assert!( + !PROMOTE_LOCK.with(Cell::get), + "and it must not have reached the floor via the lock either" + ); + + // Phase 2: once a cohort has actually gone through the survivor space + // and been followed, the ladder is allowed to move again. A cohort that + // fully dies keeps the lock out, so what is observed here is the + // occupancy rule being re-enabled and nothing else. + retune_after_scavenge(d / 64, 3 * d, 0); + retune_after_scavenge(d / 64, 3 * d, 0); + for _ in 0..8 { + retune_after_scavenge(d / 64, 0, 0); + } + assert_eq!( + tenuring_survivals(), + GC_TENURING_SURVIVALS_MAX, + "with a round measured and the influx quiet, the debounced rise must \ + still reach the ceiling — the gate delays the claim until there is \ + evidence, it does not remove the ladder" + ); + reset_for_test(); + } + + /// #9851: a cohort that DIES in its survivor round must keep the loop at the + /// occupancy floor rather than being locked to 1 — the case cc actually is. + /// Measured there: 26.1 % of each cohort dies in one survivor round, in + /// steady state, on 393 samples; the lock needs >=90 % survival, so it + /// correctly stays out. + /// + /// The arguments are the FRESH COHORT's intake and survival (#9851 + /// follow-up). Fed the whole survivor space instead — which is what the + /// call site used to pass — this same heap locks, because above a threshold + /// of 2 that space also holds objects already selected for longevity. That + /// is not a hypothetical: on cc the clamp alone left 85 % of promotion at + /// S=1, reached through this lock 8-12 times per four-turn run. + #[test] + fn a_cohort_that_dies_in_its_round_holds_at_the_occupancy_floor() { + reset_for_test(); + let d = desired_survivor_bytes(); + // Heavy influx (occupancy says 1) AND a substantial cohort of which + // ~26 % dies — cc's steady state, in miniature. + for _ in 0..8 { + retune_after_scavenge(16 * d, 4 * d, 3 * d); + } + assert!( + !PROMOTE_LOCK.with(Cell::get), + "74 % survival is below the lock's 90 % bar: the lock must stay out" + ); + assert_eq!( + tenuring_survivals(), + OCCUPANCY_MIN_SURVIVALS, + "so the loop holds at the occupancy floor and keeps aging the cohort" + ); + reset_for_test(); + } + #[test] fn survival_rate_lock_breaks_a_saturated_pipeline() { reset_for_test(); @@ -999,15 +1341,23 @@ mod tests { let d = desired_survivor_bytes(); // Medium-lived objects: a substantial intake of which only half // survives its survivor round. Aging is filtering — the lock must - // stay out and the occupancy ladder must decide. + // stay out and the occupancy ladder must age from the power-on floor. + let mut seen = Vec::new(); for _ in 0..6 { retune_after_scavenge(d / 2, d / 2, d / 4); assert!( - tenuring_survivals() >= 3, - "a cohort that dies in the survivor space must keep aging (got {})", - tenuring_survivals() + !PROMOTE_LOCK.with(Cell::get), + "50% survival is below the lock's 90% bar" ); + seen.push(tenuring_survivals()); } + assert_eq!( + seen, + [2, 2, 3, 3, 3, 3], + "power-on is the floor now, not the ceiling: after a survivor round \ + is measured, the debounced occupancy ladder must keep the dying \ + cohort aging rather than claim promote-on-first-copy" + ); reset_for_test(); } @@ -1095,8 +1445,9 @@ mod tests { let eden_live = d * 4; assert_eq!( tenuring_survivals(), - 4, - "with no input the loop is at the ceiling: the wasted copy state" + OCCUPANCY_MIN_SURVIVALS, + "power-on is the floor now, not the ceiling: with no lifetime \ + evidence the loop may not claim either extreme" ); seed_promote_lock_from_sweep(eden_live, eden_live / 50); @@ -1127,8 +1478,9 @@ mod tests { seed_promote_lock_from_sweep(eden_live, eden_dead); assert_eq!( tenuring_survivals(), - 4, - "10% Eden survival must not seed promote-on-first-copy" + OCCUPANCY_MIN_SURVIVALS, + "10% Eden survival must leave the loop at the power-on floor, not \ + seed promote-on-first-copy by claiming a threshold below 2" ); reset_for_test(); } @@ -1142,7 +1494,10 @@ mod tests { reset_for_test(); let d = desired_survivor_bytes(); seed_promote_lock_from_sweep(d / 8, 0); - assert_eq!(tenuring_survivals(), 4); + // Unchanged from power-on, which is the floor now rather than the + // ceiling (startup follow-up). The property under test is that the + // sweep seed REFUSED — it left the threshold where it found it. + assert_eq!(tenuring_survivals(), OCCUPANCY_MIN_SURVIVALS); reset_for_test(); } diff --git a/crates/perry-runtime/src/gc/tests/copying/adaptive_tenuring.rs b/crates/perry-runtime/src/gc/tests/copying/adaptive_tenuring.rs index 9abfa5a6e2..b8259420ac 100644 --- a/crates/perry-runtime/src/gc/tests/copying/adaptive_tenuring.rs +++ b/crates/perry-runtime/src/gc/tests/copying/adaptive_tenuring.rs @@ -27,22 +27,24 @@ fn heavy_influx_lowers_threshold_and_promotes_next_cycle() { let _guard = CopyingNurseryTestGuard::new(SLOTS); assert_eq!( crate::gc::tenuring::tenuring_survivals(), - GC_COPY_PROMOTION_SURVIVALS, - "guard must start every test at the power-on threshold" + crate::gc::tenuring::OCCUPANCY_MIN_SURVIVALS, + "guard must start every test at the power-on floor, not the ceiling" ); fill_slots_with_heavy_influx(); let before = (js_shadow_slot_get(0) & POINTER_MASK) as usize; assert!(crate::arena::pointer_in_nursery(before)); - // Cycle 1 runs at the power-on threshold: the cohort is copied into a - // survivor space (ages to 1), and its influx re-tunes the threshold down - // to promote-on-first-copy. + // Cycle 1 runs at the power-on floor: the cohort is copied into a survivor + // space (ages to 1), and heavy influx must not take occupancy below that + // floor by claiming promote-on-first-copy without lifetime evidence. let _ = gc_collect_minor(); assert_eq!( crate::gc::tenuring::tenuring_survivals(), - 1, - "a >desired Eden survivor influx must drop the threshold to 1" + crate::gc::tenuring::OCCUPANCY_MIN_SURVIVALS, + "a >desired Eden survivor influx must drop the threshold to the \ + occupancy floor (#9851: the occupancy rule measures space and may not \ + claim promote-on-first-copy, which is a claim about lifetime)" ); let after_first = (js_shadow_slot_get(0) & POINTER_MASK) as usize; assert!( @@ -51,7 +53,11 @@ fn heavy_influx_lowers_threshold_and_promotes_next_cycle() { ); // Cycle 2 promotes the whole cohort instead of re-copying it: this is - // the ping-pong the adaptive threshold exists to break. + // the ping-pong the adaptive threshold exists to break. #9851 did NOT + // weaken this half — the cohort was copied once in cycle 1, so its + // `next_age` here is 2, which still satisfies `next_age >= 2`. The test's + // named invariant ("lowers threshold AND promotes next cycle") is intact; + // only the literal threshold moved. let _ = gc_collect_minor(); for slot in 0..SLOTS { let addr = (js_shadow_slot_get(slot) & POINTER_MASK) as usize; @@ -215,7 +221,13 @@ fn quiet_cycles_restore_power_on_threshold_debounced() { fill_slots_with_heavy_influx(); let _ = gc_collect_minor(); - assert_eq!(crate::gc::tenuring::tenuring_survivals(), 1); + // #9851: the occupancy floor, not 1. What this test protects — a DEBOUNCED + // restore, at most one step per cycle, ending at the power-on threshold — + // is asserted structurally below and is unchanged. + assert_eq!( + crate::gc::tenuring::tenuring_survivals(), + crate::gc::tenuring::OCCUPANCY_MIN_SURVIVALS + ); // Promote the cohort out of the nursery so later cycles are quiet. let _ = gc_collect_minor(); diff --git a/crates/perry-runtime/src/gc/tests/copying/promoted_remembered_7803.rs b/crates/perry-runtime/src/gc/tests/copying/promoted_remembered_7803.rs index 45c6c25f82..18963db4af 100644 --- a/crates/perry-runtime/src/gc/tests/copying/promoted_remembered_7803.rs +++ b/crates/perry-runtime/src/gc/tests/copying/promoted_remembered_7803.rs @@ -62,6 +62,8 @@ fn young_padded_closure_capturing(bits: u64) -> usize { #[test] fn drain_promoted_parent_keeps_its_young_child_edge_remembered() { let _guard = CopyingNurseryTestGuard::new(1); + let _tenuring = + crate::gc::tenuring::set_survivals_for_test(crate::gc::tenuring::GC_TENURING_SURVIVALS_MAX); // parent captures a young leaf; intermediate captures parent. Only the // INTERMEDIATE is rooted, so the parent is reached — and, on the @@ -95,9 +97,8 @@ fn drain_promoted_parent_keeps_its_young_child_edge_remembered() { deref(capture_bits_of(spacer)) }; - // Age everyone to the brink of promotion (power-on threshold: promote on - // the fourth survival — pinned by - // `test_copying_minor_promotes_survivor_on_fourth_survival`). + // Age everyone to the explicitly pinned promotion boundary: the fourth + // survival. This test exercises drain promotion at S=4, not power-on. for _ in 0..3 { let _ = gc_collect_minor(); } diff --git a/crates/perry-runtime/src/gc/tests/copying/survival_and_malloc.rs b/crates/perry-runtime/src/gc/tests/copying/survival_and_malloc.rs index aed0b6c681..b6104a63f5 100644 --- a/crates/perry-runtime/src/gc/tests/copying/survival_and_malloc.rs +++ b/crates/perry-runtime/src/gc/tests/copying/survival_and_malloc.rs @@ -3,6 +3,8 @@ use super::*; #[test] fn test_copying_minor_promotes_survivor_on_fourth_survival() { let _guard = CopyingNurseryTestGuard::new(1); + let _tenuring = + crate::gc::tenuring::set_survivals_for_test(crate::gc::tenuring::GC_TENURING_SURVIVALS_MAX); let child = young_leaf(); js_shadow_slot_set(0, ptr_bits(child)); @@ -53,6 +55,8 @@ fn test_copying_minor_preserves_old_page_accounting_for_defrag_policy() { pinned_header: std::ptr::null_mut(), }; let _guard = CopyingNurseryTestGuard::new(1); + let _tenuring = + crate::gc::tenuring::set_survivals_for_test(crate::gc::tenuring::GC_TENURING_SURVIVALS_MAX); let _trigger_guard = GcTriggerThresholdTestGuard::suppress_automatic_triggers(); clear_marks(); clear_mark_seeds(); @@ -176,6 +180,8 @@ fn test_copying_minor_preserves_old_page_accounting_for_defrag_policy() { #[test] fn test_copying_minor_sticky_old_to_survivor_edge_promotes_on_fourth_cycle() { let _guard = CopyingNurseryTestGuard::new(0); + let _tenuring = + crate::gc::tenuring::set_survivals_for_test(crate::gc::tenuring::GC_TENURING_SURVIVALS_MAX); let child = young_leaf(); let (old_arr, elements) = unsafe { alloc_old_test_array(1) }; unsafe { @@ -964,6 +970,8 @@ fn test_movable_regexp_evacuation_migrates_all_address_owned_state() { #[test] fn test_copied_minor_promotable_census_filtered_walk_matches_unfiltered() { let _guard = CopyingNurseryTestGuard::new(1); + let _tenuring = + crate::gc::tenuring::set_survivals_for_test(crate::gc::tenuring::GC_TENURING_SURVIVALS_MAX); let child = young_leaf(); js_shadow_slot_set(0, ptr_bits(child)); @@ -1065,3 +1073,85 @@ fn nursery_regexp_that_dies_young_is_finalized_by_the_copied_minor() { ); js_shadow_slot_set(0, 0); } + +/// #9851 follow-up — THE PREMISE OF THE LOCK REWIRE, on a real heap. +/// +/// The survival-rate lock used to rate `survivor_live_bytes` (every live byte +/// leaving the from-survivor space, of any age) against the previous cycle's +/// whole `copied_bytes`. Those two scopes match — the survivor spaces are a +/// strict semispace pair, so the from-space holds exactly what the last cycle +/// copied — and the ratio is well-formed. What is wrong is *which population* +/// it rates, and that is chosen by the threshold the lock itself sets: at a +/// threshold of 2 the space holds one fresh cohort, at 3 or 4 it also holds +/// objects that have already survived a round and are therefore selected for +/// longevity. +/// +/// This test pins the fact that makes the rewire meaningful rather than a +/// rename: **at a threshold above 2 the whole-space number and the fresh-cohort +/// number are different numbers**, with the aged resident in the first and not +/// in the second. On cc that difference is the whole finding — the aggregate +/// clears the lock's 90 % bar while a fresh cohort survives at 74 %. +/// +/// Shape: at an explicitly pinned threshold above 2 (promote on the 4th +/// survival) two rooted objects are introduced one cycle apart, so by the +/// third minor the from-survivor space holds one age-2 object and one age-1 +/// object. +#[test] +fn the_survivor_space_and_the_fresh_cohort_are_different_numbers_above_threshold_two() { + // TWO shadow slots: the test needs two independently rooted objects + // introduced one cycle apart, so that the survivor space holds two age + // classes at once. With one slot B is unrooted, dies immediately, and the + // fresh-cohort number is trivially zero. + let _guard = CopyingNurseryTestGuard::new(2); + let _tenuring = + crate::gc::tenuring::set_survivals_for_test(crate::gc::tenuring::GC_TENURING_SURVIVALS_MAX); + + // Cycle 1: A enters the survivor space from Eden. The from-survivor space + // was empty, so both numbers are zero and the cohort is all of nothing. + let a = young_leaf(); + js_shadow_slot_set(0, ptr_bits(a)); + let _ = gc_collect_minor(); + let (_, _, survivor_live_1, first_round_1) = crate::gc::copying::test_last_cohort_split(); + assert_eq!( + (survivor_live_1, first_round_1), + (0, 0), + "cycle 1 evacuates Eden only: nothing came out of the survivor space" + ); + + // Cycle 2: A is re-copied (age 1 -> 2) and B enters from Eden. The + // from-survivor space held ONLY A, which is a first-round object, so the + // two numbers must still agree — this is the regime the lock was designed + // in, and the assertion that the split is not simply always different. + let b = young_leaf(); + js_shadow_slot_set(1, ptr_bits(b)); + let _ = gc_collect_minor(); + let (_, _, survivor_live_2, first_round_2) = crate::gc::copying::test_last_cohort_split(); + assert!( + survivor_live_2 > 0, + "A must have come back out of the survivor space" + ); + assert_eq!( + survivor_live_2, first_round_2, + "with a single generation resident the whole-space number IS the \ + fresh-cohort number — at threshold <= 2 the old rule was correct" + ); + + // Cycle 3: the from-survivor space now holds A (age 2) and B (age 1). + // `survivor_live_bytes` counts both; the fresh cohort is B alone. + let _ = gc_collect_minor(); + let (_, _, survivor_live_3, first_round_3) = crate::gc::copying::test_last_cohort_split(); + assert!( + first_round_3 > 0, + "B is a first-round survivor and must be counted as one" + ); + assert!( + survivor_live_3 > first_round_3, + "the aged resident A is in the whole-space number and must NOT be in \ + the fresh-cohort number: whole-space {survivor_live_3}, cohort \ + {first_round_3}. If these are equal the lock is still rating a \ + population its own threshold selected." + ); + + js_shadow_slot_set(0, 0); + js_shadow_slot_set(1, 0); +} diff --git a/crates/perry-runtime/src/gc/tests/copying/weak_holder_registry.rs b/crates/perry-runtime/src/gc/tests/copying/weak_holder_registry.rs index 581e22d237..1ea175c046 100644 --- a/crates/perry-runtime/src/gc/tests/copying/weak_holder_registry.rs +++ b/crates/perry-runtime/src/gc/tests/copying/weak_holder_registry.rs @@ -242,6 +242,8 @@ fn test_full_weak_processing_work_is_independent_of_unrelated_heap_size() { #[test] fn test_registry_tracks_holder_across_three_moving_minors() { let _guard = CopyingNurseryTestGuard::new(3); + let _tenuring = + crate::gc::tenuring::set_survivals_for_test(crate::gc::tenuring::GC_TENURING_SURVIVALS_MAX); let map = crate::weakref::js_weakmap_new(); let live_key = crate::object::js_object_alloc(0, 0); diff --git a/crates/perry-runtime/src/gc/tests/oldgen.rs b/crates/perry-runtime/src/gc/tests/oldgen.rs index 589adc3132..79b1453fd2 100644 --- a/crates/perry-runtime/src/gc/tests/oldgen.rs +++ b/crates/perry-runtime/src/gc/tests/oldgen.rs @@ -1328,6 +1328,8 @@ fn test_minor_skips_whole_heap_old_to_young_rebuild() { #[test] fn test_minor_preserves_old_to_young_edge_across_minors() { let _isolation = copying_nursery_isolation_lock(); + let _tenuring = + crate::gc::tenuring::set_survivals_for_test(crate::gc::tenuring::GC_TENURING_SURVIVALS_MAX); let _trigger_guard = GcTriggerThresholdTestGuard::suppress_automatic_triggers(); let _barrier_guard = GeneratedWriteBarrierTestGuard::active(); reset_remembered_set(); diff --git a/crates/perry-runtime/src/gc/tests/runtime_roots/hook_dispatch_handles.rs b/crates/perry-runtime/src/gc/tests/runtime_roots/hook_dispatch_handles.rs index 29217da550..cc65a5e865 100644 --- a/crates/perry-runtime/src/gc/tests/runtime_roots/hook_dispatch_handles.rs +++ b/crates/perry-runtime/src/gc/tests/runtime_roots/hook_dispatch_handles.rs @@ -131,6 +131,8 @@ fn test_timer_tick_roots_callback_args_and_previous_context_across_hooks() { let _async_hook_guard = AsyncHookRuntimeTestGuard::new(); let _guard = CopyingNurseryTestGuard::new(0); + let _tenuring = + crate::gc::tenuring::set_survivals_for_test(crate::gc::tenuring::GC_TENURING_SURVIVALS_MAX); let _trigger_guard = GcTriggerThresholdTestGuard::suppress_automatic_triggers(); register_runtime_handle_root_scanner_for_tests(); gc_register_mutable_root_scanner(crate::async_hooks::scan_async_hooks_roots_mut); @@ -289,6 +291,8 @@ fn test_array_map_runtime_handles_survive_callback_copied_minor_gc() { #[test] fn test_map_materializers_runtime_handles_survive_copied_minor_gc() { let _guard = CopyingNurseryTestGuard::new(0); + let _tenuring = + crate::gc::tenuring::set_survivals_for_test(crate::gc::tenuring::GC_TENURING_SURVIVALS_MAX); let _trigger_guard = GcTriggerThresholdTestGuard::suppress_automatic_triggers(); register_runtime_handle_root_scanner_for_tests(); diff --git a/crates/perry-runtime/src/gc/tests/support.rs b/crates/perry-runtime/src/gc/tests/support.rs index 130c0f7a08..d0cbecce02 100644 --- a/crates/perry-runtime/src/gc/tests/support.rs +++ b/crates/perry-runtime/src/gc/tests/support.rs @@ -414,9 +414,10 @@ pub(crate) struct CopyingNurseryTestGuard { } pub(super) fn reset_copying_nursery_runtime_test_state() { - // Age-sensitive tests assume the power-on tenuring threshold (promote at - // the 4th survival); pin it so a heavy-influx test earlier on the same - // thread cannot leak a lowered adaptive threshold in. + // Restore the adaptive policy to its power-on floor. Tests of mechanisms + // that require a particular promotion age pin it explicitly with + // `tenuring::set_survivals_for_test`, so a power-on policy change cannot + // silently change the mechanism they exercise. crate::gc::tenuring::reset_for_test(); // #7645: the young-pin latch is process-wide and monotone, so one // earlier pinning test would otherwise leave every later copying test diff --git a/scripts/gc_runtime_root_holders.json b/scripts/gc_runtime_root_holders.json index f1582b60c2..6da701028b 100644 --- a/scripts/gc_runtime_root_holders.json +++ b/scripts/gc_runtime_root_holders.json @@ -266,6 +266,18 @@ "verdict": "not_a_gc_pointer", "why": "Boolean census request latch, set by census_arm and consumed at full-sweep entry; contains no address or JS value." }, + { + "file": "crates/perry-runtime/src/gc/copying.rs", + "name": "LAST_COHORT_SPLIT", + "verdict": "test_only", + "why": "Declared under #[cfg(test)] at crates/perry-runtime/src/gc/copying.rs:1951; this Cell<(usize, usize, usize, usize)> holds only the byte counts of the last survivor-space/fresh-cohort split for the tenuring lock tests. It is absent from shipped binaries." + }, + { + "file": "crates/perry-runtime/src/gc/tenuring.rs", + "name": "SURVIVOR_ROUND_MEASURED", + "verdict": "not_a_gc_pointer", + "why": "Declared at crates/perry-runtime/src/gc/tenuring.rs:212; this Cell records whether any survivor round has been rated on this thread, gating the occupancy rule off its floor. A boolean, never a heap pointer." + }, { "file": "crates/perry-runtime/src/gc/census.rs", "name": "LABEL",