perf(gc): prune the per-object layout tables from a young-entry log on a minor

changelog.d/9807-layout-prune-single-pass.md

@@ -0,0 +1,33 @@
+**The per-object layout death-prune walks its tables once instead of three
+times, and no longer allocates a `Vec` of every live key** — 50.6 MB of a
+compiled claude-code turn's 304 MB in the layout tables (#9792).
+
+`prune_dead_per_object_layout_owners` visited every surviving key three times
+per collection: `retain` to drop the dead owners, then
+`layout_addr_filter_rebuild` — which first collected all of them into a
+`Vec<usize>` — and then `recount_young_layout_records` to re-derive the
+nursery-key count. The last two want exactly the survivor set `retain` is
+already walking, so both fold into its closure. The `Vec` is gone from the
+rebuild's other caller too.
+
+The measurement that prompted it also found the accelerator these tables sit
+behind unable to do its job. `layout_addr_filter_may_hold` is a 4,096-bit
+one-hash sketch documented for "one or two entries, ~0.05 % false positives";
+a new `PERRY_LAYOUT_DIAG` instrument reports **162,258 live keys and 4,096 of
+4,096 bits set** on one 400-character claude-code reply. Every probe answers
+"may hold", so the early returns in `transfer_per_object_descriptor` and
+`transfer_per_object_slot_mask` never fire, and each rebuild was an O(live
+keys) walk restoring the all-ones state it started from. Past four times the
+bit count — 16,384 keys, where the false-positive rate is already 98.2 % — the
+rebuild now sets all ones directly, the same conservative answer reached in
+O(1); below that the filter keeps exactly the selectivity it has today. The
+instrument says so out loud rather than leaving it to be inferred. Widening the sketch is not available
+from the runtime: its geometry and hash are mirrored in `perry-codegen`'s
+`emit_gated_forget_object_layout`, and discriminating at 162k keys would take
+~190 KB of inline thread-local storage per thread.
+
+`transfer_per_object_descriptor` also gained the emptiness test its shared
+flag cannot express: the flag and the filter are common to both per-object
+tables, so a full slot-mask table drags every relocation into the typed-layout
+map as well — which on cc is permanently empty (typed=0, masks=162,258). One
+`len` load replaces two hashes per evacuated object.

changelog.d/9841-layout-prune-young-log.md

@@ -0,0 +1,43 @@
+**A minor's per-object layout death-prune now walks a young-entry log instead
+of both tables** — on a compiled claude-code streamed reply it visited
+**6,792,375 entries where at most 125,367 (1.85 %) could possibly have died,
+and on 37 % of minors nothing could have died at all.**
+
+`prune_dead_per_object_layout_owners` asks "which owners died?" of every key
+in `LAYOUT_SLOT_MASKS` + `TYPED_LAYOUTS`, tables sized by everything the
+program ever created (~66k live keys on cc, from a history far larger). But
+both of a minor's deadness predicates require the owner to be in the nursery:
+`owner_is_dead_copied_minor_from_space` demands eden or the active survivor
+half, and `PostTraceProbe::owner_is_dead` on a minor demands an in-arena,
+untenured `HeapGeneration::Nursery` address. An owner that was old at the last
+prune is still old, so the walk over it cannot remove anything.
+
+So the two maps get the young-entry log of #9754 (`gc/young_log.rs`): every
+writer notes a key whose owner `layout_key_may_be_nursery` admits before the
+entry becomes findable, a minor prunes from the log, and a survivor is
+re-logged only while it is still young — a promoted owner leaves the log and
+no later minor visits it again. A full prune keeps its whole-table walk (old
+owners do die in a full trace) and rebuilds the log from the survivors it is
+already classifying, at no extra pass.
+
+**Why this table pays where the scanners of #9754 did not.** Read back per
+table on an unmodified binary, that PR's four converted tables are a net 0.78x
+on cc and `closure.dynamic_props` is a 2.56x regression, because a scanner
+keeps `addr_is_minor_relevant` — true for `Longlived` **by design**, since a
+longlived object can point at a young one — and cc allocates its shape-key
+arrays longlived, so those logs never drain (`kept/logged` median 1.000). A
+prune's predicate is `layout_key_may_be_nursery`, which excludes `Longlived`
+**and** `Old`; cc's tenuring promotes every survivor after one survival, so a
+key leaves this log after one minor. Same mechanism, opposite sign, decided
+entirely by which predicate the walk keeps on. The measured over-visit is 54x
+at a 3300-character reply and 25x at 400, with `dead <= young_before` on
+152/152 minor prunes — the empirical proof that the log's predicate is a sound
+superset of what a minor can kill.
+
+Rule 2 of the design travels with it: under `debug_assertions` the young prune
+re-derives the candidate set from the authoritative maps and panics on any
+young key the log does not name, so deleting an arming site is a red test
+rather than a dead owner's record surviving in silence. The in-borrow mask
+mint in `layout_note_slot` — the dominant insert path on cc, and the one site
+that published a young record without counting it — is armed for the first
+time here.

crates/perry-runtime/src/closure/dynamic_props.rs

@@ -341,6 +341,14 @@ pub(crate) fn prune_dead_closure_side_table_owners(is_dead_closure: &dyn Fn(usiz
 /// young), so the log is the complete candidate set (#9754).
 pub(crate) fn prune_dead_closure_side_table_owners_young(is_dead_closure: &dyn Fn(usize) -> bool) {
     super::prune_dead_closure_box_capture_owners(is_dead_closure);
+    // Rule 2 lives here now. It used to run at the top of the minor-scoped
+    // SCANNER; that walk was dropped as measured-worse-than-full (#9841), and
+    // this prune is the log's remaining consumer, so the machine check that
+    // catches an un-armed writer has to move with it. Deleting the scanner
+    // without moving this would have removed the only guard on a log a prune
+    // still trusts.
+    #[cfg(debug_assertions)]
+    debug_assert_closure_young_log_complete();
     let candidates = CLOSURE_YOUNG_OWNERS.with(|log| log.borrow_mut().take_sorted());
     let mut kept = Vec::with_capacity(candidates.len());
     for owner in candidates {
@@ -450,15 +458,33 @@ pub(crate) fn visit_closure_static_prototype_slot_mut(
 /// ajv's `validate.errors = [{ msg }]`) had its element objects freed
 /// behind the still-live array.
 ///
-/// #9754: a minor-scoped pass (`visitor.young_scope()`) visits only the
-/// owners in `CLOSURE_YOUNG_OWNERS`; a full pass walks every owner and
-/// rebuilds the log. Both go through [`scan_closure_owner`], so the per-entry
-/// work is identical and only the candidate set differs.
+/// #9754 gave this scanner a minor-scoped variant that visited only the owners
+/// in `CLOSURE_YOUNG_OWNERS`. **#9841 withdrew it**: measured, that walk was
+/// worse than the full walk it replaced on the great majority of minor cycles
+/// (see the comment in the body). Every pass is a full pass again, and it
+/// rebuilds the log for the death prune, which is now the log's only consumer
+/// and does earn it.
 pub fn scan_closure_dynamic_props_roots_mut(visitor: &mut crate::gc::RuntimeRootVisitor<'_>) {
-    if visitor.young_scope() {
-        scan_closure_side_tables_young(visitor);
-        return;
-    }
+    // #9841: this table's young walk is MEASURED WORSE THAN THE FULL WALK and
+    // is therefore not taken. On the compiled claude-code TUI (3-turn,
+    // 3300-char, perrymaster, both arms of the FC2 pair) the young-scoped
+    // walk skipped only 2.2-2.7 % of the entries a full walk visits and came
+    // out worse than full on 223 of 273 minor cycles in one arm and 217 of
+    // 272 in the other. The three tables that keep their young walk in the
+    // same capture skip 80-99 %.
+    //
+    // The reason is one predicate: a SCANNER keeps `addr_is_minor_relevant`,
+    // which returns true for `Longlived` BY DESIGN (a longlived object can
+    // point at a young one), so the "relevant" set is close to the whole
+    // table and the log cannot drain — `kept/logged` has median 1.000 here.
+    // Multi-round re-logging then makes the young walk visit MORE than the
+    // full walk it replaces. Same conclusion, same evidence, as the
+    // shape-cache log dropped for skipping 0 % and costing 35 % more.
+    //
+    // The LOG ITSELF STAYS: `prune_dead_closure_side_table_owners_young` uses
+    // it, and a prune's predicate excludes `Longlived` and `Old` both, which
+    // is exactly the asymmetry #9841 is about. This full walk rebuilds the log
+    // from what it finds, so the prune's candidate set stays complete.
     let mut owners: Vec<usize> = Vec::new();
     if let Ok(props) = get_closure_props().lock() {
         owners.extend(props.keys().copied());
@@ -498,56 +524,6 @@ pub fn scan_closure_dynamic_props_roots_mut(visitor: &mut crate::gc::RuntimeRoot
     );
 }
 
-/// The minor-scoped walk: only logged owners. Rounds repeat while visits
-/// trigger owner-move hooks that log new keys (`note_young_closure_owner_rekeyed`),
-/// which is also what closes the pre-#9754 gap where an entry re-keyed
-/// mid-walk was skipped by the mark pass and only rewritten later.
-fn scan_closure_side_tables_young(visitor: &mut crate::gc::RuntimeRootVisitor<'_>) {
-    let table_len = {
-        let props = get_closure_props().lock().map(|m| m.len()).unwrap_or(0);
-        let prototypes = get_closure_prototypes()
-            .lock()
-            .map(|m| m.len())
-            .unwrap_or(0);
-        let deleted = get_closure_deleted_keys()
-            .lock()
-            .map(|m| m.len())
-            .unwrap_or(0);
-        (props + prototypes + deleted) as u64
-    };
-    #[cfg(debug_assertions)]
-    debug_assert_closure_young_log_complete();
-    let mut logged = 0u64;
-    let mut visited = 0u64;
-    let mut kept = CLOSURE_YOUNG_OWNERS.with(|log| log.borrow_mut().take_spare());
-    loop {
-        let batch = CLOSURE_YOUNG_OWNERS.with(|log| log.borrow_mut().take_sorted());
-        if batch.is_empty() {
-            break;
-        }
-        logged += batch.len() as u64;
-        for owner in batch {
-            visited += 1;
-            let (new_owner, relevant) = scan_closure_owner(visitor, owner);
-            if relevant {
-                kept.push(new_owner);
-            }
-        }
-    }
-    let kept_len = kept.len() as u64;
-    CLOSURE_YOUNG_OWNERS.with(|log| log.borrow_mut().extend(kept));
-    crate::gc::young_log::note_walk(
-        CLOSURE_YOUNG_LOG_NAME,
-        crate::gc::young_log::YoungLogWalk {
-            partial: true,
-            logged,
-            visited,
-            kept: kept_len,
-            table_len,
-        },
-    );
-}
-
 /// Rule 2 of `gc/young_log.rs`: re-derive the relevant owners from the three
 /// tables and require the log to name each one.
 #[cfg(debug_assertions)]
@@ -1074,6 +1050,17 @@ pub fn closure_delete_own_dynamic_prop(ptr: usize, prop: &str) -> bool {
     false
 }
 
+/// Sabotage hook for the rule-2 guard in
+/// [`prune_dead_closure_side_table_owners_young`]: drop every logged owner
+/// while LEAVING the three tables populated, so the prune's re-derivation
+/// finds a minor-relevant owner the log does not name. Clearing the tables
+/// too (as `test_clear_closure_side_tables` does) would make the relevant set
+/// empty and the check would return early — i.e. it would test nothing.
+#[cfg(test)]
+pub(crate) fn test_drop_closure_young_log() {
+    CLOSURE_YOUNG_OWNERS.with(|log| log.borrow_mut().clear());
+}
+
 #[cfg(test)]
 pub(crate) fn test_clear_closure_side_tables() {
     CLOSURE_YOUNG_OWNERS.with(|log| log.borrow_mut().clear());

crates/perry-runtime/src/closure/mod.rs

@@ -75,6 +75,8 @@ pub(crate) use box_captures::{
 };
 #[cfg(test)]
 pub(crate) use dynamic_props::test_clear_closure_side_tables;
+#[cfg(test)]
+pub(crate) use dynamic_props::test_drop_closure_young_log;
 pub(crate) use dynamic_props::{
     clear_closure_side_tables_for_dead_ptr, clone_closure_rebind_this,
     closure_dynamic_props_owner_moved, closure_dynamic_side_tables_nonempty,

crates/perry-runtime/src/gc/dead_owner.rs

@@ -336,7 +336,7 @@ pub(super) const DEAD_KEY_PRUNES: &[DeadKeyPrune] = &[
         table: "LAYOUT_SLOT_MASKS + TYPED_LAYOUTS",
         owner: DeadKeyOwner::Any,
         prune: crate::gc::layout_tables::prune_dead_per_object_layout_owners,
-        young_prune: None,
+        young_prune: Some(crate::gc::layout_tables::prune_dead_per_object_layout_owners_young),
     },
     // Re-keyed by the per-object move hook, not by a metadata visitor.
     DeadKeyPrune {

crates/perry-runtime/src/gc/layout.rs

@@ -954,11 +954,23 @@ pub(crate) fn layout_note_slot(parent_user: usize, slot_index: usize, value_bits
                     } else {
                         let mut mask = LayoutSlotMask::Inline(0);
                         mask.set_slot(slot_index);
+                        // The one insert site that holds its own `borrow_mut`,
+                        // so it maintains the address filter, the young log
+                        // and the young-record count inline too. The log lives
+                        // in the hint, not in this map, so arming it here
+                        // takes no second borrow — and it goes BEFORE the
+                        // insert (`gc/young_log.rs` rule 1). Before #9841 this
+                        // site published a young record without counting it;
+                        // on cc it is the DOMINANT insert path (`TYPED_LAYOUTS`
+                        // is empty there), so it is where a missing arm would
+                        // do the most damage.
+                        let young = super::layout_tables::arm_young_layout_key(parent_user);
                         masks.insert(parent_user, mask);
                         mark_per_object_layouts_nonempty();
-                        // The one insert site that holds its own `borrow_mut`,
-                        // so it maintains the address filter inline too.
                         super::layout_tables::layout_addr_filter_note(parent_user);
+                        if young {
+                            super::layout_tables::count_new_young_layout_record();
+                        }
                         set_layout_state(header, GC_LAYOUT_SIDE_MASK);
                     }
                 } else {