Merge train: #9795, #9844, #9845, #9853, #9857, #9860, #9862, #9864, #9865, #9868 (+ #9869)

changelog.d/4644-retained-growth-verifier.md

@@ -0,0 +1 @@
+- Fix a false evacuation-verifier abort when a copying minor encounters a retained, non-moving array-growth alias, such as Solid's effect dependency array. Verification still follows the full forwarding chain and rejects nursery evacuation originals; old-page evacuation retains its strict checks.

changelog.d/9830-gc-trigger-path-hot-tls.md

@@ -0,0 +1,55 @@
+**The GC trigger path and the dirty-page barrier stop paying `_tlv_get_addr`
+per read, and the policy gate that let them stop paying it now counts the
+thing it is bounding.**
+
+`gc_check_trigger` runs on every `gc_malloc`, and its predicate
+(`gc_budgeted_due_trigger`) resolved eleven raw `thread_local!` declarations
+one out-of-line call at a time. Measured with `sample` on the compiled
+claude-code TUI streaming a 3300-char reply (14,578 active main-thread
+samples, callers resolved by an explicit ancestor walk rather than
+nearest-symbol labels): `_tlv_get_addr` was 380 main-thread leaf samples,
+**71 of them with `gc_budgeted_due_trigger` as the immediate caller**, 36 in
+`old_page_account_dirty_slots`, 31 in `scan_dirty_object_slots`, 27 in
+`gc_malloc_header_is_tracked`. `crates/perry-runtime/src/tls_hot.rs` has
+existed to abolish exactly this since #7469; the allocation path's *fields*
+were covered and the trigger path never was.
+
+Sixty-seven declarations across `gc/policy.rs`, `gc/malloc.rs`, `gc/old_free.rs`,
+`gc/tenuring.rs`, `gc/trace.rs`, `gc/barrier/mod.rs`, `arena/block.rs` and
+`arena/page_meta.rs` move to `crate::perry_thread_local!` — same syntax, same
+`.with()` at every call site, the address served from this thread's hot cache
+instead of a libdyld call.
+
+**Why they were still cold is a measurement bug in the gate, not an oversight
+anyone could have noticed.** `scripts/check_thread_locals.py` ratchets on the
+number of raw `thread_local!` **blocks** per file, while `thread_local! { … }`
+holds any number of declarations — so `gc/policy.rs` counted as **6** while
+declaring **28**, and adding a `static` to an already-recorded block passed
+the gate silently. Counted in the same unit as the hot side, `main` was **318
+hot declarations against 339 cold ones** — cold was the majority, reported as
+a 2.6:1 minority. The gate now ratchets on declarations (`385 hot / 272
+cold`), and `--self-test` grew a seventh direction that fails when a `static`
+is added to a recorded block; restoring the block count makes that case, and
+only that case, fail.
+
+Three declarations stay deliberately raw and say so at their declaration:
+`ARENA_TOTAL_BYTES`, `BLOCK_POOL` and `BLOCK_POOL_BYTES` are read from
+`Arena::new`, which runs as `tls_hot::fill`'s **first** provider, so a
+`HotKey` there re-enters `fill` — which by design has not yet written the
+`temp_roots` field it gates on — and re-runs `ARENA`'s initializer without
+bound. It is a stack overflow at thread start, not a slow path, and it is the
+first documented instance of the rule that a declaration read from inside a
+`fill` provider cannot use the macro. `gc::tests::tls_fill_reentrancy` is the
+standing guard, and it is sabotage-proved: moving `ARENA_TOTAL_BYTES` alone
+into the neighbouring hot block aborts that test with `fatal runtime error:
+stack overflow`.
+
+`gc::tests::trigger_path_tls` is the runtime half of the gate: it drives
+`gc_check_trigger` on a fresh thread and asserts every trigger-path
+declaration owns a hot slot and that the path publishes slots at all.
+Reverting any one of them to a raw `thread_local!` removes `slot_index` and
+breaks the build at that declaration's own name. It is a test that can fail
+and did: the first run rejected `GC_DEFERRED_REQUEST` with `index 4294967295`,
+correctly — `defer_gc_request` reads it only while a root lock is held, so it
+is not a fast-path read and never claims a slot. The list is what the fast
+path reads, not what the module declares.

changelog.d/9831-idle-reclaim-elapsed-rearm.md

@@ -0,0 +1,73 @@
+**A declined idle compaction is no longer a terminal state: the memory reducer
+re-arms on elapsed idle as well as on mutator collections, so a heap that parks
+1.3 points under the compactor's residue gate gets revisited instead of holding
+221 MB until the next turn.**
+
+Measured on the compiled claude-code TUI, one 400-char turn then a 120 s idle
+window, quiet host (load < 0.1), both rounds of each arm:
+
+| arm | after turn | after 120 s idle |
+|---|---|---|
+| A | 757 / 759 MB | **512 / 527 MB** |
+| R | 738 / 742 MB | **748 / 748 MB** |
+
+R *ends the turn 19 MB better than A* and finishes 221 MB worse. The reclaimer's
+own diagnostic says why, and it is a closed loop:
+
+1. **The compactor's residue gate declines**, reproducibly and narrowly.
+   `compaction_owed` gate 1 wants residue ≥ 25 % of old-gen occupancy; A is at
+   **25.94 / 25.95 %** and starts two compactions, R is at **23.68 / 23.67 %**
+   and starts none. Within-arm spread across rounds is 0.01–0.02 points: a
+   stable operating point just under a threshold, not a coin-flip.
+2. **The decline removes the only event that could revisit it.** The reducer's
+   activity gate needs `2^backoff` collections *it did not start*, and
+   `external_collections()` subtracts only the reducer's own — so a **compaction
+   is what registers as external**. A's trace shows each one contributing
+   exactly +1 (`external_collections` 13 → 14 → 15 across three attempts, one
+   compaction between each). R stays at 9, `since_attempt` never reaches 1, and
+   there is no second attempt in the whole window.
+3. So the heap parks, and the largest piece of the loss is downstream of that:
+   A right-sizes the arena from **182.45 MB of capacity to 81.79 MB** across its
+   three observations, while R holds **168.82 MB** on one. Roughly 87 MB of
+   capacity + 57 MB of young blocks + 38 MB of old-gen ≈ 182 of the 221 MB.
+
+**The fix extends an exemption that already exists twelve lines above it**, for
+the identical deadlock: `StartReason::ArenaRightSize` bypasses the same gate
+because arena blocks need a second full observation that an idle mutator will
+never produce (#9709). This adds `StartReason::IdleElapsed` on the same
+reasoning — a requirement denominated in *mutator collections* cannot be met by
+a heap whose mutator is idle, which is precisely when the reducer is wanted.
+
+**Why the gate constant was not the fix, on measurement rather than principle.**
+Lowering `IDLE_COMPACT_MIN_RESIDUE_PCT` from 25 to 23 would have let R start a
+compaction — and the same R binary in a 5 s window *did* clear the gate, at
+25.81 %, ran the compaction, and **released 0** (`kept_promise=false`,
+`backoff_shift 0→1`). Nor is that peculiar to R: A's own second compaction
+releases 0 at **54.6 %** residue. Half of A's compactions in this capture
+released nothing, aborting ~4x earlier (`pause_us` 107k/161k against 442k) on
+what looks like a budget. The knob is not merely forbidden; it does not work.
+
+**Anti-spin needs no new rule.** The elapsed wait is
+`IDLE_RECLAIM_REARM_MS << backoff_shift` — the *same* shift that prices the
+activity arm — so an unproductive full doubles it: 15 s, 30 s, 60 s, 120 s,
+240 s. And the arm is **disarmed entirely at `IDLE_RECLAIM_MAX_BACKOFF_SHIFT`**
+rather than merely slowed, because five unproductive attempts establish there is
+nothing to give and an idle process must not pay a whole-heap mark forever.
+A productive full resets the shift, so a heap still returning memory keeps being
+asked every 15 s — which is the case this exists for. `IDLE_RECLAIM_REARM_MS` is
+deliberately larger than `IDLE_RECLAIM_MIN_INTERVAL_MS` so the rate floor is
+never the binding constraint and the two gates cannot be confused in a diag.
+
+Two tests, each sabotage-proved: a parked heap with **no** external collection
+anywhere gets a second attempt at the wait and not before, identified by reason
+rather than by attempt count; and an unproductive streak doubles the wait each
+time and then stops. Removing the arm fails the first, removing the backoff
+scaling fails the second's "must not re-arm before the doubled wait", and
+removing the disarm fails its "at the maximum shift the elapsed arm is
+disarmed".
+
+The young half of the loss is **not** addressed here and is measured, not
+assumed: after R's single reclaim, `[gc-general-reclaim] examined=66 released=0
+has_live=39 aging=22` — 39 of 66 arena blocks hold a live object, against 3 of
+65 in A, and only an evacuation can consolidate those. Whether an idle young
+evacuation is also needed is a separate question and a separate change.

changelog.d/9840-regexp-header-nursery.md

@@ -0,0 +1,35 @@
+### Performance
+
+- **A `RegExp` header is allocated in the nursery instead of the malloc arm.**
+  A JS regex literal evaluates to a fresh `RegExp` every time it is reached, and
+  `js_regexp_new` allocated each header with `gc_malloc`. On the claude-code TUI
+  that is, per 400-character reply (`PERRY_GC_TRACE`), **199,873 of 199,926
+  malloc-tracked GC allocations — 100.0 %**, 80 bytes each, 99.2 % of them
+  freed, with the malloc registry swinging **101,929 entries down to 1,689**
+  across a single minor. Every one of those paid a mimalloc allocation, a push
+  onto `MALLOC_STATE.objects`, an insert into the malloc-registry `PtrHashSet`
+  (which rehashes as it grows), and at death a malloc-sweep visit and a free —
+  old-generation prices for an object that overwhelmingly dies young.
+
+  Nothing required the malloc arm. `GC_TYPE_REGEXP` is already declared
+  `ArenaOrMalloc` and movable; `GcMoveHookKind::RegExpSideTables` already rekeys
+  `REGEX_POINTERS`, `REGEX_SOURCE_TABLE` and the expando owner after evacuation,
+  and `GcLayoutSlotKind::RegExpFields` already traces the header's two string
+  edges and its `meta` record. What kept production on `gc_malloc` was
+  finalization: the copying minor's from-space flip runs no per-object finalize
+  hooks, so a nursery header that died young would leak its `Arc` programs and
+  its registry entries. That is now handled exactly as `Map`, `Set` and `Error`
+  handle theirs — `finalize_dead_copied_minor_from_space_regexps` after a copied
+  minor, `collect_dead_registered_regexps_post_trace` at sweep entry for the
+  non-copying cycle kinds, and the ordinary old-generation sweep for a header
+  that has been promoted.
+
+  Every regex program cache (`REGEX_CACHE`, `FANCY_CACHE`,
+  `REPEAT_MATCHER_CACHE`, `VALIDATED_PATTERNS`, the site cache) keys on pattern
+  and flags CONTENT, not on the header address, so a moving header costs them
+  nothing.
+
+  Note that this **changes the collection schedule** rather than only removing
+  work: the `MallocCount` trigger loses essentially all of its input on this
+  workload, while ~16 MB per reply moves into the nursery. The schedule is
+  reported with the change rather than assumed unchanged.

changelog.d/9845-gc-page-class-direct-table.md

@@ -0,0 +1,63 @@
+**The page-generation cache becomes a direct-indexed table over the arena's
+1 MiB address classes, so a classification is a bounds compare and one load
+instead of a four-way probe that missed one call in five.**
+
+`classify_heap_generation` and `classify_heap_space_in_range` sit under three
+callers with no cheaper predicate of their own. The write barrier's
+`remembered_child_needs_tracking` runs **35,871,391 times per turn** on the
+compiled claude-code TUI and **95.23 %** of those take its cheapest arm — one
+cached classification and a compare — so there is no barrier predicate left to
+fix: what remains after the predicate is already optimal is the classification
+itself. `mark_addr` (233 of 760 `classify*` leaf samples) and the side-table
+prunes pay the same cost.
+
+The structure in front of the authoritative `PageGenerationMap` was a **4-way
+round-robin set**. Measured with a dedicated counter on a 3300-char streaming
+reply: **440 M lookups per turn at 20.0–21.6 % miss**, with **59.7–61.8 % of
+misses on a key evicted within the last 64 evictions** — capacity, not conflict —
+against a working set of **402–432 registered classes**. `ways_distinct_max` was
+4, so every way was already in use and the shortfall is ~120x.
+
+**Widening it was not an option, and the reason is on the record.** #7469
+measured 16 ways as an **8.6 % regression** on the same row (0/7 pairs) for 1.5 %
+fewer misses, and five further associativity changes measured flat. The rule
+those produced — *associativity pays only when a miss is expensive* — says that a
+miss which is just a hash lookup wants the cache to become **unnecessary**, not
+larger.
+
+It can be. The registered classes occupy a span of **1,018–1,021 classes at
+~40 % density**, so a table over that span holds every one of them in **160 KB**
+and answers with one bounds compare and one load. `PageGenerationMap` stays
+authoritative and every miss falls through to it exactly as before; the change is
+confined to `PageGenerationCacheSet` and its two callers.
+
+Four things the measurement did not settle, each handled explicitly and each
+pinned by a test that fails when its guard is removed — a wrong answer here is a
+misclassified pointer, so none of them is left to inference:
+
+* **The base moves per process** (`0x43daa2` vs `0x57e3c2` on two runs — ASLR).
+  It is taken from the first insert, never compiled in.
+* **The span can grow** (1,018 → 1,021 across two runs of one binary). An insert
+  outside the table rebases it, up to a 16,384-class cap; past the cap the key is
+  left uncached and falls through to the map rather than being mis-indexed.
+* **The sizing is not obvious.** With base `first_key - S` and a table of `N`,
+  the span covered is `min(S + 1, N - S)`, maximised at `S = N / 2`. The natural
+  pairing `N = 4096, S = 1024` covers **1,025** classes — four above the measured
+  span — while `S = N / 2` covers **2,048** for the same memory. A `const` assert
+  now fails the build for any pairing covering less than twice the measured span.
+* **A key match is not an address match.** A class can hold more than one range,
+  so a hit still requires `range.contains(addr)`.
+
+Invalidation is an epoch bump: O(1), and the same "clear everything" contract the
+4-way set met by being reset wholesale. That contract matters more here, because
+the table holds ~2,000 entries where the set held 4 — a missing invalidation the
+old structure survived by luck would be a live misclassification — so all three
+`PageGenerationMap` mutation sites were enumerated and each ends with an
+unconditional `invalidate_generation_cache()`.
+
+The arm is a plain `u8` field in the set's first cache line rather than the env
+`OnceLock`: this path runs 440 M times per turn, and an acquire load on each
+would have been charged to both arms of the A/B — hiding it in the comparison
+that was meant to isolate it — while still being paid against main.
+`PERRY_GC_PAGE_CLASS_TABLE=0` restores the 4-way set in the same binary, which is
+how the numbers above and below were taken.

changelog.d/9864-for-in-callback-roots.md

@@ -0,0 +1 @@
+- Keep `for…in` receivers, accumulated keys, and Proxy descriptor state rooted across Proxy callbacks and moving garbage collection. Fixes stale pointers when Solid's universal renderer enumerates reactive spread properties.

changelog.d/9865-solid-jsx.md

@@ -0,0 +1 @@
+- Add opt-in `perry.jsx: "solid"` compilation for native Solid JSX, with reactive properties and children, components, keyed control flow, conditional widget identity, spreads, references, and fragments. Provide JSX types and examples in `perry-solid`, and compare native compilation with Solid's official universal JSX transform in the release fixture.

changelog.d/9868-formdata-upload.md

@@ -0,0 +1,4 @@
+### Fixed
+
+- Preserve `Blob` and `File` entries in `FormData`, and serialize `FormData`
+  request bodies with multipart bytes and a generated `content-type` header.

changelog.d/9869-visited-levels-rooting.md

@@ -0,0 +1,17 @@
+#9869: `for-in`'s deferred shadow set recorded each walked prototype level as a plain
+NaN-boxed `f64` in `VisitedLevels`, and dereferenced it later in
+`build_shadow_set` → `mark_own_names` → `js_object_get_own_property_names`.
+
+Between the `visited.push(current)` at level *N* and that read, the walk crosses
+`js_object_keys_value` (which allocates an array) and
+`js_object_get_prototype_of` (which can run a Proxy `getPrototypeOf` trap, i.e.
+arbitrary user JS). Either can collect and move the recorded object, so the
+stored word is a stale pointer whenever a collection lands in that window —
+the same defect #9864 fixes for `out` and `current`, in the one place its patch
+did not reach because the deferred-shadow-set rework landed after it was
+written.
+
+`VisitedLevels` now stores `RuntimeHandle`s, which the collector rewrites in
+place, and `VisitedSlice::iter` reads each level fresh from its handle.
+`RuntimeHandle` is `Copy`, so the inline arm still costs no allocation and the
+"no malloc per `for-in`" property the rework was built for is preserved.

changelog.d/gc-churn-attribution-diag.md

@@ -0,0 +1,22 @@
+### Runtime
+
+- `PERRY_GC_DIAG=1` now says WHY the collector ran, not only what it did:
+  `[gc-trigger]` prints every predicate input at each collection decision
+  (armed arena trigger vs `arena_total`, from-space vs the nursery cap,
+  old-gen reclaimable pressure vs baseline/band, the malloc pair, the
+  pending/retaining flags); `[gc-full]` names the arm behind every full
+  mark-sweep with a per-site count; `[gc-budgeted] start/done` reports each
+  incremental cycle's steps, per-phase step time and root-scan share;
+  `[gc-charge]` attributes mutator-assist and synchronous-full time to the
+  calling site (return-address chain resolved to the JS display name);
+  `[gc-survival]` gives, per copying minor, which root first reached each
+  surviving byte — shadow stack, native stack map, a named side-table
+  scanner, or the remembered set split by the old parent's type — with
+  transitive reach charged to the originating root.
+- `PERRY_ALLOC_SITE_SAMPLE=<bytes>` (arena/alloc_sample.rs): byte-proportional
+  allocation-site sampling for the GC arena, covering the runtime allocators
+  and the codegen inline bump path (the mirrored inline block limit is capped
+  at one interval while sampling). `[alloc-site]` reports bytes by object type
+  and the top sites after each copying minor and at exit. Off by default; one
+  relaxed atomic load per allocation when off; the OFF state and the magnitude
+  parse are pinned in `gc/tests/env_knob_parse.rs`.