diff --git a/changelog.d/4644-retained-growth-verifier.md b/changelog.d/4644-retained-growth-verifier.md new file mode 100644 index 0000000000..f67793ceb8 --- /dev/null +++ b/changelog.d/4644-retained-growth-verifier.md @@ -0,0 +1 @@ +- Fix a false evacuation-verifier abort when a copying minor encounters a retained, non-moving array-growth alias, such as Solid's effect dependency array. Verification still follows the full forwarding chain and rejects nursery evacuation originals; old-page evacuation retains its strict checks. diff --git a/changelog.d/9830-gc-trigger-path-hot-tls.md b/changelog.d/9830-gc-trigger-path-hot-tls.md new file mode 100644 index 0000000000..e791ac3b7d --- /dev/null +++ b/changelog.d/9830-gc-trigger-path-hot-tls.md @@ -0,0 +1,55 @@ +**The GC trigger path and the dirty-page barrier stop paying `_tlv_get_addr` +per read, and the policy gate that let them stop paying it now counts the +thing it is bounding.** + +`gc_check_trigger` runs on every `gc_malloc`, and its predicate +(`gc_budgeted_due_trigger`) resolved eleven raw `thread_local!` declarations +one out-of-line call at a time. Measured with `sample` on the compiled +claude-code TUI streaming a 3300-char reply (14,578 active main-thread +samples, callers resolved by an explicit ancestor walk rather than +nearest-symbol labels): `_tlv_get_addr` was 380 main-thread leaf samples, +**71 of them with `gc_budgeted_due_trigger` as the immediate caller**, 36 in +`old_page_account_dirty_slots`, 31 in `scan_dirty_object_slots`, 27 in +`gc_malloc_header_is_tracked`. `crates/perry-runtime/src/tls_hot.rs` has +existed to abolish exactly this since #7469; the allocation path's *fields* +were covered and the trigger path never was. + +Sixty-seven declarations across `gc/policy.rs`, `gc/malloc.rs`, `gc/old_free.rs`, +`gc/tenuring.rs`, `gc/trace.rs`, `gc/barrier/mod.rs`, `arena/block.rs` and +`arena/page_meta.rs` move to `crate::perry_thread_local!` — same syntax, same +`.with()` at every call site, the address served from this thread's hot cache +instead of a libdyld call. + +**Why they were still cold is a measurement bug in the gate, not an oversight +anyone could have noticed.** `scripts/check_thread_locals.py` ratchets on the +number of raw `thread_local!` **blocks** per file, while `thread_local! { … }` +holds any number of declarations — so `gc/policy.rs` counted as **6** while +declaring **28**, and adding a `static` to an already-recorded block passed +the gate silently. Counted in the same unit as the hot side, `main` was **318 +hot declarations against 339 cold ones** — cold was the majority, reported as +a 2.6:1 minority. The gate now ratchets on declarations (`385 hot / 272 +cold`), and `--self-test` grew a seventh direction that fails when a `static` +is added to a recorded block; restoring the block count makes that case, and +only that case, fail. + +Three declarations stay deliberately raw and say so at their declaration: +`ARENA_TOTAL_BYTES`, `BLOCK_POOL` and `BLOCK_POOL_BYTES` are read from +`Arena::new`, which runs as `tls_hot::fill`'s **first** provider, so a +`HotKey` there re-enters `fill` — which by design has not yet written the +`temp_roots` field it gates on — and re-runs `ARENA`'s initializer without +bound. It is a stack overflow at thread start, not a slow path, and it is the +first documented instance of the rule that a declaration read from inside a +`fill` provider cannot use the macro. `gc::tests::tls_fill_reentrancy` is the +standing guard, and it is sabotage-proved: moving `ARENA_TOTAL_BYTES` alone +into the neighbouring hot block aborts that test with `fatal runtime error: +stack overflow`. + +`gc::tests::trigger_path_tls` is the runtime half of the gate: it drives +`gc_check_trigger` on a fresh thread and asserts every trigger-path +declaration owns a hot slot and that the path publishes slots at all. +Reverting any one of them to a raw `thread_local!` removes `slot_index` and +breaks the build at that declaration's own name. It is a test that can fail +and did: the first run rejected `GC_DEFERRED_REQUEST` with `index 4294967295`, +correctly — `defer_gc_request` reads it only while a root lock is held, so it +is not a fast-path read and never claims a slot. The list is what the fast +path reads, not what the module declares. diff --git a/changelog.d/9831-idle-reclaim-elapsed-rearm.md b/changelog.d/9831-idle-reclaim-elapsed-rearm.md new file mode 100644 index 0000000000..0d9806f173 --- /dev/null +++ b/changelog.d/9831-idle-reclaim-elapsed-rearm.md @@ -0,0 +1,73 @@ +**A declined idle compaction is no longer a terminal state: the memory reducer +re-arms on elapsed idle as well as on mutator collections, so a heap that parks +1.3 points under the compactor's residue gate gets revisited instead of holding +221 MB until the next turn.** + +Measured on the compiled claude-code TUI, one 400-char turn then a 120 s idle +window, quiet host (load < 0.1), both rounds of each arm: + +| arm | after turn | after 120 s idle | +|---|---|---| +| A | 757 / 759 MB | **512 / 527 MB** | +| R | 738 / 742 MB | **748 / 748 MB** | + +R *ends the turn 19 MB better than A* and finishes 221 MB worse. The reclaimer's +own diagnostic says why, and it is a closed loop: + +1. **The compactor's residue gate declines**, reproducibly and narrowly. + `compaction_owed` gate 1 wants residue ≥ 25 % of old-gen occupancy; A is at + **25.94 / 25.95 %** and starts two compactions, R is at **23.68 / 23.67 %** + and starts none. Within-arm spread across rounds is 0.01–0.02 points: a + stable operating point just under a threshold, not a coin-flip. +2. **The decline removes the only event that could revisit it.** The reducer's + activity gate needs `2^backoff` collections *it did not start*, and + `external_collections()` subtracts only the reducer's own — so a **compaction + is what registers as external**. A's trace shows each one contributing + exactly +1 (`external_collections` 13 → 14 → 15 across three attempts, one + compaction between each). R stays at 9, `since_attempt` never reaches 1, and + there is no second attempt in the whole window. +3. So the heap parks, and the largest piece of the loss is downstream of that: + A right-sizes the arena from **182.45 MB of capacity to 81.79 MB** across its + three observations, while R holds **168.82 MB** on one. Roughly 87 MB of + capacity + 57 MB of young blocks + 38 MB of old-gen ≈ 182 of the 221 MB. + +**The fix extends an exemption that already exists twelve lines above it**, for +the identical deadlock: `StartReason::ArenaRightSize` bypasses the same gate +because arena blocks need a second full observation that an idle mutator will +never produce (#9709). This adds `StartReason::IdleElapsed` on the same +reasoning — a requirement denominated in *mutator collections* cannot be met by +a heap whose mutator is idle, which is precisely when the reducer is wanted. + +**Why the gate constant was not the fix, on measurement rather than principle.** +Lowering `IDLE_COMPACT_MIN_RESIDUE_PCT` from 25 to 23 would have let R start a +compaction — and the same R binary in a 5 s window *did* clear the gate, at +25.81 %, ran the compaction, and **released 0** (`kept_promise=false`, +`backoff_shift 0→1`). Nor is that peculiar to R: A's own second compaction +releases 0 at **54.6 %** residue. Half of A's compactions in this capture +released nothing, aborting ~4x earlier (`pause_us` 107k/161k against 442k) on +what looks like a budget. The knob is not merely forbidden; it does not work. + +**Anti-spin needs no new rule.** The elapsed wait is +`IDLE_RECLAIM_REARM_MS << backoff_shift` — the *same* shift that prices the +activity arm — so an unproductive full doubles it: 15 s, 30 s, 60 s, 120 s, +240 s. And the arm is **disarmed entirely at `IDLE_RECLAIM_MAX_BACKOFF_SHIFT`** +rather than merely slowed, because five unproductive attempts establish there is +nothing to give and an idle process must not pay a whole-heap mark forever. +A productive full resets the shift, so a heap still returning memory keeps being +asked every 15 s — which is the case this exists for. `IDLE_RECLAIM_REARM_MS` is +deliberately larger than `IDLE_RECLAIM_MIN_INTERVAL_MS` so the rate floor is +never the binding constraint and the two gates cannot be confused in a diag. + +Two tests, each sabotage-proved: a parked heap with **no** external collection +anywhere gets a second attempt at the wait and not before, identified by reason +rather than by attempt count; and an unproductive streak doubles the wait each +time and then stops. Removing the arm fails the first, removing the backoff +scaling fails the second's "must not re-arm before the doubled wait", and +removing the disarm fails its "at the maximum shift the elapsed arm is +disarmed". + +The young half of the loss is **not** addressed here and is measured, not +assumed: after R's single reclaim, `[gc-general-reclaim] examined=66 released=0 +has_live=39 aging=22` — 39 of 66 arena blocks hold a live object, against 3 of +65 in A, and only an evacuation can consolidate those. Whether an idle young +evacuation is also needed is a separate question and a separate change. diff --git a/changelog.d/9840-regexp-header-nursery.md b/changelog.d/9840-regexp-header-nursery.md new file mode 100644 index 0000000000..f556d23f71 --- /dev/null +++ b/changelog.d/9840-regexp-header-nursery.md @@ -0,0 +1,35 @@ +### Performance + +- **A `RegExp` header is allocated in the nursery instead of the malloc arm.** + A JS regex literal evaluates to a fresh `RegExp` every time it is reached, and + `js_regexp_new` allocated each header with `gc_malloc`. On the claude-code TUI + that is, per 400-character reply (`PERRY_GC_TRACE`), **199,873 of 199,926 + malloc-tracked GC allocations — 100.0 %**, 80 bytes each, 99.2 % of them + freed, with the malloc registry swinging **101,929 entries down to 1,689** + across a single minor. Every one of those paid a mimalloc allocation, a push + onto `MALLOC_STATE.objects`, an insert into the malloc-registry `PtrHashSet` + (which rehashes as it grows), and at death a malloc-sweep visit and a free — + old-generation prices for an object that overwhelmingly dies young. + + Nothing required the malloc arm. `GC_TYPE_REGEXP` is already declared + `ArenaOrMalloc` and movable; `GcMoveHookKind::RegExpSideTables` already rekeys + `REGEX_POINTERS`, `REGEX_SOURCE_TABLE` and the expando owner after evacuation, + and `GcLayoutSlotKind::RegExpFields` already traces the header's two string + edges and its `meta` record. What kept production on `gc_malloc` was + finalization: the copying minor's from-space flip runs no per-object finalize + hooks, so a nursery header that died young would leak its `Arc` programs and + its registry entries. That is now handled exactly as `Map`, `Set` and `Error` + handle theirs — `finalize_dead_copied_minor_from_space_regexps` after a copied + minor, `collect_dead_registered_regexps_post_trace` at sweep entry for the + non-copying cycle kinds, and the ordinary old-generation sweep for a header + that has been promoted. + + Every regex program cache (`REGEX_CACHE`, `FANCY_CACHE`, + `REPEAT_MATCHER_CACHE`, `VALIDATED_PATTERNS`, the site cache) keys on pattern + and flags CONTENT, not on the header address, so a moving header costs them + nothing. + + Note that this **changes the collection schedule** rather than only removing + work: the `MallocCount` trigger loses essentially all of its input on this + workload, while ~16 MB per reply moves into the nursery. The schedule is + reported with the change rather than assumed unchanged. diff --git a/changelog.d/9845-gc-page-class-direct-table.md b/changelog.d/9845-gc-page-class-direct-table.md new file mode 100644 index 0000000000..ddd333451d --- /dev/null +++ b/changelog.d/9845-gc-page-class-direct-table.md @@ -0,0 +1,63 @@ +**The page-generation cache becomes a direct-indexed table over the arena's +1 MiB address classes, so a classification is a bounds compare and one load +instead of a four-way probe that missed one call in five.** + +`classify_heap_generation` and `classify_heap_space_in_range` sit under three +callers with no cheaper predicate of their own. The write barrier's +`remembered_child_needs_tracking` runs **35,871,391 times per turn** on the +compiled claude-code TUI and **95.23 %** of those take its cheapest arm — one +cached classification and a compare — so there is no barrier predicate left to +fix: what remains after the predicate is already optimal is the classification +itself. `mark_addr` (233 of 760 `classify*` leaf samples) and the side-table +prunes pay the same cost. + +The structure in front of the authoritative `PageGenerationMap` was a **4-way +round-robin set**. Measured with a dedicated counter on a 3300-char streaming +reply: **440 M lookups per turn at 20.0–21.6 % miss**, with **59.7–61.8 % of +misses on a key evicted within the last 64 evictions** — capacity, not conflict — +against a working set of **402–432 registered classes**. `ways_distinct_max` was +4, so every way was already in use and the shortfall is ~120x. + +**Widening it was not an option, and the reason is on the record.** #7469 +measured 16 ways as an **8.6 % regression** on the same row (0/7 pairs) for 1.5 % +fewer misses, and five further associativity changes measured flat. The rule +those produced — *associativity pays only when a miss is expensive* — says that a +miss which is just a hash lookup wants the cache to become **unnecessary**, not +larger. + +It can be. The registered classes occupy a span of **1,018–1,021 classes at +~40 % density**, so a table over that span holds every one of them in **160 KB** +and answers with one bounds compare and one load. `PageGenerationMap` stays +authoritative and every miss falls through to it exactly as before; the change is +confined to `PageGenerationCacheSet` and its two callers. + +Four things the measurement did not settle, each handled explicitly and each +pinned by a test that fails when its guard is removed — a wrong answer here is a +misclassified pointer, so none of them is left to inference: + +* **The base moves per process** (`0x43daa2` vs `0x57e3c2` on two runs — ASLR). + It is taken from the first insert, never compiled in. +* **The span can grow** (1,018 → 1,021 across two runs of one binary). An insert + outside the table rebases it, up to a 16,384-class cap; past the cap the key is + left uncached and falls through to the map rather than being mis-indexed. +* **The sizing is not obvious.** With base `first_key - S` and a table of `N`, + the span covered is `min(S + 1, N - S)`, maximised at `S = N / 2`. The natural + pairing `N = 4096, S = 1024` covers **1,025** classes — four above the measured + span — while `S = N / 2` covers **2,048** for the same memory. A `const` assert + now fails the build for any pairing covering less than twice the measured span. +* **A key match is not an address match.** A class can hold more than one range, + so a hit still requires `range.contains(addr)`. + +Invalidation is an epoch bump: O(1), and the same "clear everything" contract the +4-way set met by being reset wholesale. That contract matters more here, because +the table holds ~2,000 entries where the set held 4 — a missing invalidation the +old structure survived by luck would be a live misclassification — so all three +`PageGenerationMap` mutation sites were enumerated and each ends with an +unconditional `invalidate_generation_cache()`. + +The arm is a plain `u8` field in the set's first cache line rather than the env +`OnceLock`: this path runs 440 M times per turn, and an acquire load on each +would have been charged to both arms of the A/B — hiding it in the comparison +that was meant to isolate it — while still being paid against main. +`PERRY_GC_PAGE_CLASS_TABLE=0` restores the 4-way set in the same binary, which is +how the numbers above and below were taken. diff --git a/changelog.d/9864-for-in-callback-roots.md b/changelog.d/9864-for-in-callback-roots.md new file mode 100644 index 0000000000..3c6dc52939 --- /dev/null +++ b/changelog.d/9864-for-in-callback-roots.md @@ -0,0 +1 @@ +- Keep `for…in` receivers, accumulated keys, and Proxy descriptor state rooted across Proxy callbacks and moving garbage collection. Fixes stale pointers when Solid's universal renderer enumerates reactive spread properties. diff --git a/changelog.d/9865-solid-jsx.md b/changelog.d/9865-solid-jsx.md new file mode 100644 index 0000000000..9db293fc1b --- /dev/null +++ b/changelog.d/9865-solid-jsx.md @@ -0,0 +1 @@ +- Add opt-in `perry.jsx: "solid"` compilation for native Solid JSX, with reactive properties and children, components, keyed control flow, conditional widget identity, spreads, references, and fragments. Provide JSX types and examples in `perry-solid`, and compare native compilation with Solid's official universal JSX transform in the release fixture. diff --git a/changelog.d/9868-formdata-upload.md b/changelog.d/9868-formdata-upload.md new file mode 100644 index 0000000000..63c3e3402b --- /dev/null +++ b/changelog.d/9868-formdata-upload.md @@ -0,0 +1,4 @@ +### Fixed + +- Preserve `Blob` and `File` entries in `FormData`, and serialize `FormData` + request bodies with multipart bytes and a generated `content-type` header. diff --git a/changelog.d/9869-visited-levels-rooting.md b/changelog.d/9869-visited-levels-rooting.md new file mode 100644 index 0000000000..bfb86a3f53 --- /dev/null +++ b/changelog.d/9869-visited-levels-rooting.md @@ -0,0 +1,17 @@ +#9869: `for-in`'s deferred shadow set recorded each walked prototype level as a plain +NaN-boxed `f64` in `VisitedLevels`, and dereferenced it later in +`build_shadow_set` → `mark_own_names` → `js_object_get_own_property_names`. + +Between the `visited.push(current)` at level *N* and that read, the walk crosses +`js_object_keys_value` (which allocates an array) and +`js_object_get_prototype_of` (which can run a Proxy `getPrototypeOf` trap, i.e. +arbitrary user JS). Either can collect and move the recorded object, so the +stored word is a stale pointer whenever a collection lands in that window — +the same defect #9864 fixes for `out` and `current`, in the one place its patch +did not reach because the deferred-shadow-set rework landed after it was +written. + +`VisitedLevels` now stores `RuntimeHandle`s, which the collector rewrites in +place, and `VisitedSlice::iter` reads each level fresh from its handle. +`RuntimeHandle` is `Copy`, so the inline arm still costs no allocation and the +"no malloc per `for-in`" property the rework was built for is preserved. diff --git a/changelog.d/gc-churn-attribution-diag.md b/changelog.d/gc-churn-attribution-diag.md new file mode 100644 index 0000000000..f4a6a2785d --- /dev/null +++ b/changelog.d/gc-churn-attribution-diag.md @@ -0,0 +1,22 @@ +### Runtime + +- `PERRY_GC_DIAG=1` now says WHY the collector ran, not only what it did: + `[gc-trigger]` prints every predicate input at each collection decision + (armed arena trigger vs `arena_total`, from-space vs the nursery cap, + old-gen reclaimable pressure vs baseline/band, the malloc pair, the + pending/retaining flags); `[gc-full]` names the arm behind every full + mark-sweep with a per-site count; `[gc-budgeted] start/done` reports each + incremental cycle's steps, per-phase step time and root-scan share; + `[gc-charge]` attributes mutator-assist and synchronous-full time to the + calling site (return-address chain resolved to the JS display name); + `[gc-survival]` gives, per copying minor, which root first reached each + surviving byte — shadow stack, native stack map, a named side-table + scanner, or the remembered set split by the old parent's type — with + transitive reach charged to the originating root. +- `PERRY_ALLOC_SITE_SAMPLE=` (arena/alloc_sample.rs): byte-proportional + allocation-site sampling for the GC arena, covering the runtime allocators + and the codegen inline bump path (the mirrored inline block limit is capped + at one interval while sampling). `[alloc-site]` reports bytes by object type + and the top sites after each copying minor and at exit. Off by default; one + relaxed atomic load per allocation when off; the OFF state and the magnitude + parse are pinned in `gc/tests/env_knob_parse.rs`. diff --git a/changelog.d/native-instance-binding-scope-9847.md b/changelog.d/native-instance-binding-scope-9847.md new file mode 100644 index 0000000000..8ff1dd9f42 --- /dev/null +++ b/changelog.d/native-instance-binding-scope-9847.md @@ -0,0 +1,34 @@ +**A native-instance tag created by a bare assignment now follows the binding it +assigns to, not the identifier's spelling** (#9847). One +`O = childProcess.spawn(...)` in a bundled helper used to type *every* binding +named `O` in the module as a `child_process` instance. + +`lower_assign` registered the tag through `push_module_native_instance`, whose +key is the identifier text and whose scope is the whole module, and its +class-name match ends in a catch-all — so any method on any native module, +assigned to a variable, claimed that name for the rest of the program. Minified +bundles reuse single letters everywhere, so the collision is the normal case, +not a corner: `cli_2.1.112.js` (claude-code) compiles as one module, contains +`let O; try { O = fA1.spawn(z.file, z.args, z.options) }`, and binds the name +`O` 5,381 more times. Among them is the `for (let { segment: O } of ... )` +binding in `string-width`, which holds a grapheme **string**; its +`O.codePointAt(0)` lowered as +`NativeMethodCall { module: "child_process", class_name: Some("Instance") }` +and reached the right answer only because native-instance dispatch falls +through to a generic path on a string receiver — once per grapheme, in the +loop that dominates a claude-code turn. + +The tag is now keyed on the `LocalId` the assignment target resolves to. That +is the same correction #7775 made for `new Proxy` bindings (`proxy_locals` → +`proxy_local_ids`) and it keeps the cross-function reach the module-wide table +existed for: a module-level `let client;` assigned inside `init()` and read +inside `handler()` resolves to the same binding in both. A target that resolves +to no local at all — a bare global — still falls back to the name-keyed table, +which is strictly no worse than the previous behaviour that used it for +everything. + +This is a mis-typing fix, not a wrong-answer fix: the mislowered calls already +degraded gracefully. But they degraded through a dispatch that had no business +seeing them, and a `child_process` method table that ever gained a +`codePointAt`, `length` or `test` entry would have captured string calls +silently. diff --git a/changelog.d/regex-backtracking-cliff.md b/changelog.d/regex-backtracking-cliff.md new file mode 100644 index 0000000000..f1b388b8e8 --- /dev/null +++ b/changelog.d/regex-backtracking-cliff.md @@ -0,0 +1,43 @@ +### Performance + +- **A capture group no longer turns a pattern into a ReDoS.** + `repeat_matcher::capture_layout` takes a pattern off the linear `regex` + engine when ECMA-262's RepeatMatcher capture semantics are observable — a + capture group directly under a quantifier, or a capture inside a negative + lookaround. That routing is a correctness requirement (the linear engine + keeps the last value of a capture nested in a quantified group; the spec + clears it on every iteration), but the engine it routes to, `regress`, is a + classical backtracker with no step budget. So adding parentheses was enough + to fall off a linear-time path onto an exponential one: + + | pattern | node | perry (before) | perry (after) | + |---|---|---|---| + | `/^(a+)+$/.test("a"×28 + "!")` | 4,798 ms | **16,522 ms** | **0 ms** | + | `/^(?:a+)+$/.test(…)` (same language, no capture) | 4,288 ms | 0 ms | 0 ms | + + **6.3 %** of the 4,463 distinct regex literals across seven real bundles + take that route — claude-code 7.1 %, dayjs 25 %, luxon 29 % — including + shapes like `^[a-z][a-z0-9]*(-[a-z0-9]+)*$`. + + The two engines accept exactly the same LANGUAGE for a pattern they both + compile; they disagree only about which capture assignment to report. So the + linear program is asked first (`linear_rules_out_match`), and when it proves + there is no match at or after the search offset — which is what every ReDoS + input is, a subject that ALMOST matches and then fails — the backtracker is + never entered. Every `&str`-subject entry point goes through + `lookup_repeat_matcher_for`: `test`, `exec`, `match`, `matchAll`, `search`, + `split` and `replace` with a string replacement. The gate disables itself + where the linear engine has no opinion (a pattern it could not compile holds + the never-match placeholder), which is exactly the lookaround shapes. + + **This removes the reachable exponential case; it does not BOUND the worst + case.** A real step budget has to be counted by the backtracker, and + `regress` has none today (`fancy-regex`, by contrast, ships + `backtrack_limit: 1_000_000`). A 101-line patch adding one has been measured + — worst hostile search 51 s → 124 ms at a budget of 1,000,000, zero answers + changed across 13,389 real searches, upstream's own 544 tests unchanged — and + is open upstream as + [ridiculousfish/regress#177](https://github.com/ridiculousfish/regress/pull/177). + Until it lands and perry picks it up, do not read "cliff fixed" as "worst + case bounded". + (`quantified_capture_pattern_does_not_backtrack_on_a_non_matching_subject`) diff --git a/changelog.d/regex-borrowed-cache-keys.md b/changelog.d/regex-borrowed-cache-keys.md new file mode 100644 index 0000000000..76d06c09fa --- /dev/null +++ b/changelog.d/regex-borrowed-cache-keys.md @@ -0,0 +1,26 @@ +### Performance + +- **Probing the compiled-program caches no longer materialises the key.** The + three thread-local caches were `HashMap<(String, String), _>`, and + `HashMap::get` needs a `&(String, String)` — so **every probe allocated two + Strings and copied the pattern text into them**, on a path that runs once per + RegExp OBJECT, and a JS regex literal evaluates to a fresh object every time + it is reached. A native-churn census of the claude-code binary (2026-09-05) + put `js_regexp_test` → `lookup_repeat_matcher` → `build_and_install_programs` + at **6,044 MB of 8,334 MB of estimated allocation with zero live bytes** — + 73 % of all remaining native churn — split across the three probe sites: the + `get_or_compile_regex` probe (2,071 MB) and two `core::fmt::Formatter::pad` + frames (1,989 MB and 1,984 MB), which is what `.to_string()` on an `Arc` + lowers to. + + The caches are now keyed by `ProgramKey = (Arc, Arc)`. Every caller + that matters already holds those `Arc`s — `REGEX_SOURCE_TABLE` and + `regex::site_cache` share one allocation of a literal's text with every + header built from it — so a probe is two refcount increments and no + allocation at all. The two remaining `Arc::from` materialisations are on cold + paths: the syntax-error fallback in `js_regexp_new` (a pattern the linear + engine's parser refused, 7.7 % of real literals, once each) and + `RegExp.prototype.compile` (once per call from user code). + + Hashing still walks the pattern bytes; the allocation is what the census + measured and what this removes. diff --git a/changelog.d/regex-engine-prototype-switch.md b/changelog.d/regex-engine-prototype-switch.md new file mode 100644 index 0000000000..2d3e4f8a68 --- /dev/null +++ b/changelog.d/regex-engine-prototype-switch.md @@ -0,0 +1,31 @@ +### Internal + +- **`PERRY_REGEX_ENGINE=regress` — a measurable tier-0 engine prototype.** + Routes every pattern through `regress` (the ECMAScript backtracker perry + already links for RepeatMatcher capture semantics) instead of only the ones + whose capture semantics require it, and installs a shared never-match + placeholder as the standard program so no NFA is built. Every exec-family + entry point already consults the repeat matcher first, so this exercises the + whole engine surface — `exec`, `test`, `match`, `matchAll`, `search`, + `split`, `replace` — without a second implementation. + + It exists so the engine question is settled on measurements from a real + binary rather than on a corpus harness. Measured over 4,463 distinct regex + literals extracted from seven real bundles (two claude-code builds, ethers, + moment, dayjs, luxon, mongodb) with a tracking allocator and the programs + held live: + + | engine | accepted | compile µs (med) | bytes/program (med) | corpus total | + |---|---|---|---|---| + | `regex` crate (tier 1 today) | 92.3 % | 48.5 | 12,492 | 136.7 MB | + | `regress` | **100 %** | **2.2** | **512** | **4.9 MB** | + | `fancy-regex` (tier 2 today) | 97.8 % | 59.2 | 12,623 | 146.6 MB | + + node/V8, measured the same session, is ~2,600 bytes per program. A + differential over 4,119 patterns × 13 subjects (53,547 comparisons of match + presence, span and every capture span) found **0 disagreements** between the + linear engine and `regress`. + + **Not a supported configuration**: the backtracker has no step budget, so a + pathological pattern can run unbounded. Off by default, one relaxed atomic + load when unset. diff --git a/crates/perry-codegen/src/lower_call/options/fetch.rs b/crates/perry-codegen/src/lower_call/options/fetch.rs index 72eaee489f..63e1f6dd84 100644 --- a/crates/perry-codegen/src/lower_call/options/fetch.rs +++ b/crates/perry-codegen/src/lower_call/options/fetch.rs @@ -664,6 +664,11 @@ pub(in crate::lower_call) fn lower_fetch_native_method( } else { double_literal(f64::from_bits(crate::nanbox::TAG_UNDEFINED)) }; + let filename = if args.len() >= 3 { + lower_expr(ctx, &args[2])? + } else { + double_literal(f64::from_bits(crate::nanbox::TAG_UNDEFINED)) + }; let runtime_fn = if method == "append" { "js_form_data_append" } else { @@ -672,7 +677,12 @@ pub(in crate::lower_call) fn lower_fetch_native_method( ctx.block().call( DOUBLE, runtime_fn, - &[(DOUBLE, &handle), (DOUBLE, &name), (DOUBLE, &value)], + &[ + (DOUBLE, &handle), + (DOUBLE, &name), + (DOUBLE, &value), + (DOUBLE, &filename), + ], ); return Ok(Some(double_literal(f64::from_bits( crate::nanbox::TAG_UNDEFINED, diff --git a/crates/perry-codegen/src/runtime_decls/strings_part2.rs b/crates/perry-codegen/src/runtime_decls/strings_part2.rs index 97d14748e1..5017076121 100644 --- a/crates/perry-codegen/src/runtime_decls/strings_part2.rs +++ b/crates/perry-codegen/src/runtime_decls/strings_part2.rs @@ -1052,8 +1052,16 @@ pub(crate) fn declare_phase_b_strings_part2(module: &mut LlModule) { module.declare_function("js_response_bytes", I64, &[DOUBLE]); module.declare_function("js_response_form_data", I64, &[DOUBLE]); module.declare_function("js_form_data_new", DOUBLE, &[]); - module.declare_function("js_form_data_append", DOUBLE, &[DOUBLE, DOUBLE, DOUBLE]); - module.declare_function("js_form_data_set", DOUBLE, &[DOUBLE, DOUBLE, DOUBLE]); + module.declare_function( + "js_form_data_append", + DOUBLE, + &[DOUBLE, DOUBLE, DOUBLE, DOUBLE], + ); + module.declare_function( + "js_form_data_set", + DOUBLE, + &[DOUBLE, DOUBLE, DOUBLE, DOUBLE], + ); module.declare_function("js_form_data_delete", DOUBLE, &[DOUBLE, I64]); module.declare_function("js_form_data_get", DOUBLE, &[DOUBLE, I64]); module.declare_function("js_form_data_get_all", DOUBLE, &[DOUBLE, I64]); diff --git a/crates/perry-hir/src/lib.rs b/crates/perry-hir/src/lib.rs index 5e416db78c..e1fa2b3308 100644 --- a/crates/perry-hir/src/lib.rs +++ b/crates/perry-hir/src/lib.rs @@ -25,6 +25,7 @@ pub(crate) mod lower_patterns; pub(crate) mod lower_types; pub mod monomorph; pub mod native_profile; +pub mod solid_jsx; pub mod stable_hash; pub mod type_alias_resolve; pub mod types; diff --git a/crates/perry-hir/src/lower/context.rs b/crates/perry-hir/src/lower/context.rs index 0f392b3e71..0a5826a26d 100644 --- a/crates/perry-hir/src/lower/context.rs +++ b/crates/perry-hir/src/lower/context.rs @@ -150,6 +150,7 @@ impl LoweringContext { namespace_vars: Vec::new(), current_namespace: None, module_native_instances: Vec::new(), + local_id_native_instances: HashMap::new(), uses_fetch: false, uses_webassembly: false, react_default_import_local: None, @@ -1639,10 +1640,41 @@ impl LoweringContext { // zero-arg FFI calls. .filter(|(_, module, class)| !exposes_plain_object_fields(module, class)) .map(|(_, module, class)| (module.as_str(), class.as_str())) + .or_else(|| { + // #9847: a bare assignment (`O = cp.spawn(...)`) tags the + // RESOLVED binding, not the spelling. Consulted before the + // name-keyed module-wide table below, so a same-named binding + // in another function is simply a different binding and cannot + // inherit the tag. A different binding has a different + // `LocalId`, so no scope-exit truncation is needed here. + // + // This arm sits on the miss path of every identifier property + // access, so short-circuit the common module that has no + // bare-assignment native handle at all before touching the + // locals index. + if self.local_id_native_instances.is_empty() { + return None; + } + let id = self.lookup_local(name)?; + self.local_id_native_instances + .get(&id) + .filter(|(module, class)| !exposes_plain_object_fields(module, class)) + .map(|(module, class)| (module.as_str(), class.as_str())) + }) .or_else(|| { // Check module-level instances (survive scope exits). // Same last-match-wins rule for consistency — the index stores // the LAST pushed entry per name. + // + // #9847 KNOWN HOLE, stated plainly (the same one #7775 left for + // proxies): a receiver that resolves to NO local — a bare + // global, or a module-level binding referenced from a function + // body lowered before that binding was pre-registered — is + // still answered by spelling alone. That arm is kept because + // dropping it would regress the genuine cross-function handles + // that reach the lowering only through a name, and it is + // strictly no worse than the pre-#9847 behaviour, which used it + // for every assignment. self.module_native_instances_index .get(name) .map(|&idx| &self.module_native_instances[idx]) @@ -1686,6 +1718,26 @@ impl LoweringContext { .insert(entry.0.clone(), idx); self.module_native_instances.push(entry); } + + /// #9847: tag the RESOLVED binding `id` as holding a native instance. + /// + /// Used by the bare-assignment path (`O = cp.spawn(...)`) in place of + /// `push_module_native_instance`, whose name key was module-wide: in a + /// minified single-module bundle a single native handle poisoned every + /// homonym in the program. Keyed on the `LocalId` the target resolves to, + /// this keeps the cross-function reach the module-wide table was there to + /// provide (a module-level `let client;` assigned inside `init()` and read + /// inside `handler()` resolves to the SAME id in both) while making a + /// same-named binding in another scope a different binding. + pub(crate) fn register_local_id_native_instance( + &mut self, + id: LocalId, + module_name: String, + class_name: String, + ) { + self.local_id_native_instances + .insert(id, (module_name, class_name)); + } } // Internal anchor — keeps the file's outer impl block intact while diff --git a/crates/perry-hir/src/lower/expr_assign.rs b/crates/perry-hir/src/lower/expr_assign.rs index c07f56e2bb..123f562e05 100644 --- a/crates/perry-hir/src/lower/expr_assign.rs +++ b/crates/perry-hir/src/lower/expr_assign.rs @@ -195,11 +195,48 @@ pub(super) fn lower_assign(ctx: &mut LoweringContext, assign: &ast::AssignExpr) _ => Some("Instance"), }; if let Some(class_name) = class_name { - ctx.push_module_native_instance(( - var_name.clone(), - module_name.to_string(), - class_name.to_string(), - )); + // #9847: tag the BINDING this assigns + // to, not its spelling. The catch-all + // arm above makes any method on any + // native module tag the target, and + // `push_module_native_instance` keyed + // that on the identifier text for the + // whole module — so one + // `O = cp.spawn(...)` in a bundled + // helper typed all 5,381 bindings named + // `O` in claude-code's single-module + // bundle as `child_process::Instance`, + // including a `for (let {segment: O} of + // …)` binding holding a grapheme + // string. When the target resolves to a + // local we key on its `LocalId`, which + // keeps the cross-function reach the + // module-wide table existed for (a + // module-level `let client;` assigned + // inside one function and read inside + // another resolves to the same id in + // both) without the homonym collision. + // An unresolvable target — a bare + // global with no binding — has no id to + // key on and keeps the old name-keyed + // registration; see the matching arm in + // `lookup_native_instance`. + match ctx.lookup_local(&var_name) { + Some(local_id) => { + ctx.register_local_id_native_instance( + local_id, + module_name.to_string(), + class_name.to_string(), + ); + } + None => { + ctx.push_module_native_instance(( + var_name.clone(), + module_name.to_string(), + class_name.to_string(), + )); + } + } } } } diff --git a/crates/perry-hir/src/lower/lowering_context.rs b/crates/perry-hir/src/lower/lowering_context.rs index d96b0c504e..f69b838286 100644 --- a/crates/perry-hir/src/lower/lowering_context.rs +++ b/crates/perry-hir/src/lower/lowering_context.rs @@ -447,7 +447,29 @@ pub struct LoweringContext { pub(crate) current_namespace: Option, /// Module-level native instances that survive scope exits. /// Used for variables assigned from native calls inside functions (e.g., `mongoClient = await MongoClient.connect(uri)`). + /// + /// NAME-keyed and module-wide, so it is no longer authoritative for the + /// bare-assignment form: see `local_id_native_instances`, which supersedes + /// it whenever the assignment target resolves to a binding (#9847). This + /// stays the fallback for a target that resolves to no local at all. pub(crate) module_native_instances: Vec<(String, String, String)>, + /// #9847: the RESOLVED bindings that hold a native instance produced by a + /// bare assignment (`O = cp.spawn(...)`), so "this binding is a + /// `child_process` handle" stops meaning "something in this module spells + /// it this way". + /// + /// `module_native_instances` alone tagged the NAME for the whole module and + /// was never truncated, so one `O = cp.spawn(...)` in a bundled helper made + /// every other `O` in a 13 MB single-module bundle a + /// `child_process::Instance` — including a `for (let {segment: O} of …)` + /// binding holding a grapheme string, whose `O.codePointAt(0)` then lowered + /// as `NativeMethodCall{module:"child_process", class_name:Some("Instance")}` + /// and only worked because native-instance dispatch falls through to a + /// generic path on a string receiver. Same defect class as #7775's + /// `proxy_locals` → `proxy_local_ids`, and keyed the same way: a different + /// binding has a different `LocalId`, so a stale entry can never be reached + /// through a homonym, and no scope-exit truncation is needed. + pub(crate) local_id_native_instances: HashMap, /// Whether this module uses fetch() — requires perry-stdlib pub(crate) uses_fetch: bool, /// Issue #76 — set when any `WebAssembly.*` HIR variant is lowered. diff --git a/crates/perry-hir/src/solid_jsx.rs b/crates/perry-hir/src/solid_jsx.rs new file mode 100644 index 0000000000..67910401de --- /dev/null +++ b/crates/perry-hir/src/solid_jsx.rs @@ -0,0 +1,518 @@ +//! Solid universal JSX expansion before ordinary closure/accessor HIR lowering. +//! +//! Native nodes are constructed once. Property getters and child accessors keep +//! signal reads inside Solid effects; component render-prop functions stay values. + +use std::collections::BTreeSet; + +use swc_common::{Spanned, DUMMY_SP}; +use swc_ecma_ast as ast; +use swc_ecma_visit::{Visit, VisitMut, VisitMutWith, VisitWith}; + +/// Expand JSX for an explicitly selected universal renderer. Returns `None` +/// without cloning when the module contains no JSX. +pub fn lower_solid_jsx(module: &ast::Module, runtime: &str) -> Option { + #[derive(Default)] + struct Names { + names: BTreeSet, + jsx: bool, + } + impl Visit for Names { + fn visit_ident(&mut self, ident: &ast::Ident) { + self.names.insert(ident.sym.to_string()); + } + fn visit_jsx_element(&mut self, element: &ast::JSXElement) { + self.jsx = true; + element.visit_children_with(self); + } + fn visit_jsx_fragment(&mut self, fragment: &ast::JSXFragment) { + self.jsx = true; + fragment.visit_children_with(self); + } + } + let mut names = Names::default(); + module.visit_with(&mut names); + if !names.jsx { + return None; + } + let prefix = (0..) + .map(|n| format!("__perry_solid_{n}_")) + .find(|prefix| !names.names.iter().any(|name| name.starts_with(prefix))) + .expect("finite source identifiers leave a free helper prefix"); + let mut lowering = SolidJsx { + prefix, + next: 0, + helpers: BTreeSet::new(), + }; + let mut result = module.clone(); + result.visit_mut_with(&mut lowering); + if lowering.helpers.is_empty() { + return Some(result); + } + let specifiers = lowering + .helpers + .iter() + .map(|name| { + ast::ImportSpecifier::Named(ast::ImportNamedSpecifier { + span: DUMMY_SP, + local: ident(&format!("{}{name}", lowering.prefix)), + imported: Some(ast::ModuleExportName::Ident(ident(name))), + is_type_only: false, + }) + }) + .collect(); + result.body.insert( + 0, + ast::ModuleItem::ModuleDecl(ast::ModuleDecl::Import(ast::ImportDecl { + span: DUMMY_SP, + specifiers, + src: Box::new(ast::Str { + span: DUMMY_SP, + value: runtime.into(), + raw: None, + }), + type_only: false, + with: None, + phase: Default::default(), + })), + ); + Some(result) +} + +struct SolidJsx { + prefix: String, + next: usize, + helpers: BTreeSet, +} + +fn ident(name: &str) -> ast::Ident { + ast::Ident::new(name.into(), DUMMY_SP, Default::default()) +} + +fn string(value: &str) -> ast::Expr { + ast::Expr::Lit(ast::Lit::Str(ast::Str { + span: DUMMY_SP, + value: value.into(), + raw: None, + })) +} + +fn call(callee: ast::Expr, args: Vec) -> ast::Expr { + ast::Expr::Call(ast::CallExpr { + callee: ast::Callee::Expr(Box::new(callee)), + args: args.into_iter().map(|expr| expr.into()).collect(), + ..Default::default() + }) +} + +fn arrow(value: ast::Expr) -> ast::Expr { + ast::Expr::Arrow(ast::ArrowExpr { + body: Box::new(ast::BlockStmtOrExpr::Expr(Box::new(value))), + ..Default::default() + }) +} + +fn statement(expr: ast::Expr) -> ast::Stmt { + ast::Stmt::Expr(ast::ExprStmt { + span: expr.span(), + expr: Box::new(expr), + }) +} + +fn binding(name: ast::Ident, value: ast::Expr) -> ast::Stmt { + ast::Stmt::Decl(ast::Decl::Var(Box::new(ast::VarDecl { + kind: ast::VarDeclKind::Const, + decls: vec![ast::VarDeclarator { + span: DUMMY_SP, + name: ast::Pat::Ident(name.into()), + init: Some(Box::new(value)), + definite: false, + }], + ..Default::default() + }))) +} + +fn block_expr(mut statements: Vec, result: ast::Expr) -> ast::Expr { + statements.push(ast::Stmt::Return(ast::ReturnStmt { + span: DUMMY_SP, + arg: Some(Box::new(result)), + })); + call( + ast::Expr::Arrow(ast::ArrowExpr { + body: Box::new(ast::BlockStmtOrExpr::BlockStmt(ast::BlockStmt { + stmts: statements, + ..Default::default() + })), + ..Default::default() + }), + vec![], + ) +} + +fn property(name: &str, value: ast::Expr, getter: bool) -> ast::PropOrSpread { + let key = ast::PropName::Str(ast::Str { + span: DUMMY_SP, + value: name.into(), + raw: None, + }); + let prop = if getter { + ast::Prop::Getter(ast::GetterProp { + span: value.span(), + key, + type_ann: None, + body: Some(ast::BlockStmt { + stmts: vec![ast::Stmt::Return(ast::ReturnStmt { + span: value.span(), + arg: Some(Box::new(value)), + })], + ..Default::default() + }), + }) + } else { + ast::Prop::KeyValue(ast::KeyValueProp { + key, + value: Box::new(value), + }) + }; + ast::PropOrSpread::Prop(Box::new(prop)) +} + +fn object(props: Vec) -> ast::Expr { + ast::Expr::Object(ast::ObjectLit { + span: DUMMY_SP, + props, + }) +} + +fn array(elements: Vec) -> ast::Expr { + ast::Expr::Array(ast::ArrayLit { + span: DUMMY_SP, + elems: elements.into_iter().map(|expr| Some(expr.into())).collect(), + }) +} + +fn is_static_value(expr: &ast::Expr) -> bool { + matches!( + expr, + ast::Expr::Lit(_) | ast::Expr::Arrow(_) | ast::Expr::Fn(_) + ) +} + +fn contains_jsx(expr: &ast::Expr) -> bool { + struct Find(bool); + impl Visit for Find { + fn visit_jsx_element(&mut self, _: &ast::JSXElement) { + self.0 = true; + } + fn visit_jsx_fragment(&mut self, _: &ast::JSXFragment) { + self.0 = true; + } + } + let mut find = Find(false); + expr.visit_with(&mut find); + find.0 +} + +fn boolean(expr: ast::Expr) -> ast::Expr { + ast::Expr::Unary(ast::UnaryExpr { + span: expr.span(), + op: ast::UnaryOp::Bang, + arg: Box::new(ast::Expr::Unary(ast::UnaryExpr { + span: expr.span(), + op: ast::UnaryOp::Bang, + arg: Box::new(expr), + })), + }) +} + +impl SolidJsx { + fn helper(&mut self, name: &str, args: Vec) -> ast::Expr { + self.helpers.insert(name.to_string()); + call( + ast::Expr::Ident(ident(&format!("{}{name}", self.prefix))), + args, + ) + } + + fn temporary(&mut self) -> ast::Ident { + let name = ident(&format!("{}node_{}", self.prefix, self.next)); + self.next += 1; + name + } + + fn expression(&mut self, mut expression: ast::Expr) -> ast::Expr { + expression.visit_mut_with(self); + expression + } + + fn getter_expression(&mut self, mut expression: ast::Expr) -> ast::Expr { + let condition = match &mut expression { + ast::Expr::Cond(cond) if contains_jsx(&cond.cons) || contains_jsx(&cond.alt) => { + Some(&mut cond.test) + } + ast::Expr::Bin(binary) + if binary.op == ast::BinaryOp::LogicalAnd && contains_jsx(&binary.right) => + { + Some(&mut binary.left) + } + _ => None, + }; + if let Some(condition) = condition { + let test = self.expression(*condition.clone()); + let memo = self.helper("memo", vec![arrow(boolean(test))]); + *condition = Box::new(call(memo, vec![])); + } + self.expression(expression) + } + + fn child_accessor(&mut self, expr: ast::Expr) -> ast::Expr { + // A truthy-to-truthy update must retain an existing conditional branch. + // Track the condition's boolean value separately from the branch factory. + let mut expr = expr; + let condition = match &mut expr { + ast::Expr::Cond(cond) if contains_jsx(&cond.cons) || contains_jsx(&cond.alt) => { + Some(&mut cond.test) + } + ast::Expr::Bin(binary) + if binary.op == ast::BinaryOp::LogicalAnd && contains_jsx(&binary.right) => + { + Some(&mut binary.left) + } + _ => None, + }; + let mut setup = Vec::new(); + if let Some(condition) = condition { + let value = self.expression(*condition.clone()); + let memo = self.helper("memo", vec![arrow(boolean(value))]); + let name = self.temporary(); + setup.push(binding(name.clone(), memo)); + *condition = Box::new(call(ast::Expr::Ident(name), vec![])); + } + let accessor = arrow(self.expression(expr)); + if setup.is_empty() { + accessor + } else { + block_expr(setup, accessor) + } + } + + fn element_name(&mut self, name: &ast::JSXElementName) -> (ast::Expr, bool) { + match name { + ast::JSXElementName::Ident(name) if name.sym.starts_with(char::is_lowercase) => { + (string(&name.sym), true) + } + ast::JSXElementName::Ident(name) => (ast::Expr::Ident(name.clone()), false), + ast::JSXElementName::JSXMemberExpr(member) => (Self::member(member), false), + ast::JSXElementName::JSXNamespacedName(name) => { + (string(&format!("{}:{}", name.ns.sym, name.name.sym)), true) + } + } + } + + fn member(member: &ast::JSXMemberExpr) -> ast::Expr { + ast::Expr::Member(ast::MemberExpr { + span: member.span, + obj: Box::new(match &member.obj { + ast::JSXObject::Ident(name) => ast::Expr::Ident(name.clone()), + ast::JSXObject::JSXMemberExpr(parent) => Self::member(parent), + }), + prop: ast::MemberProp::Ident(member.prop.clone()), + }) + } + + fn attribute_value(&mut self, value: &ast::JSXAttrValue) -> ast::Expr { + match value { + ast::JSXAttrValue::Str(value) => ast::Expr::Lit(ast::Lit::Str(value.clone())), + ast::JSXAttrValue::JSXExprContainer(container) => match &container.expr { + ast::JSXExpr::Expr(expr) => self.getter_expression(*expr.clone()), + ast::JSXExpr::JSXEmptyExpr(_) => ast::Expr::Ident(ident("undefined")), + }, + ast::JSXAttrValue::JSXElement(element) => self.element(element), + ast::JSXAttrValue::JSXFragment(fragment) => self.fragment(fragment), + } + } + + fn ref_value(&mut self, value: ast::Expr) -> ast::Expr { + let target = ast::AssignTarget::try_from(Box::new(value.clone())).ok(); + let node = self.temporary(); + let current = self.temporary(); + let invoke = call( + ast::Expr::Ident(current.clone()), + vec![ast::Expr::Ident(node.clone())], + ); + let action = if let Some(target) = target { + let assign = ast::Expr::Assign(ast::AssignExpr { + span: DUMMY_SP, + op: ast::AssignOp::Assign, + left: target, + right: Box::new(ast::Expr::Ident(node.clone())), + }); + ast::Expr::Cond(ast::CondExpr { + span: DUMMY_SP, + test: Box::new(ast::Expr::Bin(ast::BinExpr { + span: DUMMY_SP, + op: ast::BinaryOp::EqEqEq, + left: Box::new(ast::Expr::Unary(ast::UnaryExpr { + span: DUMMY_SP, + op: ast::UnaryOp::TypeOf, + arg: Box::new(ast::Expr::Ident(current.clone())), + })), + right: Box::new(string("function")), + })), + cons: Box::new(invoke), + alt: Box::new(assign), + }) + } else { + invoke + }; + let callback = ast::Expr::Arrow(ast::ArrowExpr { + body: Box::new(ast::BlockStmtOrExpr::BlockStmt(ast::BlockStmt { + stmts: vec![binding(current, value), statement(action)], + ..Default::default() + })), + ..Default::default() + }); + // Universal `use` invokes its callback untracked. Evaluating both the + // reference expression and its callback there avoids replaying refs when + // they happen to read a signal during widget construction. + let untracked = self.helper("use", vec![callback, ast::Expr::Ident(node.clone())]); + ast::Expr::Arrow(ast::ArrowExpr { + params: vec![ast::Pat::Ident(node.into())], + body: Box::new(ast::BlockStmtOrExpr::Expr(Box::new(untracked))), + ..Default::default() + }) + } + + fn child(&mut self, child: &ast::JSXElementChild, native: bool) -> Option { + match child { + ast::JSXElementChild::JSXText(text) => { + let text = crate::jsx::normalize_jsx_text(&text.value); + (!text.is_empty()).then(|| string(&text)) + } + ast::JSXElementChild::JSXElement(element) => Some(self.element(element)), + ast::JSXElementChild::JSXFragment(fragment) => Some(self.fragment(fragment)), + ast::JSXElementChild::JSXExprContainer(container) => match &container.expr { + ast::JSXExpr::JSXEmptyExpr(_) => None, + ast::JSXExpr::Expr(expr) => { + let value = *expr.clone(); + Some( + if native + && !is_static_value(&value) + && !matches!( + value, + ast::Expr::JSXElement(_) | ast::Expr::JSXFragment(_) + ) + { + self.child_accessor(value) + } else { + if native { + self.expression(value) + } else { + self.getter_expression(value) + } + }, + ) + } + }, + ast::JSXElementChild::JSXSpreadChild(child) => { + let expr = self.expression(*child.expr.clone()); + Some(if native { arrow(expr) } else { expr }) + } + } + } + + fn element(&mut self, element: &ast::JSXElement) -> ast::Expr { + let (name, native) = self.element_name(&element.opening.name); + let mut chunks = Vec::new(); + let mut props = Vec::new(); + let mut has_spread = false; + for attribute in &element.opening.attrs { + match attribute { + ast::JSXAttrOrSpread::SpreadElement(spread) => { + has_spread = true; + if !props.is_empty() { + chunks.push(object(std::mem::take(&mut props))); + } + let source = self.expression(*spread.expr.clone()); + chunks.push(arrow(source)); + } + ast::JSXAttrOrSpread::JSXAttr(attribute) => { + let key = match &attribute.name { + ast::JSXAttrName::Ident(name) => name.sym.to_string(), + ast::JSXAttrName::JSXNamespacedName(name) => { + format!("{}:{}", name.ns.sym, name.name.sym) + } + }; + let mut value = attribute + .value + .as_ref() + .map(|value| self.attribute_value(value)) + .unwrap_or_else(|| { + ast::Expr::Lit(ast::Lit::Bool(ast::Bool { + span: DUMMY_SP, + value: true, + })) + }); + if key == "ref" { + value = self.ref_value(value); + } + let getter = !is_static_value(&value); + props.push(property(&key, value, getter)); + } + } + } + let mut children = element + .children + .iter() + .filter_map(|child| self.child(child, native)) + .collect::>(); + if !children.is_empty() { + let children = if children.len() == 1 { + children.remove(0) + } else { + array(children) + }; + let getter = !native && !is_static_value(&children); + props.push(property("children", children, getter)); + } + if !props.is_empty() || chunks.is_empty() { + chunks.push(object(props)); + } + let props = if chunks.len() == 1 && !has_spread { + chunks.remove(0) + } else { + self.helper("mergeProps", chunks) + }; + if native { + let node = self.temporary(); + let create = self.helper("createElement", vec![name]); + let spread = self.helper("spread", vec![ast::Expr::Ident(node.clone()), props]); + block_expr( + vec![binding(node.clone(), create), statement(spread)], + ast::Expr::Ident(node), + ) + } else { + self.helper("createComponent", vec![name, props]) + } + } + + fn fragment(&mut self, fragment: &ast::JSXFragment) -> ast::Expr { + array( + fragment + .children + .iter() + .filter_map(|child| self.child(child, true)) + .collect(), + ) + } +} + +impl VisitMut for SolidJsx { + fn visit_mut_expr(&mut self, expression: &mut ast::Expr) { + match expression { + ast::Expr::JSXElement(element) => *expression = self.element(element), + ast::Expr::JSXFragment(fragment) => *expression = self.fragment(fragment), + _ => expression.visit_mut_children_with(self), + } + } +} diff --git a/crates/perry-hir/tests/native_instance_binding_scope.rs b/crates/perry-hir/tests/native_instance_binding_scope.rs new file mode 100644 index 0000000000..1d9fe3731d --- /dev/null +++ b/crates/perry-hir/tests/native_instance_binding_scope.rs @@ -0,0 +1,210 @@ +//! #9847 — a native-instance tag created by a bare assignment +//! (`O = cp.spawn(...)`) must follow the BINDING it assigns to, not the +//! identifier's spelling. +//! +//! Before the fix, `lower_assign` registered the tag through +//! `push_module_native_instance`, whose key is the identifier text and whose +//! scope is the whole module. `cli_2.1.112.js` (claude-code) compiles as ONE +//! module containing `let O; try { O = fA1.spawn(...) }`, and `O` is a binding +//! 5,381 times in that file — so every `O` in the 13 MB program was typed +//! `child_process::Instance`, including the `for (let {segment: O} of ...)` +//! binding that holds a grapheme STRING in the hottest loop of a turn. Its +//! `O.codePointAt(0)` lowered as +//! `NativeMethodCall{module:"child_process", class_name:Some("Instance")}` +//! and reached the right answer only because native-instance dispatch falls +//! through to a generic path on a string receiver — once per grapheme. +//! +//! Note the ORDER dependency these fixtures encode deliberately: the tag can +//! only reach a function lowered AFTER the poisoning assignment, so `spawner` +//! precedes `widthLike` in every fixture below. With the order reversed the +//! defect does not reproduce and the test could not fail. + +use perry_diagnostics::SourceCache; +use perry_hir::lower_module; +use perry_parser::parse_typescript_with_cache; + +fn lower(src: &str) -> perry_hir::Module { + let src = src.to_string(); + std::thread::Builder::new() + .stack_size(32 * 1024 * 1024) + .spawn(move || { + let mut cache = SourceCache::new(); + let parsed = + parse_typescript_with_cache(&src, "native_instance_binding_scope.ts", &mut cache) + .expect("parse should succeed"); + lower_module(&parsed.module, "test", "native_instance_binding_scope.ts") + .expect("lowering should succeed") + }) + .expect("spawn lower thread") + .join() + .expect("lower thread panicked") +} + +/// Debug-format just one function's body, so an assertion about `widthLike` +/// cannot be satisfied (or broken) by a node belonging to `spawner`. +fn body_of(module: &perry_hir::Module, name: &str) -> String { + let func = module + .functions + .iter() + .find(|f| f.name == name) + .unwrap_or_else(|| { + panic!( + "function `{name}` not found; module has {:?}", + module + .functions + .iter() + .map(|f| f.name.as_str()) + .collect::>() + ) + }); + format!("{:#?}", func.body) +} + +/// The whole defect and both halves of the contract in one module: a +/// module-level handle read from a second function, a function-local handle, +/// and an unrelated binding that merely shares the local handle's spelling. +/// +/// DO NOT REORDER `spawner` AND `widthLike` "for readability". The tag can only +/// reach a function lowered AFTER the assignment that creates it, so with +/// `widthLike` first the defect does not reproduce and +/// `a_same_named_binding_in_another_function_does_not_inherit_the_tag` passes +/// on the unfixed compiler — a test that cannot fail. This order was checked +/// against a pre-fix binary: `widthLike` lowered +/// `NativeMethodCall{module:"child_process", method:"codePointAt"}` there, and +/// the reversed order lowered the correct `PropertyGet`. +const FIXTURE: &str = r#" +import * as cp from "child_process"; + +let client: any; + +export function init(): void { + client = cp.spawn("true", []); +} + +export function handler(): void { + client.kill(); +} + +export function spawner(): any { + let O: any; + O = cp.spawn("false", []); + O.kill(); + return O; +} + +export function widthLike(q: any): number { + let Y = 0; + for (let { segment: O } of q) { + Y += O.codePointAt(0) >= 4352 ? 2 : 1; + } + return Y; +} +"#; + +#[test] +fn a_same_named_binding_in_another_function_does_not_inherit_the_tag() { + let module = lower(FIXTURE); + let width_like = body_of(&module, "widthLike"); + + assert!( + !width_like.contains("method: \"codePointAt\""), + "the for-of `segment` binding holds a string and must not lower its \ + `.codePointAt` through child_process native-instance dispatch just \ + because an unrelated function spells its spawn handle `O` too: \ + {width_like}" + ); + assert!( + width_like.contains("property: \"codePointAt\""), + "`O.codePointAt(0)` should lower as an ordinary property call: \ + {width_like}" + ); +} + +#[test] +fn a_function_local_native_handle_still_dispatches_natively() { + let module = lower(FIXTURE); + let spawner = body_of(&module, "spawner"); + + assert!( + spawner.contains("method: \"kill\""), + "the binding actually assigned from `cp.spawn(...)` must keep native \ + dispatch — scoping the tag to the binding must not lose the real \ + case: {spawner}" + ); + assert!( + spawner.contains("module: \"child_process\""), + "and it must still be tagged as child_process: {spawner}" + ); +} + +#[test] +fn a_module_level_handle_assigned_in_one_function_is_seen_in_another() { + let module = lower(FIXTURE); + let handler = body_of(&module, "handler"); + + // This is what the module-wide table existed for: `client` is bound at + // module level, assigned inside `init`, and read inside `handler`. Keyed + // on the resolved binding it reaches just as far, because both functions + // resolve `client` to the same LocalId. + assert!( + handler.contains("method: \"kill\"") && handler.contains("module: \"child_process\""), + "a module-level handle assigned in another function must still \ + dispatch natively: {handler}" + ); +} + +/// The issue's one-identifier A/B, as a test: two sources differing only in +/// whether the spawner's variable is spelled `O`. After the fix the two must +/// lower `widthLike` identically. +#[test] +fn renaming_the_spawner_variable_no_longer_changes_an_unrelated_function() { + const ARM: &str = r#" +import * as cp from "child_process"; + +export function unrelatedSpawner(): any { + let NAME: any; + try { NAME = cp.spawn("true", []); } catch (M) { NAME = null; } + return NAME; +} + +export function widthLike(q: any): number { + let Y = 0; + for (let { segment: O } of q) { + Y += O.codePointAt(0) >= 4352 ? 2 : 1; + } + return Y; +} +"#; + + // The rename is 3 characters longer, so every `byte_offset` in the file + // shifts. Those are source positions, not lowering decisions — normalise + // them, and nothing else, so the comparison is about the shape of the + // lowered code. (Before the fix this comparison failed on the node itself: + // `NativeMethodCall{module:"child_process", ...}` vs `Call{PropertyGet}`.) + let normalise = |body: String| { + body.split("byte_offset: ") + .enumerate() + .map(|(i, part)| { + if i == 0 { + return part.to_string(); + } + let rest = part.trim_start_matches(|c: char| c.is_ascii_digit()); + format!("byte_offset: N{rest}") + }) + .collect::() + }; + + let arm_a = normalise(body_of(&lower(&ARM.replace("NAME", "notO")), "widthLike")); + let arm_b = normalise(body_of(&lower(&ARM.replace("NAME", "O")), "widthLike")); + + assert_eq!( + arm_a, arm_b, + "renaming a spawn handle in an unrelated function must not change how \ + `widthLike` lowers" + ); + assert!( + !arm_b.contains("method: \"codePointAt\""), + "and neither arm may route the string read through native dispatch: \ + {arm_b}" + ); +} diff --git a/crates/perry-runtime/src/arena/allocators.rs b/crates/perry-runtime/src/arena/allocators.rs index 70c4a2979c..1e2e87c5e1 100644 --- a/crates/perry-runtime/src/arena/allocators.rs +++ b/crates/perry-runtime/src/arena/allocators.rs @@ -226,7 +226,7 @@ pub(crate) fn arena_alloc_old_excluding_pages( /// linear dedup scan that grows as the page fills. Allocation policy is /// deliberately UNCHANGED: the `old_free_take_exact` hole probe below stays, /// so this is a bookkeeping change only. See the flush discipline in -/// `arena/page_meta.rs`. +/// `arena/page_meta/`. pub fn arena_alloc_gc_old(size: usize, align: usize, obj_type: u8) -> *mut u8 { use crate::gc::{GcHeader, GC_FLAG_ARENA, GC_HEADER_SIZE}; diff --git a/crates/perry-runtime/src/arena/mod.rs b/crates/perry-runtime/src/arena/mod.rs index 12f68b2271..90a45f0c66 100644 --- a/crates/perry-runtime/src/arena/mod.rs +++ b/crates/perry-runtime/src/arena/mod.rs @@ -49,13 +49,14 @@ pub(crate) use block::{ /// allocation path uses instead of a per-access `_tlv_get_addr`. pub(crate) use block::{arena_hot_addr, hot_arena, hot_inline_state, inline_state_hot_addr}; #[cfg(test)] +#[cfg(test)] pub(crate) use block::{ block_pool_bytes_for_test, block_pool_explicit_drained_bytes_for_test, block_pool_put, force_next_block_alloc_failure, gc_trigger_arena_borrow_depth, gc_trigger_arena_calls, reset_gc_trigger_arena_probe, }; pub(crate) use page_meta::{ - address_span_overlaps_pages, defer_old_object_page_registration, + address_span_overlaps_pages, defer_old_object_page_registration, page_class_table_report, register_block_space_with_object_starts, register_old_object_pages, unregister_block_generation, unregister_old_block_pages, OLD_GEN_RECLAIM_POOLED_BYTES, OLD_GEN_RECLAIM_RETURNED_BYTES, OLD_GEN_RECLAIM_REUSABLE_BYTES, diff --git a/crates/perry-runtime/src/arena/page_meta.rs b/crates/perry-runtime/src/arena/page_meta/mod.rs similarity index 90% rename from crates/perry-runtime/src/arena/page_meta.rs rename to crates/perry-runtime/src/arena/page_meta/mod.rs index 0fe26f3c9b..708b16b90f 100644 --- a/crates/perry-runtime/src/arena/page_meta.rs +++ b/crates/perry-runtime/src/arena/page_meta/mod.rs @@ -138,55 +138,11 @@ impl PageGenerationCache { // that — an 8.6% regression on the same row (0/7 pairs). Keep the scan short. const PAGE_GENERATION_CACHE_WAYS: usize = 4; -/// Small direct-probed cache in front of [`PageGenerationMap`]. -/// -/// Pure accelerator: a miss, a stale way, or a full set all fall through to -/// the authoritative map, so the only thing correctness depends on is that -/// every invalidation clears **all** ways — which is why -/// [`invalidate_generation_cache`] resets the whole set rather than one entry. -/// -/// Stored behind an `UnsafeCell`, not a `Cell`: `Cell::get` returns a **copy**, -/// and copying ~200 bytes on every classification cost more than the map lookup -/// the cache exists to avoid (measured as a ~2% regression on `retain.ts` -/// before this was switched). Access is single-threaded by construction — the -/// cache is thread-local and no path holds a reference across a call that could -/// re-enter classification. -#[derive(Clone, Copy)] -struct PageGenerationCacheSet { - ways: [PageGenerationCache; PAGE_GENERATION_CACHE_WAYS], - /// Round-robin victim for the next insert. - next: usize, -} - -impl PageGenerationCacheSet { - const fn empty() -> Self { - Self { - ways: [PageGenerationCache::empty(); PAGE_GENERATION_CACHE_WAYS], - next: 0, - } - } - - #[inline(always)] - fn lookup(&self, key: usize, addr: usize) -> Option { - for way in self.ways.iter() { - if way.valid && way.key == key && way.range.contains(addr) { - return Some(way.range); - } - } - None - } +mod page_class; +pub(crate) use page_class::*; - #[inline] - fn insert(&mut self, key: usize, range: PageGenerationRange) { - let slot = self.next % PAGE_GENERATION_CACHE_WAYS; - self.ways[slot] = PageGenerationCache { - key, - range, - valid: true, - }; - self.next = slot.wrapping_add(1); - } -} +#[cfg(test)] +mod tests; /// #7187: this map used to carry a bespoke identity hasher (`write_usize` /// stored the key verbatim). `HashMap` is hashbrown, which takes the bucket @@ -418,7 +374,7 @@ pub(crate) fn generation_page_base(page: usize) -> usize { fn invalidate_generation_cache() { // Every way, not one — a stale way is exactly what this guards against. // SAFETY: thread-local, single-threaded. - PAGE_GENERATION_CACHE.with(|cache| unsafe { *cache.get() = PageGenerationCacheSet::empty() }); + PAGE_GENERATION_CACHE.with(|cache| unsafe { (*cache.get()).invalidate() }); } fn register_old_block_pages(base: usize, size: usize) { @@ -1795,108 +1751,6 @@ pub(crate) fn old_page_meta_for_tests(page: usize) -> Option { }) } -#[cfg(test)] -mod page_generation_hasher_tests { - use super::*; - use std::collections::HashSet; - use std::hash::BuildHasher; - - /// #7187 regression guard for `PageGenerationMap`'s hasher. - /// - /// `HashMap` is hashbrown: the bucket index comes from the hash's low bits, - /// but the SIMD control byte — the filter that decides whether a group - /// probe needs a real key comparison — is `hash >> 57`. Generation class - /// keys are `addr >> GENERATION_CLASS_SHIFT`, so an identity hasher (which - /// this map carried until #7187) produces a value around 2^26 whose top - /// seven bits are zero for **every** key in the table. Every occupied slot - /// in a probed group then matches, and each match costs a scattered load - /// plus a key comparison — on a lookup the write barrier performs several - /// times per heap store. - /// - /// This asserts the property directly rather than asserting "we call - /// `PtrHasher`": reinstating any non-mixing hasher collapses the control - /// byte to a single value and fails here. - #[test] - fn control_byte_is_spread_across_generation_class_keys() { - let map = PageGenerationMap::default(); - let build = map.hasher(); - - // Realistic 48-bit heap addresses, one per 1 MiB generation bucket — - // the exact key population `classify_heap_generation` looks up. - let base: usize = 0x0000_7f31_0000_0000; - let control_bytes: HashSet = (0..64) - .map(|i| { - let addr = base + i * (1usize << GENERATION_CLASS_SHIFT); - (build.hash_one(generation_class_key_for_addr(addr)) >> 57) & 0x7f - }) - .collect(); - - assert!( - control_bytes.len() >= 32, - "hashbrown control byte must vary across generation class keys, got {} \ - distinct values from 64 consecutive buckets (an identity hasher yields 1)", - control_bytes.len() - ); - } - - /// The bucket index (low bits) must stay well spread too — mixing that put - /// all the entropy in the high bits and left the low bits constant would - /// trade a control-byte collision for a far worse bucket collision. This is - /// the failure `fast_hash`'s `mix` step exists for. - #[test] - fn bucket_index_is_spread_across_generation_class_keys() { - let map = PageGenerationMap::default(); - let build = map.hasher(); - - let base: usize = 0x0000_7f31_0000_0000; - let low_bits: HashSet = (0..64) - .map(|i| { - let addr = base + i * (1usize << GENERATION_CLASS_SHIFT); - build.hash_one(generation_class_key_for_addr(addr)) & 0x3f - }) - .collect(); - - assert!( - low_bits.len() >= 32, - "bucket index must vary across generation class keys, got {} distinct \ - values from 64 consecutive buckets", - low_bits.len() - ); - } - - /// The map must still answer correctly after the hasher change — a - /// point-query round trip over many buckets, which is the only way this map - /// is ever used. - #[test] - fn point_queries_round_trip_across_many_buckets() { - let mut map = PageGenerationMap::default(); - let base: usize = 0x0000_7f31_0000_0000; - for i in 0..256usize { - let addr = base + i * (1usize << GENERATION_CLASS_SHIFT); - map.insert( - generation_class_key_for_addr(addr), - PageGenerationSlot::Single(PageGenerationRange { - base: addr, - end: addr + (1 << GENERATION_CLASS_SHIFT), - generation: HeapGeneration::Old, - space: HeapSpace::Old, - object_starts: std::ptr::null_mut(), - }), - ); - } - for i in 0..256usize { - let addr = base + i * (1usize << GENERATION_CLASS_SHIFT); - let found = map - .get(&generation_class_key_for_addr(addr)) - .and_then(|slot| slot.find(addr + 0x40)) - .expect("every inserted bucket must be found by point query"); - assert_eq!(found.generation, HeapGeneration::Old); - assert_eq!(found.base, addr); - } - assert_eq!(map.len(), 256); - } -} - /// `PERRY_GC_CENSUS`: estimated bytes held by the per-page side tables. pub(crate) fn page_meta_census() -> Vec { use crate::gc::census::{hash_table_bytes, vec_bytes}; @@ -1942,31 +1796,3 @@ pub(crate) fn page_meta_census() -> Vec { }); rows } - -#[cfg(test)] -mod block_range_tests { - use super::old_arena_block_range_index; - - /// `old_arena_block_range_index` is the whole reason #9772's selection can - /// group pages by block, so it gets a test that can fail: gaps between - /// blocks must not be attributed to the block below them. - #[test] - fn block_range_lookup_respects_gaps_and_ends() { - // Two 1 MiB blocks with a 1 MiB hole between them. - let ranges = vec![ - (0x1000_0000, 0x1010_0000, 7, 0x10_0000), - (0x1020_0000, 0x1030_0000, 9, 0x10_0000), - ]; - assert_eq!(old_arena_block_range_index(&ranges, 0x1000_0000), Some(0)); - assert_eq!(old_arena_block_range_index(&ranges, 0x100F_FFFF), Some(0)); - // One past the end of block 0 is the gap, not block 0. - assert_eq!(old_arena_block_range_index(&ranges, 0x1010_0000), None); - assert_eq!(old_arena_block_range_index(&ranges, 0x1018_0000), None); - assert_eq!(old_arena_block_range_index(&ranges, 0x1020_0000), Some(1)); - assert_eq!(old_arena_block_range_index(&ranges, 0x102F_FFFF), Some(1)); - // Above every block, and below every block. - assert_eq!(old_arena_block_range_index(&ranges, 0x1030_0000), None); - assert_eq!(old_arena_block_range_index(&ranges, 0x0FFF_FFFF), None); - assert_eq!(old_arena_block_range_index(&[], 0x1000_0000), None); - } -} diff --git a/crates/perry-runtime/src/arena/page_meta/page_class.rs b/crates/perry-runtime/src/arena/page_meta/page_class.rs new file mode 100644 index 0000000000..610aa5ba41 --- /dev/null +++ b/crates/perry-runtime/src/arena/page_meta/page_class.rs @@ -0,0 +1,645 @@ +//! The direct-indexed page-class table and the 4-way generation cache it +//! replaced (#9853), split out of `page_meta` for the 2000-line file cap. +//! +//! A child module, so `use super::*` sees the parent's private items +//! (`PageGenerationRange`, `PageGenerationSlot`, `HeapGeneration`, …) +//! without widening any visibility. + +use super::*; + +/// One entry of the direct-indexed table: the range last confirmed for this +/// 1 MiB class, stamped with the invalidation epoch it was confirmed under. +#[derive(Clone, Copy)] +struct PageClassEntry { + range: PageGenerationRange, + epoch: u64, +} + +impl PageClassEntry { + /// The filler every unwritten slot holds. `epoch: 0` is the sentinel no + /// live epoch ever takes (`epoch` starts at 1 and [`PageGenerationCacheSet:: + /// invalidate`] steps over 0 on wrap), so a freshly allocated table is + /// entirely dead without a second "valid" flag to keep coherent. + const DEAD: Self = Self { + range: PageGenerationCache::empty().range, + epoch: 0, + }; +} + +/// Initial span of the direct table, in 1 MiB classes, and how far below the +/// first registered key the base is placed. +/// +/// Measured on the compiled claude-code TUI: the live span is **1,018-1,021 +/// classes** at ~40 % density, with the base moving per process (ASLR). The +/// base is `first_registered_key - SLACK`, and the first registration can fall +/// anywhere in the eventual span, so both ends have to be covered by the two +/// constants alone. Writing `S = SLACK`, `N = SPAN` and `W = 1,021` for the +/// measured width, a table that covers every case without rebasing needs +/// +/// * `S >= W - 1` — otherwise a first registration at the TOP of the span +/// leaves the classes below the base uncovered; and +/// * `N > W - 1 + S` — otherwise a first registration at the BOTTOM leaves the +/// classes above `base + N` uncovered. +/// +/// Both must hold, so the width actually covered is +/// `W <= min(S + 1, N - S)` — **maximised at `S = N / 2`**, where it is `N / 2`. +/// That is the whole of the sizing argument, and it is worth writing down +/// because the obvious pairing gets it wrong: `N = 4096, S = 1024` pays for +/// 4,096 entries and covers a span of only **1,025** — four classes above the +/// measured 1,021, which is not a margin. `S = N / 2` covers **2,048** for the +/// same 4,096 entries: **twice the measured span at identical cost**. +/// +/// So `N = 4096, S = 2048`: 4,096 x 40 B = **160 KB** on each thread that +/// classifies, allocated only on that thread's first insert, covering any span +/// up to 2,048 classes wherever the first registration falls within it. +/// +/// Exceeding it is not a correctness problem — [`PageGenerationCacheSet:: +/// rebase_to_cover`] widens the table and the `rebases` counter says how often +/// that happened — so these are sized to make the rebase rare, not to make it +/// impossible. +const PAGE_CLASS_TABLE_INITIAL_SPAN: usize = 4096; +const PAGE_CLASS_TABLE_BASE_SLACK: usize = PAGE_CLASS_TABLE_INITIAL_SPAN / 2; + +/// The span these two constants actually cover, `min(S + 1, N - S)`, and the +/// compile-time guard that keeps the derivation above load-bearing rather than +/// decorative. The pairing this replaced (`N = 4096, S = 1024`) covers 1,025 — +/// four classes above the measured span — and fails this assert, which is the +/// point: the sizing is not obvious and a plausible-looking edit gets it wrong. +const PAGE_CLASS_TABLE_COVERED_SPAN: usize = { + let below = PAGE_CLASS_TABLE_BASE_SLACK + 1; + let above = PAGE_CLASS_TABLE_INITIAL_SPAN - PAGE_CLASS_TABLE_BASE_SLACK; + if below < above { + below + } else { + above + } +}; +/// Measured live span on the compiled claude-code TUI, worst of two runs. +const PAGE_CLASS_TABLE_MEASURED_SPAN: usize = 1021; +const _: () = assert!( + PAGE_CLASS_TABLE_COVERED_SPAN >= 2 * PAGE_CLASS_TABLE_MEASURED_SPAN, + "the initial table must cover at least twice the measured span, wherever \ + the first registration falls in it — otherwise the common case rebases" +); +/// Above this span the table stops growing and out-of-span keys simply fall +/// through to the authoritative map uncached. 16 GiB of address span is far +/// past any arena this runtime places; the cap exists so a stray registration +/// at a wild address cannot allocate an unbounded table. +const PAGE_CLASS_TABLE_MAX_SPAN: usize = 16 * 1024; + +/// Which arm [`PageGenerationCacheSet`] is running, resolved once per thread on +/// the first insert and then read as a PLAIN FIELD in the hot path. +/// +/// Not `page_class_table_enabled()` on the lookup path, deliberately: that is a +/// `OnceLock` and a `OnceLock` read is an ACQUIRE load. This path runs +/// **440 M times per turn** — an `ldar` plus a branch on every one of them is a +/// cost the table is supposed to be removing, and it would land on BOTH arms, +/// so the A/B would have hidden it while the comparison against main paid it. +/// The field shares the first cache line with `base`/`epoch`/`table`, which a +/// lookup loads anyway, so the arm test is free. +/// +/// `ARM_UNRESOLVED` behaves as the table arm and is CORRECT for both: before +/// the first insert the table is empty and every way is invalid, so either arm +/// answers "miss" for every key. +const ARM_UNRESOLVED: u8 = 0; +const ARM_TABLE: u8 = 1; +const ARM_WAYS: u8 = 2; + +/// The cache in front of [`PageGenerationMap`]: a **direct-indexed table** keyed +/// by `addr >> GENERATION_CLASS_SHIFT`, with the previous 4-way set retained +/// behind `PERRY_GC_PAGE_CLASS_TABLE=0` as the control arm. +/// +/// # Why a table and not a bigger cache +/// The 4-way set was measured (`PERRY_CLASSIFY_DIAG`, 3300-char claude-code +/// reply) at **440 M lookups per turn, 20 % miss, 60 % of those misses on a key +/// evicted within the last 64 evictions** — pure capacity, against a working +/// set of **402-432 registered classes**. All four ways were in use +/// (`ways_distinct_max = 4`), so the shortfall is ~120x, which no associativity +/// reaches; #7469 already measured 16 ways as an 8.6 % regression for 1.5 % +/// fewer misses, and five further associativity changes measured flat. The +/// registered classes sit in a span of **1,018-1,021** at ~40 % density, so a +/// table over the span holds every one of them in ~33 KB and answers a lookup +/// with one bounds compare and one load. The same bounds check rejects the +/// ~8,000 candidate addresses per turn that are in no registered block — the +/// other 22 % of misses — without a separate filter. +/// +/// # What it is not +/// A cache, not the truth. `PageGenerationMap` stays authoritative: every miss +/// falls through to it exactly as before, and every registration, unregistration +/// and retag invalidates the whole table by bumping `epoch` (O(1), and the same +/// "clear everything" contract the 4-way set had, for the same reason: a stale +/// entry is exactly what this guards against). A hit still requires +/// `range.contains(addr)` — a key match at a range boundary is not an address +/// match. +/// +/// # The one place the table is WEAKER than the set it replaces +/// A class can hold more than one range (`PageGenerationSlot::Multiple`). The +/// 4-way set could hold two of them at once, in two ways under the same key, +/// and hit on both; the table has one slot per class, so ranges sharing a +/// class evict each other and alternate accesses miss. This is a real +/// regression in kind, bounded by how many classes are `Multiple` — and it is +/// what the `[gc-page-class]` miss rate would show if the collapse predicted +/// below fails to appear. Registered blocks are `BLOCK_SIZE`-sized and +/// `BLOCK_SIZE == 1 << GENERATION_CLASS_SHIFT`, so one block is exactly one +/// class and the multi-range case is the sub-block registration, not the norm. +/// +/// # The two things measurement did not settle, handled explicitly +/// * **The base moves per process** (observed: `0x43daa2` vs `0x57e3c2` on two +/// runs). It is taken from the first insert, minus slack — never compiled in. +/// * **The span can grow** (observed: 1,018 vs 1,021 on two runs of one +/// binary). An insert outside `[base, base + len)` rebases the table to cover +/// it, up to `PAGE_CLASS_TABLE_MAX_SPAN`; past the cap the key is left +/// uncached and falls through. Both paths are pinned by tests that fail when +/// the fallback is removed, because a wrong answer here is a misclassified +/// pointer — a collector that moves the wrong thing. +/// +/// Stored behind an `UnsafeCell`, not a `Cell`, for the reason recorded on the +/// 4-way set when it was switched: `Cell::get` returns a **copy**, and copying +/// the set on every classification cost more than the map lookup the cache +/// exists to avoid (a ~2 % regression on `retain.ts`). That argument is +/// stronger here, not weaker — the table is far larger than the set was. +/// Access is single-threaded by construction: the cell is thread-local and no +/// path holds a reference across a call that could re-enter classification. +// `repr(C)` for field ORDER, not for FFI: the four fields a lookup touches are +// declared first so they share one cache line. Under `repr(Rust)` the layout is +// unspecified and the 192-byte `ways` array — dead weight in the table arm — +// may be placed in front of them, which would make the spec's "one bounds +// compare and one load" two lines' worth of traffic. `align(64)` is what makes +// that claim true rather than likely: at the struct's natural 8-byte alignment +// the hot group could straddle two lines depending on where the thread-local +// block lands. +#[repr(C, align(64))] +pub(super) struct PageGenerationCacheSet { + // ---- the table: everything `lookup` reads, in one line ---- + /// `ARM_UNRESOLVED` / `ARM_TABLE` / `ARM_WAYS`. See the constants above for + /// why the arm is a field and not the `OnceLock` read. + arm: u8, + /// First class covered. Meaningful only when `table` is non-empty. + base: usize, + /// Bumped on every invalidation; an entry is live only if its `epoch` + /// matches. Starts at 1 so a zeroed entry is never live. + epoch: u64, + /// Entries for classes `base .. base + table.len()`. + table: Vec, + /// Counted unconditionally (a field increment on a `&mut` we already hold) + /// and reported only under `PERRY_GC_DIAG`. This is the falsifier: the + /// table's whole claim is that the miss rate collapses. Both arms count, + /// so the control arm carries the same increment and the comparison is + /// symmetric. + hits: u64, + misses: u64, + // ---- cold: written on the miss path or rarer ---- + /// Misses the authoritative map could answer, i.e. misses that cached + /// something. `misses - inserts` is the population that is in no + /// registered block at all — the 22 % the bounds check is supposed to + /// reject for free. + inserts: u64, + /// Rebases performed and inserts refused past the cap — the two paths the + /// span measurement could not rule out. + rebases: u64, + refused: u64, + /// Lookups that missed because the key was OUTSIDE `[base, base + len)`. + /// See the increment site for why this is the counter that matters. + oos: u64, + // ---- control arm: the 4-way round-robin set, unchanged ---- + ways: [PageGenerationCache; PAGE_GENERATION_CACHE_WAYS], + /// Round-robin victim for the next insert. + next: usize, +} + +impl PageGenerationCacheSet { + pub(super) const fn empty() -> Self { + Self { + arm: ARM_UNRESOLVED, + base: 0, + epoch: 1, + table: Vec::new(), + hits: 0, + misses: 0, + inserts: 0, + rebases: 0, + refused: 0, + oos: 0, + ways: [PageGenerationCache::empty(); PAGE_GENERATION_CACHE_WAYS], + next: 0, + } + } + + #[inline(always)] + pub(super) fn lookup(&mut self, key: usize, addr: usize) -> Option { + if self.arm != ARM_WAYS { + // `wrapping_sub` folds `key < base` into the same out-of-range + // check as `key >= base + len`: a key below the base wraps to a + // huge index and fails `< len`. + let idx = key.wrapping_sub(self.base); + if idx < self.table.len() { + let e = &self.table[idx]; + if e.epoch == self.epoch && e.range.contains(addr) { + self.hits += 1; + return Some(e.range); + } + } else { + // Miss-path only, so it costs nothing on a hit — and it is the + // counter that decides between the two explanations for a + // residual miss rate. Out of span: the key is a candidate + // address in no registered block (the population the table was + // never able to hold, since the map has no answer to cache + // either). In span: the table itself failed — a class holding + // more than one range, or invalidation churn. + self.oos += 1; + } + self.misses += 1; + return None; + } + for way in self.ways.iter() { + if way.valid && way.key == key && way.range.contains(addr) { + self.hits += 1; + return Some(way.range); + } + } + self.misses += 1; + None + } + + #[inline] + pub(super) fn insert(&mut self, key: usize, range: PageGenerationRange) { + if self.arm == ARM_UNRESOLVED { + // The one env read, on the cold path, once per thread. + self.arm = if page_class_table_enabled() { + ARM_TABLE + } else { + ARM_WAYS + }; + } + if self.arm == ARM_TABLE { + if self.table.is_empty() { + // The base is taken from the FIRST insert, minus slack. Never a + // constant: the arena's placement moves with ASLR. + self.base = key.saturating_sub(PAGE_CLASS_TABLE_BASE_SLACK); + self.table = vec![PageClassEntry::DEAD; PAGE_CLASS_TABLE_INITIAL_SPAN]; + } + let mut idx = key.wrapping_sub(self.base); + if idx >= self.table.len() { + if !self.rebase_to_cover(key) { + // Past the cap: leave it uncached. The caller already has + // the authoritative answer and returns it; only the + // acceleration is forgone. + self.refused += 1; + return; + } + idx = key - self.base; + } + self.table[idx] = PageClassEntry { + range, + epoch: self.epoch, + }; + self.inserts += 1; + return; + } + self.inserts += 1; + let slot = self.next % PAGE_GENERATION_CACHE_WAYS; + self.ways[slot] = PageGenerationCache { + key, + range, + valid: true, + }; + self.next = slot.wrapping_add(1); + } + + /// Grow the table so that `key` is inside it, keeping every class it + /// already covered. Returns false — and changes nothing — if the resulting + /// span would exceed the cap. + #[cold] + #[inline(never)] + fn rebase_to_cover(&mut self, key: usize) -> bool { + let old_lo = self.base; + let old_hi = self.base + self.table.len(); // exclusive + let new_lo = old_lo.min(key.saturating_sub(PAGE_CLASS_TABLE_BASE_SLACK)); + let new_hi = old_hi.max(key.saturating_add(1 + PAGE_CLASS_TABLE_BASE_SLACK)); + let span = new_hi - new_lo; + if span > PAGE_CLASS_TABLE_MAX_SPAN { + return false; + } + // Entries are a cache; dropping them is always correct. Rebasing by + // bumping the epoch rather than copying keeps this simple and it is + // rare — measured span growth was 1,018 -> 1,021 over two whole runs. + self.epoch = self.epoch.wrapping_add(1); + self.table = vec![PageClassEntry::DEAD; span]; + self.base = new_lo; + self.rebases += 1; + true + } + + /// Invalidate everything, both arms. O(1) for the table: an epoch bump + /// makes every entry stale at once, which is the same contract the 4-way + /// set met by being reset wholesale — and the reason the table can meet it + /// without touching ~2,000 entries. + /// + /// The bump is the whole of the table's correctness. Without it a retagged + /// block keeps answering with its previous generation, which is a + /// misclassified pointer: the collector treats an old object as young, or + /// declines to trace a young one. `a_registration_change_invalidates_every_entry` + /// is the standing guard. + #[inline] + pub(super) fn invalidate(&mut self) { + self.ways = [PageGenerationCache::empty(); PAGE_GENERATION_CACHE_WAYS]; + self.next = 0; + self.epoch = self.epoch.wrapping_add(1); + if self.epoch == 0 { + // Wrapped: 0 is the "never live" sentinel a zeroed entry carries, + // so step past it. Reaching this needs 2^64 invalidations; the + // branch is here so the sentinel cannot be forged rather than + // because the wrap is expected. + self.epoch = 1; + } + } + + /// `(hits, misses, inserts, span, rebases, refused)` for the diagnostic + /// line and for the tests. + fn stats(&self) -> PageClassStats { + PageClassStats { + arm: self.arm, + hits: self.hits, + misses: self.misses, + inserts: self.inserts, + span: self.table.len(), + rebases: self.rebases, + refused: self.refused, + oos: self.oos, + } + } +} + +/// What [`PageGenerationCacheSet::stats`] reports. A named struct rather than a +/// tuple because the report and four tests read different fields of it and a +/// six-tuple's positions are not self-describing at the call site. +#[derive(Clone, Copy)] +struct PageClassStats { + arm: u8, + hits: u64, + misses: u64, + inserts: u64, + span: usize, + rebases: u64, + refused: u64, + oos: u64, +} + +/// `PERRY_GC_PAGE_CLASS_TABLE=0` restores the 4-way set. The kill switch, and +/// the positive control: both arms live in ONE binary so no build difference +/// can be confounded with the change. +#[inline(always)] +fn page_class_table_enabled() -> bool { + static ENABLED: std::sync::OnceLock = std::sync::OnceLock::new(); + *ENABLED.get_or_init(|| crate::gc::env_default_on_enabled("PERRY_GC_PAGE_CLASS_TABLE")) +} + +/// One line under `PERRY_GC_DIAG=1`, emitted per copying minor from the +/// collector (never at exit — the rig SIGKILLs the process). +pub(crate) fn page_class_table_report() { + if !crate::gc::gc_diag_enabled() { + return; + } + // SAFETY: thread-local, single-threaded, shared borrow ends here. + let st = unsafe { (*hot_page_generation_cache()).stats() }; + let tot = st.hits + st.misses; + if tot == 0 { + return; + } + // `misses - inserts` is the population in no registered block at all: the + // map had no answer either, so nothing was cached. Reported apart because + // the two halves are removed by different properties of the table — the + // first by capacity, the second by the bounds check. + let unregistered = st.misses.saturating_sub(st.inserts); + let arm_name = match st.arm { + ARM_WAYS => "4way", + ARM_TABLE => "table", + // Never inserted, so never resolved: report what it WOULD pick. + _ if page_class_table_enabled() => "table(unresolved)", + _ => "4way(unresolved)", + }; + eprintln!( + "[gc-page-class] arm={} lookups={tot} hit={} ({:.3}%) miss={} ({:.3}%) \ +miss_registered={} miss_unregistered={} miss_out_of_span={} span={} rebases={} refused={}", + arm_name, + st.hits, + 100.0 * st.hits as f64 / tot as f64, + st.misses, + 100.0 * st.misses as f64 / tot as f64, + st.inserts, + unregistered, + st.oos, + st.span, + st.rebases, + st.refused, + ); +} + +#[cfg(test)] +mod page_class_table_tests { + //! The direct-indexed page-class table, pinned at the two points the span + //! measurement could not settle. A wrong answer from this structure is a + //! misclassified pointer — a collector that moves the wrong thing — so + //! each path has a test that fails when its fallback is removed. + use super::*; + + fn fresh(f: impl FnOnce() -> T + Send + 'static) -> T { + // Thread-local table, thread-local map: a fresh thread is a fresh world. + std::thread::spawn(f) + .join() + .expect("page-class table test panicked") + } + + fn table_stats() -> PageClassStats { + // SAFETY: thread-local, single-threaded, borrow ends here. + unsafe { (*hot_page_generation_cache()).stats() } + } + + const MB: usize = 1 << GENERATION_CLASS_SHIFT; + + /// The base is taken from the FIRST registration, wherever it is — not + /// from a constant. An arena that starts at a high address (ASLR moved the + /// base by 0x142920 classes between two measured runs) must hit the table, + /// not fall through to the map forever. + /// + /// Sabotage: hard-wire `self.base = 0` in `insert` — the classification + /// still returns the right generation (the map is authoritative) but every + /// lookup misses, and this test fails on the hit counter. + #[test] + fn base_is_taken_from_the_first_registration_not_a_constant() { + if !page_class_table_enabled() { + return; + } + fresh(|| { + // Far from zero, and not 1 MiB-aligned so the key math is exercised. + let base = 0x5f0_0000_0000usize + 0x3_8000; + register_block_space(base, MB, HeapGeneration::Old, HeapSpace::Old); + let inside = base + 0x1234; + // First classification: a miss that fills the entry. + assert_eq!(classify_heap_generation(inside), HeapGeneration::Old); + let before = table_stats(); + assert!(before.span > 0, "the first insert must allocate the table"); + assert_eq!( + (before.rebases, before.refused), + (0, 0), + "a base derived from the first registration must cover that \ + registration in the initial table — no rebase, no refusal" + ); + // Second: MUST be a table hit. + assert_eq!(classify_heap_generation(inside), HeapGeneration::Old); + let after = table_stats(); + assert_eq!( + after.hits, + before.hits + 1, + "a re-classification of a registered address must hit the table; \ + a base that is not derived from the first registration leaves \ + every key out of span and the table permanently cold" + ); + }); + } + + /// A key OUTSIDE the current span must still classify correctly, through + /// the authoritative map — either by rebasing the table to cover it or, past + /// the cap, by falling through uncached. Both are exercised. + /// + /// Sabotage: in `insert`, replace the out-of-span branch with an unchecked + /// `self.table[idx]` — the first assertion below panics on the bounds + /// check, and a release build without bounds checks would write past the + /// allocation. Or make `lookup` return the entry without `idx < len` — the + /// far address then reads a garbage entry and this test's generation + /// assertion fails. + #[test] + fn a_key_outside_the_span_still_classifies_correctly() { + if !page_class_table_enabled() { + return; + } + fresh(|| { + let near = 0x6a0_0000_0000usize; + register_block_space(near, MB, HeapGeneration::Old, HeapSpace::Old); + assert_eq!(classify_heap_generation(near + 8), HeapGeneration::Old); + let s0 = table_stats(); + assert_eq!(s0.span, PAGE_CLASS_TABLE_INITIAL_SPAN); + + // 1. Within the cap: a block 4,000 classes away. Must rebase and + // then hit. + let far = near + 4_000 * MB; + register_block_space(far, MB, HeapGeneration::Nursery, HeapSpace::NurseryEden); + assert_eq!( + classify_heap_generation(far + 8), + HeapGeneration::Nursery, + "an out-of-span key must classify through the map" + ); + let s1 = table_stats(); + assert_eq!( + s1.rebases, + s0.rebases + 1, + "a key inside the cap must rebase the table" + ); + assert!(s1.span > s0.span, "rebasing must widen the span"); + assert_eq!(s1.refused, 0); + // And the ORIGINAL block is still answered correctly after rebase. + assert_eq!(classify_heap_generation(near + 8), HeapGeneration::Old); + let h_before = table_stats().hits; + assert_eq!(classify_heap_generation(far + 8), HeapGeneration::Nursery); + assert_eq!( + table_stats().hits, + h_before + 1, + "after rebase the far key must hit" + ); + + // 2. Past the cap: 40,000 classes away. Must NOT rebase (the cap + // bounds the allocation) and must STILL classify correctly, + // uncached. + let wild = near + 40_000 * MB; + register_block_space(wild, MB, HeapGeneration::Longlived, HeapSpace::Old); + assert_eq!( + classify_heap_generation(wild + 8), + HeapGeneration::Longlived, + "a key past the cap must fall through to the map, not be dropped" + ); + let s2 = table_stats(); + assert_eq!( + s2.rebases, s1.rebases, + "a key past the cap must not grow the table" + ); + assert_eq!(s2.span, s1.span); + assert!(s2.refused >= 1, "the refusal must be counted, not silent"); + // Classify it again: still correct, still uncached. + assert_eq!( + classify_heap_generation(wild + 8), + HeapGeneration::Longlived + ); + }); + } + + /// A key match is NOT an address match. Two ranges can share a 1 MiB class + /// (`PageGenerationSlot::Multiple`); an entry confirmed for one must not + /// answer for an address in the other. + /// + /// Sabotage: drop `e.range.contains(addr)` from `lookup` — the second + /// classification returns the first range's generation for an address that + /// is not in it. + /// + /// Deliberately NOT gated on the arm: it asserts only on classification + /// results, which must hold whichever structure answers, so a run with + /// `PERRY_GC_PAGE_CLASS_TABLE=0` exercises the 4-way control arm through + /// this test. (The 4-way set can hold both ranges at once, in two ways + /// under one key; the table holds the last-confirmed one and misses to the + /// map for the other. Both are correct, which is what is pinned here.) + #[test] + fn a_hit_requires_range_containment_not_just_key_equality() { + fresh(|| { + // Two half-class ranges in the SAME class, different generations. + let class_base = 0x7b0_0000_0000usize; + let half = MB / 2; + register_block_space(class_base, half, HeapGeneration::Old, HeapSpace::Old); + register_block_space( + class_base + half, + half, + HeapGeneration::Nursery, + HeapSpace::NurseryEden, + ); + assert_eq!( + classify_heap_generation(class_base + 8), + HeapGeneration::Old + ); + // Same key, other half: the cached entry (Old) must NOT answer. + assert_eq!( + classify_heap_generation(class_base + half + 8), + HeapGeneration::Nursery, + "an entry for another range in the same class answered for this address" + ); + assert_eq!( + classify_heap_generation(class_base + 8), + HeapGeneration::Old + ); + }); + } + + /// Registration invalidates: a retagged block must never be answered from + /// a stale entry. This is the 4-way set's original contract carried over. + /// + /// Sabotage: make `invalidate` a no-op for the table — the second + /// classification returns the pre-retag generation. + /// + /// Also ungated on the arm: "a retag is never answered from a stale entry" + /// is the contract of BOTH structures, and running it under + /// `PERRY_GC_PAGE_CLASS_TABLE=0` is what keeps the control arm from + /// rotting untested while the table is the default. + #[test] + fn a_registration_change_invalidates_every_entry() { + fresh(|| { + let base = 0x8c0_0000_0000usize; + register_block_space(base, MB, HeapGeneration::Nursery, HeapSpace::NurseryEden); + assert_eq!(classify_heap_generation(base + 8), HeapGeneration::Nursery); + assert_eq!(classify_heap_generation(base + 8), HeapGeneration::Nursery); // cached + unregister_block_generation(base, MB); + register_block_space(base, MB, HeapGeneration::Old, HeapSpace::Old); + assert_eq!( + classify_heap_generation(base + 8), + HeapGeneration::Old, + "a stale table entry answered after the block was retagged" + ); + }); + } +} diff --git a/crates/perry-runtime/src/arena/page_meta/tests.rs b/crates/perry-runtime/src/arena/page_meta/tests.rs new file mode 100644 index 0000000000..109303c040 --- /dev/null +++ b/crates/perry-runtime/src/arena/page_meta/tests.rs @@ -0,0 +1,131 @@ +//! Tests for `page_meta`, split out for the 2000-line file cap. + +#[cfg(test)] +mod page_generation_hasher_tests { + use super::super::*; + use std::collections::HashSet; + use std::hash::BuildHasher; + + /// #7187 regression guard for `PageGenerationMap`'s hasher. + /// + /// `HashMap` is hashbrown: the bucket index comes from the hash's low bits, + /// but the SIMD control byte — the filter that decides whether a group + /// probe needs a real key comparison — is `hash >> 57`. Generation class + /// keys are `addr >> GENERATION_CLASS_SHIFT`, so an identity hasher (which + /// this map carried until #7187) produces a value around 2^26 whose top + /// seven bits are zero for **every** key in the table. Every occupied slot + /// in a probed group then matches, and each match costs a scattered load + /// plus a key comparison — on a lookup the write barrier performs several + /// times per heap store. + /// + /// This asserts the property directly rather than asserting "we call + /// `PtrHasher`": reinstating any non-mixing hasher collapses the control + /// byte to a single value and fails here. + #[test] + fn control_byte_is_spread_across_generation_class_keys() { + let map = PageGenerationMap::default(); + let build = map.hasher(); + + // Realistic 48-bit heap addresses, one per 1 MiB generation bucket — + // the exact key population `classify_heap_generation` looks up. + let base: usize = 0x0000_7f31_0000_0000; + let control_bytes: HashSet = (0..64) + .map(|i| { + let addr = base + i * (1usize << GENERATION_CLASS_SHIFT); + (build.hash_one(generation_class_key_for_addr(addr)) >> 57) & 0x7f + }) + .collect(); + + assert!( + control_bytes.len() >= 32, + "hashbrown control byte must vary across generation class keys, got {} \ + distinct values from 64 consecutive buckets (an identity hasher yields 1)", + control_bytes.len() + ); + } + + /// The bucket index (low bits) must stay well spread too — mixing that put + /// all the entropy in the high bits and left the low bits constant would + /// trade a control-byte collision for a far worse bucket collision. This is + /// the failure `fast_hash`'s `mix` step exists for. + #[test] + fn bucket_index_is_spread_across_generation_class_keys() { + let map = PageGenerationMap::default(); + let build = map.hasher(); + + let base: usize = 0x0000_7f31_0000_0000; + let low_bits: HashSet = (0..64) + .map(|i| { + let addr = base + i * (1usize << GENERATION_CLASS_SHIFT); + build.hash_one(generation_class_key_for_addr(addr)) & 0x3f + }) + .collect(); + + assert!( + low_bits.len() >= 32, + "bucket index must vary across generation class keys, got {} distinct \ + values from 64 consecutive buckets", + low_bits.len() + ); + } + + /// The map must still answer correctly after the hasher change — a + /// point-query round trip over many buckets, which is the only way this map + /// is ever used. + #[test] + fn point_queries_round_trip_across_many_buckets() { + let mut map = PageGenerationMap::default(); + let base: usize = 0x0000_7f31_0000_0000; + for i in 0..256usize { + let addr = base + i * (1usize << GENERATION_CLASS_SHIFT); + map.insert( + generation_class_key_for_addr(addr), + PageGenerationSlot::Single(PageGenerationRange { + base: addr, + end: addr + (1 << GENERATION_CLASS_SHIFT), + generation: HeapGeneration::Old, + space: HeapSpace::Old, + object_starts: std::ptr::null_mut(), + }), + ); + } + for i in 0..256usize { + let addr = base + i * (1usize << GENERATION_CLASS_SHIFT); + let found = map + .get(&generation_class_key_for_addr(addr)) + .and_then(|slot| slot.find(addr + 0x40)) + .expect("every inserted bucket must be found by point query"); + assert_eq!(found.generation, HeapGeneration::Old); + assert_eq!(found.base, addr); + } + assert_eq!(map.len(), 256); + } +} + +#[cfg(test)] +mod block_range_tests { + use super::super::old_arena_block_range_index; + + /// `old_arena_block_range_index` is the whole reason #9772's selection can + /// group pages by block, so it gets a test that can fail: gaps between + /// blocks must not be attributed to the block below them. + #[test] + fn block_range_lookup_respects_gaps_and_ends() { + // Two 1 MiB blocks with a 1 MiB hole between them. + let ranges = vec![ + (0x1000_0000, 0x1010_0000, 7, 0x10_0000), + (0x1020_0000, 0x1030_0000, 9, 0x10_0000), + ]; + assert_eq!(old_arena_block_range_index(&ranges, 0x1000_0000), Some(0)); + assert_eq!(old_arena_block_range_index(&ranges, 0x100F_FFFF), Some(0)); + // One past the end of block 0 is the gap, not block 0. + assert_eq!(old_arena_block_range_index(&ranges, 0x1010_0000), None); + assert_eq!(old_arena_block_range_index(&ranges, 0x1018_0000), None); + assert_eq!(old_arena_block_range_index(&ranges, 0x1020_0000), Some(1)); + assert_eq!(old_arena_block_range_index(&ranges, 0x102F_FFFF), Some(1)); + // Above every block, and below every block. + assert_eq!(old_arena_block_range_index(&ranges, 0x1030_0000), None); + assert_eq!(old_arena_block_range_index(&ranges, 0x0FFF_FFFF), None); + assert_eq!(old_arena_block_range_index(&[], 0x1000_0000), None); + } +} diff --git a/crates/perry-runtime/src/arena/tests.rs b/crates/perry-runtime/src/arena/tests.rs index ef195d2f2a..9dadc3a0a0 100644 --- a/crates/perry-runtime/src/arena/tests.rs +++ b/crates/perry-runtime/src/arena/tests.rs @@ -1544,10 +1544,20 @@ fn deferred_registration_flush_sites() { ), ]; - let src = std::fs::read_to_string( - std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("src/arena/page_meta.rs"), - ) - .expect("page_meta.rs must be readable"); + // `page_meta` is a directory since the #9853 page-class table pushed it past + // the file cap. Read EVERY part: scanning only `mod.rs` would silently drop + // the functions that moved into `page_class.rs` from this audit, leaving a + // green test that covers less than it did before the split. + let page_meta_dir = + std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("src/arena/page_meta"); + let mut src = String::new(); + for part in ["mod.rs", "page_class.rs", "tests.rs"] { + src.push_str( + &std::fs::read_to_string(page_meta_dir.join(part)) + .unwrap_or_else(|e| panic!("page_meta/{part} must be readable: {e:?}")), + ); + src.push('\n'); + } // Split into function bodies by tracking `fn ` headers at any indent. let mut current: Option = None; @@ -1597,7 +1607,7 @@ fn deferred_registration_flush_sites() { assert!( offenders.is_empty(), - "these functions in arena/page_meta.rs read or mutate OLD_GEN_PAGE_OBJECTS / \ + "these functions in arena/page_meta/ read or mutate OLD_GEN_PAGE_OBJECTS / \ OLD_GEN_PAGE_META without first calling flush_deferred_old_page_registrations(), \ and are not listed as exempt: {offenders:?}.\n\ A deferred registration is invisible to a reader that does not flush, and a \ diff --git a/crates/perry-runtime/src/arena/tests_promoted_runs.rs b/crates/perry-runtime/src/arena/tests_promoted_runs.rs index 90a780be0f..1c5584e8f3 100644 --- a/crates/perry-runtime/src/arena/tests_promoted_runs.rs +++ b/crates/perry-runtime/src/arena/tests_promoted_runs.rs @@ -261,10 +261,20 @@ fn every_page_object_reader_expands_promoted_runs() { ), ]; - let src = std::fs::read_to_string( - std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("src/arena/page_meta.rs"), - ) - .expect("page_meta.rs must be readable"); + // `page_meta` is a directory since the #9853 page-class table pushed it past + // the file cap. Read EVERY part: scanning only `mod.rs` would silently drop + // the functions that moved into `page_class.rs` from this audit, leaving a + // green test that covers less than it did before the split. + let page_meta_dir = + std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("src/arena/page_meta"); + let mut src = String::new(); + for part in ["mod.rs", "page_class.rs", "tests.rs"] { + src.push_str( + &std::fs::read_to_string(page_meta_dir.join(part)) + .unwrap_or_else(|e| panic!("page_meta/{part} must be readable: {e:?}")), + ); + src.push('\n'); + } let mut bodies: Vec<(String, String)> = Vec::new(); for line in src.lines() { @@ -307,7 +317,7 @@ fn every_page_object_reader_expands_promoted_runs() { assert!( offenders.is_empty(), - "these functions in arena/page_meta.rs read or mutate \ + "these functions in arena/page_meta/ read or mutate \ OLD_GEN_PAGE_OBJECTS without first expanding pending promoted page \ runs: {offenders:?}.\n\ A promoted page's object list is DESCRIBED until someone asks for it, \ diff --git a/crates/perry-runtime/src/gc/copying.rs b/crates/perry-runtime/src/gc/copying.rs index d42daabea7..d0b25515f6 100644 --- a/crates/perry-runtime/src/gc/copying.rs +++ b/crates/perry-runtime/src/gc/copying.rs @@ -1690,6 +1690,9 @@ pub(super) fn run_copied_minor_attempt( collector.sticky.restore(); if !collector.skip_remembering { restore_surviving_dirty_coverage(&snapshot, &dirty_scan_covered, "copying_minor"); + // Per minor, not at exit: the rig SIGKILLs cc. Cumulative counters, so + // the last line before the kill is the answer. + crate::arena::page_class_table_report(); } // The mechanism, counted rather than assumed: with the pre-size working, // `capacity` is already >= `len` on entry and hashbrown never grows the @@ -1908,6 +1911,7 @@ fn finalize_dead_copied_minor_from_space_side_allocations() { crate::map::finalize_dead_copied_minor_from_space_maps(); crate::set::finalize_dead_copied_minor_from_space_sets(); crate::node_submodules::diagnostics_gc::finalize_dead_copied_minor_from_space_errors(); + crate::regex::finalize_dead_copied_minor_from_space_regexps(); // 2026-07-09 GC audit wave 2: the from-space flip runs no per-object // finalize hooks, so entries keyed by dead from-space owners in the // object-address-keyed side tables are pruned here (headers still intact). diff --git a/crates/perry-runtime/src/gc/dead_owner.rs b/crates/perry-runtime/src/gc/dead_owner.rs index 563e404c93..33e2884256 100644 --- a/crates/perry-runtime/src/gc/dead_owner.rs +++ b/crates/perry-runtime/src/gc/dead_owner.rs @@ -170,6 +170,13 @@ impl PostTraceProbe { /// from-space (eden or the active survivor half) and was neither marked nor /// forwarded — every live from-space object was evacuated (FORWARDED) or is /// pinned-and-marked by this point. Mirrors `is_dead_copied_minor_from_space_map`. +/// Crate-visible form for the per-type registry walkers that finalize their +/// own dead from-space instances after a copied minor (`regex`): is `addr` a +/// from-space `obj_type` cell that was neither evacuated nor pinned? +pub(crate) fn owner_is_dead_copied_minor_from_space_of_type(addr: usize, obj_type: u8) -> bool { + owner_is_dead_copied_minor_from_space(addr, Some(obj_type)) +} + fn owner_is_dead_copied_minor_from_space(addr: usize, expected_obj_type: Option) -> bool { let space = crate::arena::classify_heap_space(addr); if !matches!(space, crate::arena::HeapSpace::NurseryEden) diff --git a/crates/perry-runtime/src/gc/idle_reclaim.rs b/crates/perry-runtime/src/gc/idle_reclaim.rs index 8fbf67a92a..cf51815c77 100644 --- a/crates/perry-runtime/src/gc/idle_reclaim.rs +++ b/crates/perry-runtime/src/gc/idle_reclaim.rs @@ -51,9 +51,20 @@ //! 1. **Activity or arena debt.** Normally at least `2^backoff` collections //! the reducer did not start itself have completed since its last full. A //! collection is the signal that the mutator allocated enough to matter. -//! The exception is a bounded [`super::arena_right_size`] episode: arena -//! blocks need two full observations before their mappings can be returned, -//! and an idle heap cannot create the second through mutator activity. +//! There are two exceptions, and they are the same argument twice: a +//! requirement denominated in *mutator collections* cannot be met by a heap +//! whose mutator is idle, which is exactly when the reducer is wanted. +//! First, a bounded [`super::arena_right_size`] episode: arena blocks need +//! two full observations before their mappings can be returned, and an idle +//! heap cannot create the second through mutator activity. Second, elapsed +//! idle — see [`IDLE_RECLAIM_REARM_MS`] — because a *declined* follow-up +//! would otherwise be terminal (#9831): measured on the claude-code TUI, the +//! compactor's residue gate declines at 23.7 %, so no compaction runs, so no +//! collection is registered, so `since_attempt` stays 0 and the reducer +//! never runs again. The compaction IS the event that re-arms the reducer, +//! so declining one removes the only thing that could revisit the decision, +//! and the heap parks 221 MB above where the same workload settles when the +//! first compaction happens to fire. //! 2. **Quiet.** At least [`IDLE_RECLAIM_QUIET_MS`] since the last such //! collection was observed — a burst still in progress collects every few //! hundred milliseconds and must not be interleaved with a whole-heap mark. @@ -125,6 +136,23 @@ pub const IDLE_RECLAIM_PRODUCTIVE_PCT: usize = 5; /// exceeds `2^this` collections. pub const IDLE_RECLAIM_MAX_BACKOFF_SHIFT: u32 = 5; +/// Elapsed idle that substitutes for the activity requirement, at +/// `backoff_shift == 0`; the wait is `IDLE_RECLAIM_REARM_MS << backoff_shift`, +/// so it is the SAME backoff that prices the activity arm. +/// +/// This is the whole of the anti-spin argument and it needs no new rule: an +/// unproductive full doubles the wait, so a heap with nothing to give is asked +/// at 15 s, 30 s, 60 s, 120 s, 240 s and then — because the arm is disarmed at +/// [`IDLE_RECLAIM_MAX_BACKOFF_SHIFT`] — **not again until real mutator activity +/// resets the shift**. Five bounded attempts over ~8 minutes, then silence. A +/// PRODUCTIVE full resets the shift to zero, so a heap that is still giving +/// memory back keeps being asked every 15 s, which is the case this exists for. +/// +/// Larger than [`IDLE_RECLAIM_MIN_INTERVAL_MS`] on purpose, so the rate floor +/// is never the binding constraint on this arm and the two gates cannot be +/// confused for one another when reading a diag. +pub const IDLE_RECLAIM_REARM_MS: u64 = 15_000; + /// Most collector work the park hook will do in any one wall-clock second /// while a budgeted cycle is open; past this the loop parks instead. pub const IDLE_RECLAIM_MAX_WORK_MS_PER_SECOND: u64 = 500; @@ -147,6 +175,10 @@ enum StartReason { /// Sustained arena slack still needs full observations before empty blocks /// can be returned, even though the mutator has done nothing new. ArenaRightSize, + /// The activity requirement has not been met, but enough idle time has + /// passed that waiting for a mutator collection is waiting for something + /// that is not coming. See [`IDLE_RECLAIM_REARM_MS`]. + IdleElapsed, } impl StartReason { @@ -154,6 +186,7 @@ impl StartReason { match self { StartReason::Activity => "activity", StartReason::ArenaRightSize => "arena_right_size", + StartReason::IdleElapsed => "idle_elapsed", } } } @@ -247,6 +280,10 @@ static YIELDS: AtomicU64 = AtomicU64::new(0); static START_BLOCKED: AtomicU64 = AtomicU64::new(0); static WORK_CAPPED: AtomicU64 = AtomicU64::new(0); static POST_PURGES: AtomicU64 = AtomicU64::new(0); +/// Fulls started because idle time elapsed rather than because the mutator +/// collected. Counted so a test can assert WHICH arm started a full — the +/// attempt count alone cannot tell the two apart. +static IDLE_ELAPSED_STARTS: AtomicU64 = AtomicU64::new(0); /// Reducer fulls started in this process. pub fn idle_reclaim_attempts() -> u64 { @@ -300,6 +337,10 @@ pub fn idle_reclaim_post_purges() -> u64 { } /// Current unproductive-streak backoff shift on this thread. +pub fn idle_reclaim_elapsed_starts() -> u64 { + IDLE_ELAPSED_STARTS.load(Ordering::Relaxed) +} + pub fn idle_reclaim_backoff_shift() -> u32 { STATE.with(|s| s.borrow().backoff_shift) } @@ -366,10 +407,28 @@ fn start_reason(now: u64) -> Option { return Some(StartReason::ArenaRightSize); } let since_attempt = external.saturating_sub(st.external_at_last_attempt); - if since_attempt < (1u64 << st.backoff_shift) { - return None; + if since_attempt >= (1u64 << st.backoff_shift) { + return Some(StartReason::Activity); } - Some(StartReason::Activity) + // The activity requirement is denominated in collections the reducer + // did not start, and on a quiet heap the only such collections are the + // compactor's — which run only once the reducer has already moved the + // residue ratio past the compactor's own gate. When that gate declines, + // nothing else can move it, and the decline is permanent. Elapsed idle + // is the same requirement in the one unit a quiet heap still produces. + // + // Disarmed at the maximum shift rather than merely slowed: five + // unproductive attempts are enough to establish there is nothing to + // give, and after them this arm must stop entirely or an idle process + // pays a whole-heap mark forever. Real activity resets the shift (via a + // productive full) and re-enables it. + if st.attempts > 0 + && st.backoff_shift < IDLE_RECLAIM_MAX_BACKOFF_SHIFT + && now.saturating_sub(st.last_attempt_ms) >= (IDLE_RECLAIM_REARM_MS << st.backoff_shift) + { + return Some(StartReason::IdleElapsed); + } + None }) } @@ -384,6 +443,9 @@ fn note_started(now: u64, reason: StartReason) { if reason == StartReason::ArenaRightSize { super::arena_right_size::note_started(); } + if reason == StartReason::IdleElapsed { + IDLE_ELAPSED_STARTS.fetch_add(1, Ordering::Relaxed); + } if gc_diag_enabled() { let (_, right_size_fulls_remaining, _, usage) = super::arena_right_size::snapshot(); eprintln!( diff --git a/crates/perry-runtime/src/gc/mod.rs b/crates/perry-runtime/src/gc/mod.rs index 844be2e5e9..64409430cb 100644 --- a/crates/perry-runtime/src/gc/mod.rs +++ b/crates/perry-runtime/src/gc/mod.rs @@ -60,12 +60,12 @@ pub use idle_compact::{ }; pub use idle_reclaim::{ idle_reclaim_attempts, idle_reclaim_backoff_shift, idle_reclaim_completions, - idle_reclaim_enabled_from_value, idle_reclaim_freed_bytes, idle_reclaim_old_reclaimed_bytes, - idle_reclaim_post_purges, idle_reclaim_productive, idle_reclaim_slices, - idle_reclaim_start_blocked, idle_reclaim_work_capped, idle_reclaim_yields, + idle_reclaim_elapsed_starts, idle_reclaim_enabled_from_value, idle_reclaim_freed_bytes, + idle_reclaim_old_reclaimed_bytes, idle_reclaim_post_purges, idle_reclaim_productive, + idle_reclaim_slices, idle_reclaim_start_blocked, idle_reclaim_work_capped, idle_reclaim_yields, IDLE_RECLAIM_MAX_BACKOFF_SHIFT, IDLE_RECLAIM_MAX_WORK_MS_PER_SECOND, IDLE_RECLAIM_MIN_INTERVAL_MS, IDLE_RECLAIM_PRODUCTIVE_MIN_BYTES, IDLE_RECLAIM_PRODUCTIVE_PCT, - IDLE_RECLAIM_QUIET_MS, IDLE_RECLAIM_SLICE_US, + IDLE_RECLAIM_QUIET_MS, IDLE_RECLAIM_REARM_MS, IDLE_RECLAIM_SLICE_US, }; pub(crate) use idle_reclaim::{park_hook as idle_reclaim_park_hook, ParkVerdict}; mod telemetry; @@ -185,6 +185,7 @@ pub(crate) use copying_pointer_set::CopyingPointerSet; #[cfg(test)] pub(crate) use copying::MAX_YOUNG_MOVE_BYTES; mod dead_owner; +pub(crate) use dead_owner::owner_is_dead_copied_minor_from_space_of_type; mod old_free; use old_free::*; pub(crate) use old_free::{old_free_bytes, old_free_filter_range, old_free_take_exact}; diff --git a/crates/perry-runtime/src/gc/oldgen.rs b/crates/perry-runtime/src/gc/oldgen.rs index 2a331eaa89..0c0ef47b5d 100644 --- a/crates/perry-runtime/src/gc/oldgen.rs +++ b/crates/perry-runtime/src/gc/oldgen.rs @@ -1154,6 +1154,7 @@ pub(super) struct IncrementalSweepState { subphase: SweepCycleSubphase, dead_maps: Vec, dead_sets: Vec, + dead_regexps: Vec, dead_buffers: Vec, dead_typed_arrays: Vec, dead_lazy_arrays: Vec, @@ -1177,6 +1178,7 @@ impl IncrementalSweepState { subphase: SweepCycleSubphase::Malloc, dead_maps: Vec::new(), dead_sets: Vec::new(), + dead_regexps: Vec::new(), dead_buffers: Vec::new(), dead_typed_arrays: Vec::new(), dead_lazy_arrays: Vec::new(), @@ -1215,6 +1217,7 @@ impl IncrementalSweepState { ); self.dead_maps = crate::map::collect_dead_registered_maps_post_trace(full_trace); self.dead_sets = crate::set::collect_dead_registered_sets_post_trace(full_trace); + self.dead_regexps = crate::regex::collect_dead_registered_regexps_post_trace(full_trace); self.dead_buffers = crate::buffer::collect_dead_registered_buffers_post_trace(full_trace); self.dead_typed_arrays = crate::typedarray::collect_dead_registered_typed_arrays_post_trace(full_trace); @@ -1227,6 +1230,7 @@ impl IncrementalSweepState { }); if !self.dead_maps.is_empty() || !self.dead_sets.is_empty() + || !self.dead_regexps.is_empty() || !self.dead_buffers.is_empty() || !self.dead_typed_arrays.is_empty() || !self.dead_lazy_arrays.is_empty() @@ -1245,6 +1249,8 @@ impl IncrementalSweepState { crate::map::finalize_collected_dead_map(addr); } else if let Some(addr) = self.dead_sets.pop() { crate::set::finalize_collected_dead_set(addr); + } else if let Some(addr) = self.dead_regexps.pop() { + crate::regex::finalize_collected_dead_regexp(addr); } else if let Some(addr) = self.dead_buffers.pop() { crate::buffer::finalize_collected_dead_buffer(addr); } else if let Some(addr) = self.dead_typed_arrays.pop() { @@ -1259,6 +1265,7 @@ impl IncrementalSweepState { } if self.dead_maps.is_empty() && self.dead_sets.is_empty() + && self.dead_regexps.is_empty() && self.dead_buffers.is_empty() && self.dead_typed_arrays.is_empty() && self.dead_lazy_arrays.is_empty() diff --git a/crates/perry-runtime/src/gc/tests/copying/survival_and_malloc.rs b/crates/perry-runtime/src/gc/tests/copying/survival_and_malloc.rs index d35ad10f9f..aed0b6c681 100644 --- a/crates/perry-runtime/src/gc/tests/copying/survival_and_malloc.rs +++ b/crates/perry-runtime/src/gc/tests/copying/survival_and_malloc.rs @@ -1017,3 +1017,51 @@ fn test_copied_minor_promotable_census_filtered_walk_matches_unfiltered() { the equivalence assert above must not be vacuously 0 == 0" ); } + +/// #9819 follow-up: `js_regexp_new` allocates the header in the NURSERY. A +/// header that dies young must be finalized by the copied minor — its `Arc` +/// program released and its registry entries removed — because the from-space +/// flip runs no per-object finalize hooks. Without +/// `finalize_dead_copied_minor_from_space_regexps` the dead address stays in +/// `REGEX_POINTERS` and the program's strong count never comes back down. +#[test] +fn nursery_regexp_that_dies_young_is_finalized_by_the_copied_minor() { + let _guard = CopyingNurseryTestGuard::new(1); + let dead = crate::regex::test_construct_regexp_and_exec_once("b(?:c)+d-die-young", "g"); + let live = crate::regex::test_construct_regexp_and_exec_once("b(?:c)+d-die-young", "g"); + let dead_addr = dead as usize; + let live_addr = live as usize; + // Premise: production construction is nursery-allocated now. + assert!( + crate::arena::pointer_in_nursery(dead_addr), + "the header must be nursery-allocated" + ); + assert!(crate::regex::test_regex_pointer_entry_exists(dead_addr)); + assert!(crate::regex::test_regex_source_entry_exists(dead_addr)); + // Both headers share one program through the site cache. + let count_before = crate::regex::test_regexp_std_program_strong_count(live); + assert!(count_before >= 2); + + // Only `live` is rooted; `dead` is garbage. + js_shadow_slot_set(0, ptr_bits(live_addr)); + let _ = gc_collect_minor(); + + let live_new = (js_shadow_slot_get(0) & POINTER_MASK) as usize; + assert_ne!(live_new, 0, "the rooted RegExp must survive"); + assert_ne!(live_new, live_addr, "the rooted RegExp must be evacuated"); + assert!(crate::regex::regex_header_has_magic(live_new as *const _)); + assert!(crate::regex::test_regex_pointer_entry_exists(live_new)); + assert!(crate::regex::test_regex_source_entry_exists(live_new)); + + assert!( + !crate::regex::test_regex_pointer_entry_exists(dead_addr), + "a nursery RegExp that died must be removed from REGEX_POINTERS by the copied minor" + ); + assert!(!crate::regex::test_regex_source_entry_exists(dead_addr)); + assert_eq!( + crate::regex::test_regexp_std_program_strong_count(live_new as *const _), + count_before - 1, + "the dead header's Arc clone of the shared program must have been dropped" + ); + js_shadow_slot_set(0, 0); +} diff --git a/crates/perry-runtime/src/gc/tests/idle_reclaim.rs b/crates/perry-runtime/src/gc/tests/idle_reclaim.rs index eacee0ba1f..63f609223d 100644 --- a/crates/perry-runtime/src/gc/tests/idle_reclaim.rs +++ b/crates/perry-runtime/src/gc/tests/idle_reclaim.rs @@ -223,6 +223,147 @@ fn sustained_arena_slack_gets_one_bounded_followup_without_mutator_activity() { ); } +#[test] +fn a_parked_heap_is_re_armed_by_elapsed_idle_alone() { + // #9831. The activity requirement is denominated in collections the + // reducer did not start. On a quiet heap the only such collections are the + // compactor's, and the compactor runs only once the reducer has already + // moved the residue ratio past the compactor's gate — so when that gate + // declines, the decline is permanent and the heap parks. Measured on the + // claude-code TUI: 23.7 % against a 25 % gate, one reclaim attempt, and + // 221 MB never returned. Elapsed idle must be able to start attempt 2 with + // no new collection anywhere. + // + // Sabotage: delete the `StartReason::IdleElapsed` arm from `start_reason` + // and this test fails at "a second attempt must start" — attempts stay 1 + // forever, which is exactly the production symptom. + let _guard = CopyingNurseryTestGuard::new(1); + let _triggers = GcTriggerThresholdTestGuard::suppress_automatic_triggers(); + let _reducer = IdleReclaimTestGuard::new(0); + // Dead old-gen litter so the full is PRODUCTIVE and the shift stays 0: + // this test is about the re-arm, and the backoff is the next test's + // subject. (The guard has already pinned arena usage to live == capacity, + // so the `arena_right_size` arm cannot be what starts anything here.) + litter_old_gen_with_dead_promises(); + + // The ordinary activity arm starts attempt 1. This is the ONLY external + // collection in the whole test. + external_collection_observed_at(0); + set_test_now_ms(Some(IDLE_RECLAIM_QUIET_MS)); + assert!(resumes(idle_reclaim_park_hook(1000))); + drive_until_idle(IDLE_RECLAIM_QUIET_MS, 1000); + assert_eq!(thread_attempts(), 1); + assert_eq!( + idle_reclaim_backoff_shift(), + 0, + "the litter must have made this full productive, or the wait below is \ + doubled and this test is measuring the backoff instead of the re-arm" + ); + let elapsed_before = idle_reclaim_elapsed_starts(); + + // Past the rate floor (10 s) but short of the re-arm (15 s), so the ONLY + // thing that can hold the reducer here is the new gate. + assert!(IDLE_RECLAIM_REARM_MS > IDLE_RECLAIM_MIN_INTERVAL_MS); + set_test_now_ms(Some(IDLE_RECLAIM_QUIET_MS + IDLE_RECLAIM_REARM_MS - 1)); + assert!(parks(idle_reclaim_park_hook(1000))); + assert_eq!( + thread_attempts(), + 1, + "elapsed idle must not re-arm before the wait" + ); + + // At the wait: a second attempt, with no collection having happened. + let at = IDLE_RECLAIM_QUIET_MS + IDLE_RECLAIM_REARM_MS; + set_test_now_ms(Some(at)); + assert!( + resumes(idle_reclaim_park_hook(1000)), + "a second attempt must start on elapsed idle alone" + ); + assert_eq!(thread_attempts(), 2); + assert_eq!( + idle_reclaim_elapsed_starts(), + elapsed_before + 1, + "LIVE SUBJECT: the follow-up must identify ELAPSED IDLE as its reason, \ + not activity and not arena debt" + ); +} + +#[test] +fn an_unproductive_elapsed_streak_doubles_the_wait_and_then_disarms() { + // The anti-spin argument, and it needs no new rule: the elapsed arm is + // priced by the SAME `backoff_shift` as the activity arm, so a heap with + // nothing to give is asked at 15 s, 30 s, 60 s, 120 s, 240 s — and then not + // again, because the arm is disarmed at the maximum shift. Without the + // disarm an idle process would pay a whole-heap mark every 8 minutes + // forever. + // + // Two sabotages, one per guard. Drop `<< st.backoff_shift` from the wait + // and the "must not re-arm before the doubled wait" assertions fail. Drop + // the `backoff_shift < IDLE_RECLAIM_MAX_BACKOFF_SHIFT` term and the final + // assertion fails: the reducer keeps waking a heap that has already proved + // five times over that it has nothing to give. + let _guard = CopyingNurseryTestGuard::new(1); + let _triggers = GcTriggerThresholdTestGuard::suppress_automatic_triggers(); + let _reducer = IdleReclaimTestGuard::new(0); + // No litter: every full is unproductive, so every attempt doubles the wait. + + external_collection_observed_at(0); + set_test_now_ms(Some(IDLE_RECLAIM_QUIET_MS)); + assert!(resumes(idle_reclaim_park_hook(1000))); + drive_until_idle(IDLE_RECLAIM_QUIET_MS, 1000); + assert_eq!(thread_attempts(), 1); + assert_eq!(idle_reclaim_backoff_shift(), 1); + + let elapsed_before = idle_reclaim_elapsed_starts(); + let mut now = IDLE_RECLAIM_QUIET_MS; + let mut attempts = 1; + for shift in 1..IDLE_RECLAIM_MAX_BACKOFF_SHIFT { + let wait = IDLE_RECLAIM_REARM_MS << shift; + set_test_now_ms(Some(now + wait - 1)); + assert!( + parks(idle_reclaim_park_hook(1000)), + "shift {shift}: must not re-arm before the doubled wait" + ); + assert_eq!(thread_attempts(), attempts); + + now += wait; + set_test_now_ms(Some(now)); + assert!( + resumes(idle_reclaim_park_hook(1000)), + "shift {shift}: the doubled wait has elapsed" + ); + attempts += 1; + assert_eq!(thread_attempts(), attempts); + drive_until_idle(now, 1000); + assert_eq!( + idle_reclaim_backoff_shift(), + shift + 1, + "still unproductive: the shift must keep growing" + ); + } + assert_eq!(idle_reclaim_backoff_shift(), IDLE_RECLAIM_MAX_BACKOFF_SHIFT); + assert_eq!( + idle_reclaim_elapsed_starts(), + elapsed_before + u64::from(IDLE_RECLAIM_MAX_BACKOFF_SHIFT - 1), + "every follow-up in the streak must have come from the elapsed arm" + ); + + // Disarmed. However long the heap stays idle, it is not asked again. + let far = now + (IDLE_RECLAIM_REARM_MS << IDLE_RECLAIM_MAX_BACKOFF_SHIFT) * 16; + set_test_now_ms(Some(far)); + assert!( + parks(idle_reclaim_park_hook(1000)), + "at the maximum shift the elapsed arm is disarmed: the hook must park, \ + not open another whole-heap mark on a heap that has nothing to give" + ); + assert_eq!( + thread_attempts(), + attempts, + "at the maximum shift the elapsed arm must stop entirely: a heap with \ + nothing to give gets no new collection" + ); +} + #[test] fn idle_reclaim_rate_floor_holds_between_two_owed_fulls() { let _guard = CopyingNurseryTestGuard::new(1); diff --git a/crates/perry-runtime/src/gc/tests/mod.rs b/crates/perry-runtime/src/gc/tests/mod.rs index b181eb1838..a698870282 100644 --- a/crates/perry-runtime/src/gc/tests/mod.rs +++ b/crates/perry-runtime/src/gc/tests/mod.rs @@ -48,6 +48,7 @@ mod retention_9628_9629; mod root_words; mod rooted_container_values; mod rooted_define_property; +mod rooted_for_in; mod roots; mod runtime_roots; mod scan_fallback; diff --git a/crates/perry-runtime/src/gc/tests/rooted_for_in.rs b/crates/perry-runtime/src/gc/tests/rooted_for_in.rs new file mode 100644 index 0000000000..34c4c24fc8 --- /dev/null +++ b/crates/perry-runtime/src/gc/tests/rooted_for_in.rs @@ -0,0 +1,224 @@ +//! Enumeration retains its output and receiver across Proxy callbacks (#4644). + +use super::super::*; +use super::support::*; +use crate::gc::{RuntimeHandle, RuntimeHandleScope}; + +thread_local! { + static COPIED: std::cell::Cell = const { std::cell::Cell::new(0) }; +} + +extern "C" fn moving_own_keys(_closure: *const crate::closure::ClosureHeader, target: f64) -> f64 { + let scope = RuntimeHandleScope::new(); + let target = scope.root_nanbox_f64(target); + let trace = collect_minor_trace(GcTriggerKind::Direct); + COPIED.with(|count| count.set(count.get() + trace.copying_nursery.copied_objects)); + crate::object::js_object_get_own_property_names(target.get_nanbox_f64()) +} + +fn object(scope: &RuntimeHandleScope) -> RuntimeHandle<'_> { + scope.root_raw_mut_ptr(crate::object::js_object_alloc(0, 0)) +} + +fn boxed(handle: RuntimeHandle<'_>) -> f64 { + handle.with_const_ptr(|ptr: *const crate::object::ObjectHeader| { + f64::from_bits(ptr_bits(ptr as usize)) + }) +} + +fn set(handle: RuntimeHandle<'_>, name: &str, value: f64) { + let key = crate::string::js_string_from_bytes(name.as_ptr(), name.len() as u32); + handle.with_mut_ptr(|ptr| crate::object::js_object_set_field_by_name(ptr, key, value)); +} + +fn run(inherited_proxy: bool, descriptor_trap: bool) { + let _guard = CopyingNurseryTestGuard::new(0); + let _triggers = GcTriggerThresholdTestGuard::suppress_automatic_triggers(); + gc_register_mutable_root_scanner_with_source( + scan_runtime_handle_roots_mut, + MutableRootScannerSource::RuntimeHandles, + ); + gc_register_mutable_root_scanner(crate::proxy::scan_proxy_roots_mut); + COPIED.with(|count| count.set(0)); + let scope = RuntimeHandleScope::new(); + let target = object(&scope); + crate::object::js_object_set_prototype_of( + boxed(target), + f64::from_bits(crate::value::TAG_NULL), + ); + for index in 0..14 { + set(target, &format!("property_{index}"), index as f64); + } + let handler = object(&scope); + let (name, function) = if descriptor_trap { + ("getOwnPropertyDescriptor", moving_descriptor as *const u8) + } else { + ("ownKeys", moving_own_keys as *const u8) + }; + let callback = crate::closure::js_closure_alloc(function, 0); + set(handler, name, f64::from_bits(ptr_bits(callback as usize))); + let proxy = scope.root_nanbox_f64(crate::proxy::js_proxy_new(boxed(target), boxed(handler))); + let receiver = object(&scope); + if inherited_proxy { + // These entries grow the result before reaching the Proxy prototype. + for index in 0..10 { + set(receiver, &format!("local_{index}"), index as f64); + } + crate::object::js_object_set_prototype_of(boxed(receiver), proxy.get_nanbox_f64()); + } + let target_before = boxed(target).to_bits(); + let receiver_before = boxed(receiver).to_bits(); + let result = crate::object::js_for_in_keys_value(if inherited_proxy { + boxed(receiver) + } else { + proxy.get_nanbox_f64() + }); + let result = scope.root_raw_const_ptr(result); + assert!( + COPIED.with(|count| count.get()) > 0, + "the callback must move live objects" + ); + assert_ne!( + boxed(target).to_bits(), + target_before, + "the Proxy target must relocate" + ); + assert_ne!( + boxed(receiver).to_bits(), + receiver_before, + "the receiver must relocate" + ); + let local_count = if inherited_proxy { 10 } else { 0 }; + assert_eq!( + result.with_const_ptr(|array| crate::array::js_array_length(array)), + local_count + 14 + ); + for index in 0..local_count + 14 { + let expected = if index < local_count { + format!("local_{index}") + } else { + format!("property_{}", index - local_count) + }; + let value = result.with_const_ptr(|ptr| crate::array::js_array_get(ptr, index)); + unsafe { + assert_string_bytes( + (value.bits() & POINTER_MASK) as *const crate::StringHeader, + expected.as_bytes(), + ); + } + } +} + +#[test] +fn for_in_result_survives_own_keys_collection() { + run(false, false); +} + +#[test] +fn for_in_grown_result_and_receiver_survive_prototype_collection() { + run(true, false); +} + +extern "C" fn moving_descriptor( + _closure: *const crate::closure::ClosureHeader, + target: f64, + key: f64, +) -> f64 { + let scope = RuntimeHandleScope::new(); + let target = scope.root_nanbox_f64(target); + let key = scope.root_nanbox_f64(key); + let trace = collect_minor_trace(GcTriggerKind::Direct); + COPIED.with(|count| count.set(count.get() + trace.copying_nursery.copied_objects)); + crate::object::js_object_get_own_property_descriptor( + target.get_nanbox_f64(), + key.get_nanbox_f64(), + ) +} + +#[test] +fn descriptor_trap_collection_preserves_for_in_target_and_keys() { + run(false, true); +} + +extern "C" fn moving_value(_closure: *const crate::closure::ClosureHeader) -> f64 { + let trace = collect_minor_trace(GcTriggerKind::Direct); + COPIED.with(|count| count.set(count.get() + trace.copying_nursery.copied_objects)); + 23.0 +} + +extern "C" fn descriptor_with_moving_field( + _closure: *const crate::closure::ClosureHeader, + _target: f64, + _key: f64, +) -> f64 { + let scope = RuntimeHandleScope::new(); + let result = object(&scope); + for name in ["enumerable", "configurable", "writable"] { + set(result, name, f64::from_bits(crate::value::TAG_TRUE)); + } + let getter = crate::closure::js_closure_alloc(moving_value as *const u8, 0); + let descriptor = object(&scope); + set(descriptor, "get", f64::from_bits(ptr_bits(getter as usize))); + let key = crate::string::js_string_from_bytes(b"value".as_ptr(), 5); + crate::object::js_object_define_property( + boxed(result), + f64::from_bits(string_bits(key as usize)), + boxed(descriptor), + ); + boxed(result) +} + +#[test] +fn descriptor_completion_reloads_after_field_getter_collection() { + let _guard = CopyingNurseryTestGuard::new(0); + let _triggers = GcTriggerThresholdTestGuard::suppress_automatic_triggers(); + gc_register_mutable_root_scanner_with_source( + scan_runtime_handle_roots_mut, + MutableRootScannerSource::RuntimeHandles, + ); + gc_register_mutable_root_scanner(crate::proxy::scan_proxy_roots_mut); + COPIED.with(|count| count.set(0)); + let scope = RuntimeHandleScope::new(); + let target = object(&scope); + set(target, "property_name", 7.0); + let handler = object(&scope); + let callback = crate::closure::js_closure_alloc(descriptor_with_moving_field as *const u8, 0); + set( + handler, + "getOwnPropertyDescriptor", + f64::from_bits(ptr_bits(callback as usize)), + ); + let proxy = scope.root_nanbox_f64(crate::proxy::js_proxy_new(boxed(target), boxed(handler))); + let key = crate::string::js_string_from_bytes(b"property_name".as_ptr(), 13); + let before = boxed(target).to_bits(); + let result = crate::proxy::js_reflect_get_own_property_descriptor( + proxy.get_nanbox_f64(), + f64::from_bits(string_bits(key as usize)), + ); + let result = scope.root_nanbox_f64(result); + assert!(COPIED.with(|count| count.get()) > 0); + assert_ne!( + boxed(target).to_bits(), + before, + "the descriptor getter must move live objects" + ); + for (name, expected) in [ + ("value", 23.0), + ("writable", f64::from_bits(crate::value::TAG_TRUE)), + ("enumerable", f64::from_bits(crate::value::TAG_TRUE)), + ("configurable", f64::from_bits(crate::value::TAG_TRUE)), + ] { + let value = unsafe { + crate::value::js_get_property( + result.get_nanbox_f64(), + name.as_ptr() as i64, + name.len() as i64, + ) + }; + assert_eq!( + value.to_bits(), + expected.to_bits(), + "descriptor field {name}" + ); + } +} diff --git a/crates/perry-runtime/src/hot_diag.rs b/crates/perry-runtime/src/hot_diag.rs index 43ec8fba16..ca92f9f93b 100644 --- a/crates/perry-runtime/src/hot_diag.rs +++ b/crates/perry-runtime/src/hot_diag.rs @@ -131,6 +131,15 @@ pub struct RegexDiag { pub replace_calls: u64, pub replace_matches: u64, pub split_calls: u64, + /// `may_have_descriptor_entry` calls whose owner is a `GC_TYPE_REGEXP` + /// cell — the `lastIndex` writability question `set_last_index_throwing` + /// asks on every global/sticky `test()`/`exec()`. + pub desc_regexp_probes: u64, + /// Of those, the ones the per-object meta summary proved absent, so no + /// `key.to_string()` and no SipHash of `(usize, String)` ran. Before the + /// meta edge was wired for RegExp this was 0 by construction: the filter + /// answered "maybe" for every one of them. + pub desc_regexp_meta_negative: u64, per_pattern: HashMap, } @@ -244,7 +253,8 @@ impl RegexDiag { "[regex-diag] t={secs:.1}s new={} validated_hit={} site_hit={} pattern_bytes={} \ compiles std={} fancy={} repeat={} cache_clears={} lazy_builds={} lazy_cache_hits={} \ exec={} exec_matched={} capture_slots={} capture_bytes={} test={} test_global={} \ - match={} replace={} replace_matches={} split={} flags_alloc={}", + match={} replace={} replace_matches={} split={} flags_alloc={} \ + desc_regexp_probes={} desc_regexp_meta_negative={}", self.new_calls, self.new_validated_hit, self.new_site_hit, @@ -266,6 +276,8 @@ impl RegexDiag { self.replace_matches, self.split_calls, self.new_flags_allocated, + self.desc_regexp_probes, + self.desc_regexp_meta_negative, ); // Merge by content (prefix, len, flags): distinct literal sites with // the same pattern are one row. diff --git a/crates/perry-runtime/src/object/descriptor_state.rs b/crates/perry-runtime/src/object/descriptor_state.rs index 3e90299f56..31d0a54ce4 100644 --- a/crates/perry-runtime/src/object/descriptor_state.rs +++ b/crates/perry-runtime/src/object/descriptor_state.rs @@ -678,6 +678,40 @@ pub(crate) fn test_descriptor_key_bit(key: &str) -> u64 { descriptor_key_bit(key) } +/// #6759 phase 1 follow-up: the owner's meta record for descriptor-summary +/// purposes, for ANY cell type that owns one. +/// +/// [`super::prototype_chain::meta_capable_object`] answers only for +/// `GC_TYPE_OBJECT`, because its other callers need an `ObjectHeader` to work +/// with. The descriptor summary does not — it needs the `ObjectMeta` edge and +/// nothing else — and every exotic cell has carried that edge since #6759 +/// phase 1 unified it behind [`super::cell_meta_slot`]. Asking the narrower +/// question is what lets a `RegExp` receiver answer a summary probe at all. +/// +/// * `None` — the cell type has no meta edge, so the caller must stay +/// conservative and probe the tables. +/// * `Some(null)` — the cell HAS the edge and no record was ever installed, +/// which proves the tables hold no entry for this owner. +/// * `Some(meta)` — read the summary words. +/// +/// The three-way answer is the whole contract: collapsing "no edge" and "edge, +/// but null" into one `None` would turn a conservative *maybe* into a false +/// *no* for the cell types that still lack an edge. +#[inline] +unsafe fn descriptor_summary_meta(owner: usize) -> Option<*mut ObjectMeta> { + Some(*super::cell_meta_slot(owner)?) +} + +/// Installing twin of [`descriptor_summary_meta`]. Install and probe MUST use +/// the same predicate: a probe that admits a cell type whose installs do not +/// set the key bits would answer a proven-absent for an owner that really has +/// a descriptor — e.g. `Object.defineProperty(re, "lastIndex", {writable:false})` +/// would stop throwing (test262 prototype/{exec,test}/y-fail-lastindex-no-write). +#[inline] +unsafe fn descriptor_summary_meta_ensure(owner: usize) -> Option<*mut ObjectMeta> { + super::object_meta_ensure_for_cell(owner) +} + /// #6759 Phase C2: record `key` in the owner's per-object meta summary so /// hot-path probes for OTHER keys can skip the descriptor tables. No-op for /// owners that cannot carry a meta record (handle-band ids, typed arrays, @@ -694,13 +728,11 @@ pub(crate) fn test_descriptor_key_bit(key: &str) -> u64 { /// owner left behind can no longer be misread as the new tenant's. fn note_meta_descriptor_key(owner: usize, key: &str, accessor: bool) { unsafe { - if let Some(obj) = super::prototype_chain::meta_capable_object(owner) { - // No-move window: `object_meta_ensure` allocates, and a - // triggered collection could MOVE `owner` — installers - // (freeze/seal loops, defineProperty) hold raw owner pointers - // across repeated installs. - let _no_gc = crate::gc::GcSuppressScope::new(); - let meta = super::object_meta_ensure(obj); + // No-move window: the ensure below allocates, and a triggered + // collection could MOVE `owner` — installers (freeze/seal loops, + // defineProperty) hold raw owner pointers across repeated installs. + let _no_gc = crate::gc::GcSuppressScope::new(); + if let Some(meta) = descriptor_summary_meta_ensure(owner) { let bit = descriptor_key_bit(key); if accessor { (*meta).accessor_key_bits |= bit; @@ -718,22 +750,60 @@ fn note_meta_descriptor_key(owner: usize, key: &str, accessor: bool) { #[inline] pub(crate) fn may_have_descriptor_entry(owner: usize, key: &str, accessor: bool) -> bool { unsafe { - match super::prototype_chain::meta_capable_object(owner) { - Some(obj) => { - let meta = (*obj).meta; + let answer = match descriptor_summary_meta(owner) { + Some(meta) => { if meta.is_null() { - return false; - } - let word = if accessor { - (*meta).accessor_key_bits + false } else { - (*meta).attr_key_bits - }; - word & descriptor_key_bit(key) != 0 + let word = if accessor { + (*meta).accessor_key_bits + } else { + (*meta).attr_key_bits + }; + word & descriptor_key_bit(key) != 0 + } } None => true, + }; + // Diagnostic only, and only when the instrument is armed: one relaxed + // load otherwise. Counts the RegExp receivers this filter sees and how + // many it now proves absent — before the meta edge was wired for + // RegExp the second number was 0 by construction. + if crate::hot_diag::regex_on() { + note_regexp_descriptor_probe(owner, answer); } + answer + } +} + +/// Test-only view of [`may_have_descriptor_entry`], so a test can assert the +/// FILTER's answer rather than only the value it filters to. Without this a +/// test can see that `get_property_attrs` returns `None`, which is equally +/// true when the fast negative never fired — it would pass against a change +/// that did nothing. +#[cfg(test)] +pub(crate) fn test_may_have_descriptor_entry(owner: usize, key: &str, accessor: bool) -> bool { + may_have_descriptor_entry(owner, key, accessor) +} + +/// Diagnostic counter for [`may_have_descriptor_entry`]: is this owner a +/// RegExp cell, and did the summary prove the key absent? Split out and marked +/// cold so the armed check costs the hot path a predictable branch and nothing +/// else. +#[cold] +unsafe fn note_regexp_descriptor_probe(owner: usize, answer: bool) { + let Some(header) = crate::value::addr_class::try_read_gc_header(owner) else { + return; + }; + if header.obj_type != crate::gc::GC_TYPE_REGEXP { + return; } + crate::hot_diag::regex_with(|d| { + d.desc_regexp_probes += 1; + if !answer { + d.desc_regexp_meta_negative += 1; + } + }); } /// #6759 Phase C2: can an OWN string-keyed descriptor (attr or accessor) @@ -748,9 +818,8 @@ unsafe fn own_descriptor_may_cover_key(addr: usize, key: f64) -> bool { ) else { return true; }; - match super::prototype_chain::meta_capable_object(addr) { - Some(obj) => { - let meta = (*obj).meta; + match descriptor_summary_meta(addr) { + Some(meta) => { if meta.is_null() { return false; } @@ -768,9 +837,8 @@ unsafe fn own_descriptor_may_cover_key(addr: usize, key: f64) -> bool { #[inline] pub(crate) fn owner_may_have_descriptor_entries(owner: usize, accessor: bool) -> bool { unsafe { - match super::prototype_chain::meta_capable_object(owner) { - Some(obj) => { - let meta = (*obj).meta; + match descriptor_summary_meta(owner) { + Some(meta) => { if meta.is_null() { return false; } @@ -1214,11 +1282,10 @@ fn owner_index_push_proven_new( /// single-kind form's no-op arm). fn note_meta_descriptor_key_both(owner: usize, key: &str) -> Option<(bool, bool)> { unsafe { - let obj = super::prototype_chain::meta_capable_object(owner)?; - // No-move window: `object_meta_ensure` allocates (see + // No-move window: the ensure allocates (see // `note_meta_descriptor_key`). let _no_gc = crate::gc::GcSuppressScope::new(); - let meta = super::object_meta_ensure(obj); + let meta = descriptor_summary_meta_ensure(owner)?; let bit = descriptor_key_bit(key); let accessor_bit_was_set = (*meta).accessor_key_bits & bit != 0; let attr_bit_was_set = (*meta).attr_key_bits & bit != 0; diff --git a/crates/perry-runtime/src/object/field_get_set/enumeration.rs b/crates/perry-runtime/src/object/field_get_set/enumeration.rs index 397973a8e1..1d84623080 100644 --- a/crates/perry-runtime/src/object/field_get_set/enumeration.rs +++ b/crates/perry-runtime/src/object/field_get_set/enumeration.rs @@ -294,20 +294,29 @@ pub(crate) fn for_in_keys_with(value: f64, lazy_shadow: bool) -> *mut ArrayHeade if jv.is_null() || jv.is_undefined() { return crate::array::js_array_alloc(0); } - let mut out = crate::array::js_array_alloc(8); + // #9864: ownKeys, getOwnPropertyDescriptor and getPrototypeOf can invoke + // user callbacks. Keep every value needed after them in relocatable + // handles, including the output accumulated while walking earlier + // prototypes. + let scope = crate::gc::RuntimeHandleScope::new(); + let current = scope.root_nanbox_f64(value); + let out = scope.root_raw_mut_ptr(crate::array::js_array_alloc(8)); // Non-pointer primitives (number/boolean, boxed string) have only their own // enumerable keys; every prototype property they inherit is non-enumerable. if !jv.is_pointer() { if diag { crate::hot_diag::enum_with(|d| d.for_in_primitive += 1); } - let own = js_object_keys_value(value); - let n = crate::array::js_array_length(own); + let own = scope.root_raw_const_ptr(js_object_keys_value(current.get_nanbox_f64())); + let n = own.with_const_ptr(|array| crate::array::js_array_length(array)); for i in 0..n { - let kv = crate::array::js_array_get(own, i); - out = crate::array::js_array_push_f64(out, f64::from_bits(kv.bits())); + let kv = own.with_const_ptr(|own| crate::array::js_array_get(own, i)); + let updated = out.with_mut_ptr(|out| { + crate::array::js_array_push_f64(out, f64::from_bits(kv.bits())) + }); + out.set_raw_mut_ptr(updated); } - return out; + return out.with_mut_ptr(|out: *mut ArrayHeader| out); } let key_string = |kv: JSValue, scratch: &mut [u8; crate::value::SHORT_STRING_MAX_LEN]| { let made = unsafe { crate::string::js_string_key_bytes(kv, scratch) } @@ -325,7 +334,6 @@ pub(crate) fn for_in_keys_with(value: f64, lazy_shadow: bool) -> *mut ArrayHeade }; let mut seen: std::collections::HashSet = std::collections::HashSet::new(); let mut scratch = [0u8; crate::value::SHORT_STRING_MAX_LEN]; - let mut current = value; // #9792 follow-up: the shadow set is DEFERRED. // @@ -359,83 +367,100 @@ pub(crate) fn for_in_keys_with(value: f64, lazy_shadow: bool) -> *mut ArrayHeade let mut level: u32 = 0; // Depth cap guards against pathological / cyclic prototype graphs. for _ in 0..1000 { - let cv = JSValue::from_bits(current.to_bits()); + let cv = JSValue::from_bits(current.get_nanbox_u64()); if cv.is_null() || cv.is_undefined() || !cv.is_pointer() { break; } // Emit this level's enumerable own keys (OrdinaryOwnPropertyKeys order), // skipping any name already shadowed by a closer level. - let enum_arr = js_object_keys_value(current); - let en = crate::array::js_array_length(enum_arr); - if diag { - let en64 = en as u64; - crate::hot_diag::enum_with(|d| { - d.for_in_levels += 1; - d.for_in_key_arrays += 1; - d.for_in_keys_seen += en64; - }); - } - // Level 0 can be shadowed by nothing, so its own enumerable names go - // straight out — own property names are unique within one object, which - // is the only thing the set was doing for this level. - if lazy_shadow && level == 0 && !shadow_live { - for i in 0..en { - let kv = crate::array::js_array_get(enum_arr, i); - out = crate::array::js_array_push_f64(out, f64::from_bits(kv.bits())); - } + // The runtime handle stack is strictly LIFO: dropping this per-level + // scope truncates everything pushed after it, including a push into an + // OUTER scope. `visited.push` roots into `scope`, so it must happen + // after this block closes, not inside it. + { + let level_scope = crate::gc::RuntimeHandleScope::new(); + let enum_arr = + level_scope.root_raw_const_ptr(js_object_keys_value(current.get_nanbox_f64())); + let en = enum_arr.with_const_ptr(|array| crate::array::js_array_length(array)); if diag { let en64 = en as u64; - crate::hot_diag::enum_with(|d| d.for_in_keys_emitted += en64); + crate::hot_diag::enum_with(|d| { + d.for_in_levels += 1; + d.for_in_key_arrays += 1; + d.for_in_keys_seen += en64; + }); } - } else { - if en > 0 && !shadow_live { - // First level >= 1 with something to filter: pay for the set - // now, over exactly the levels already walked. - build_shadow_set(visited.as_slice(), &mut seen, &mut scratch, diag); - shadow_live = true; - if diag { - crate::hot_diag::enum_with(|d| d.for_in_shadow_built += 1); + // Level 0 can be shadowed by nothing, so its own enumerable names go + // straight out — own property names are unique within one object, which + // is the only thing the set was doing for this level. + if lazy_shadow && level == 0 && !shadow_live { + for i in 0..en { + let kv = enum_arr.with_const_ptr(|keys| crate::array::js_array_get(keys, i)); + let updated = out.with_mut_ptr(|out| { + crate::array::js_array_push_f64(out, f64::from_bits(kv.bits())) + }); + out.set_raw_mut_ptr(updated); } - } - for i in 0..en { - let kv = crate::array::js_array_get(enum_arr, i); - let name = match key_string(kv, &mut scratch) { - Some(s) => s, - None => continue, - }; - let fresh = seen.insert(name); if diag { - let deep = level > 0; - crate::hot_diag::enum_with(|d| { - d.for_in_seen_inserts += 1; - if !fresh { - d.for_in_seen_dupes += 1; - } else { - d.for_in_keys_emitted += 1; - if deep { - d.for_in_keys_emitted_deep += 1; - } - } - }); + let en64 = en as u64; + crate::hot_diag::enum_with(|d| d.for_in_keys_emitted += en64); } - if fresh { - out = crate::array::js_array_push_f64(out, f64::from_bits(kv.bits())); + } else { + if en > 0 && !shadow_live { + // First level >= 1 with something to filter: pay for the set + // now, over exactly the levels already walked. + build_shadow_set(visited.as_slice(), &mut seen, &mut scratch, diag); + shadow_live = true; + if diag { + crate::hot_diag::enum_with(|d| d.for_in_shadow_built += 1); + } + } + for i in 0..en { + let kv = enum_arr.with_const_ptr(|keys| crate::array::js_array_get(keys, i)); + let name = match key_string(kv, &mut scratch) { + Some(s) => s, + None => continue, + }; + let fresh = seen.insert(name); + if diag { + let deep = level > 0; + crate::hot_diag::enum_with(|d| { + d.for_in_seen_inserts += 1; + if !fresh { + d.for_in_seen_dupes += 1; + } else { + d.for_in_keys_emitted += 1; + if deep { + d.for_in_keys_emitted_deep += 1; + } + } + }); + } + if fresh { + let updated = out.with_mut_ptr(|out| { + crate::array::js_array_push_f64(out, f64::from_bits(kv.bits())) + }); + out.set_raw_mut_ptr(updated); + } } } + // Mark ALL own names (incl non-enumerable) so they shadow the remainder + // of the chain — but only once the set is live. Until then the level is + // recorded and the array is not materialised at all: this is the second + // of the four key arrays per call that the measurement found. } - // Mark ALL own names (incl non-enumerable) so they shadow the remainder - // of the chain — but only once the set is live. Until then the level is - // recorded and the array is not materialised at all: this is the second - // of the four key arrays per call that the measurement found. + if shadow_live { - mark_own_names(current, &mut seen, &mut scratch, diag); + mark_own_names(current.get_nanbox_f64(), &mut seen, &mut scratch, diag); } else { - visited.push(current); + visited.push(&scope, current.get_nanbox_f64()); } - current = super::super::object_ops::js_object_get_prototype_of(current); + current.set_nanbox_f64(super::super::object_ops::js_object_get_prototype_of( + current.get_nanbox_f64(), + )); level += 1; } - out + out.with_mut_ptr(|out: *mut ArrayHeader| out) } /// Prototype levels recorded for a possible shadow-set rebuild, inline for the @@ -445,37 +470,48 @@ pub(crate) fn for_in_keys_with(value: f64, lazy_shadow: bool) -> *mut ArrayHeade /// compiled claude-code TUI, so the heap arm is for prototype chains an order /// of magnitude deeper than anything the workload produces. It exists because /// the depth cap is 1000, not because it is expected. -struct VisitedLevels { - inline: [f64; Self::INLINE], +/// See `VisitedLevels`. A free const rather than an associated one: an +/// associated `Self::INLINE` is not permitted in the array length of a +/// generic struct. +const VISITED_INLINE: usize = 8; + +struct VisitedLevels<'s> { + inline: [Option>; VISITED_INLINE], len: usize, - spill: Vec, + spill: Vec>, } -impl Default for VisitedLevels { +impl Default for VisitedLevels<'_> { fn default() -> Self { Self { - inline: [0.0; Self::INLINE], + inline: [None; VISITED_INLINE], len: 0, spill: Vec::new(), } } } -impl VisitedLevels { - const INLINE: usize = 8; - - fn push(&mut self, v: f64) { - if self.len < Self::INLINE { - self.inline[self.len] = v; +impl<'s> VisitedLevels<'s> { + /// #9864 follow-up: a recorded level is a NaN-boxed heap pointer that is + /// dereferenced later, by `build_shadow_set`, after the walk has crossed + /// `js_object_keys_value` (which allocates) and `getPrototypeOf` (which + /// can run a Proxy trap). Stored as a plain `f64` it goes stale across + /// any collection in that window; stored as a handle the collector + /// rewrites it. `RuntimeHandle` is `Copy`, so the inline arm still costs + /// no allocation. + fn push(&mut self, scope: &'s crate::gc::RuntimeHandleScope, v: f64) { + let handle = scope.root_nanbox_f64(v); + if self.len < VISITED_INLINE { + self.inline[self.len] = Some(handle); self.len += 1; } else { - self.spill.push(v); + self.spill.push(handle); } } /// The recorded levels in walk order. Borrows rather than copies, and the /// spill arm concatenates only when it is non-empty. - fn as_slice(&self) -> VisitedSlice<'_> { + fn as_slice(&self) -> VisitedSlice<'_, 's> { VisitedSlice { head: &self.inline[..self.len], tail: &self.spill, @@ -483,14 +519,20 @@ impl VisitedLevels { } } -struct VisitedSlice<'a> { - head: &'a [f64], - tail: &'a [f64], +struct VisitedSlice<'a, 's> { + head: &'a [Option>], + tail: &'a [crate::gc::RuntimeHandle<'s>], } -impl VisitedSlice<'_> { - fn iter(&self) -> impl Iterator { - self.head.iter().chain(self.tail.iter()) +impl VisitedSlice<'_, '_> { + /// Read each level FRESH from its handle — a level recorded before a + /// collection has been rewritten in place by then. + fn iter(&self) -> impl Iterator + '_ { + self.head + .iter() + .filter_map(|h| h.as_ref()) + .map(|h| h.get_nanbox_f64()) + .chain(self.tail.iter().map(|h| h.get_nanbox_f64())) } } @@ -555,13 +597,13 @@ fn mark_own_names( /// at most once per `for-in`, and only when a level >= 1 has an enumerable key /// that something closer might hide. fn build_shadow_set( - visited: VisitedSlice<'_>, + visited: VisitedSlice<'_, '_>, seen: &mut std::collections::HashSet, scratch: &mut [u8; crate::value::SHORT_STRING_MAX_LEN], diag: bool, ) { for recv in visited.iter() { - mark_own_names(*recv, seen, scratch, diag); + mark_own_names(recv, seen, scratch, diag); } } diff --git a/crates/perry-runtime/src/object/field_get_set/enumeration_tests.rs b/crates/perry-runtime/src/object/field_get_set/enumeration_tests.rs index 1e463c06af..af92382f34 100644 --- a/crates/perry-runtime/src/object/field_get_set/enumeration_tests.rs +++ b/crates/perry-runtime/src/object/field_get_set/enumeration_tests.rs @@ -84,7 +84,7 @@ mod lazy_shadow_tests { ); } - /// A prototype chain deeper than `VisitedLevels::INLINE` where the ONLY + /// A prototype chain deeper than `VISITED_INLINE` where the ONLY /// level that shadows the name lives PAST the inline array. /// /// This arm never runs on the measured workload (the shadow set was built 0 @@ -115,7 +115,7 @@ mod lazy_shadow_tests { /// route is measuring the second route. #[test] fn only_a_spilled_level_shadows_the_root_and_the_rebuild_must_see_it() { - let shadow_level = VisitedLevels::INLINE + 2; + let shadow_level = VISITED_INLINE + 2; let depth = shadow_level + 2; // Root (deepest): the enumerable `marker` that must stay hidden. @@ -149,7 +149,7 @@ mod lazy_shadow_tests { ); assert!( !lazy.contains(&"marker".to_string()), - "the only level owning `marker` sits past VisitedLevels::INLINE, so \ + "the only level owning `marker` sits past VISITED_INLINE, so \ a rebuild that cannot see the spilled levels would leak the root's \ enumerable `marker` — got {lazy:?}" ); diff --git a/crates/perry-runtime/src/object/mod.rs b/crates/perry-runtime/src/object/mod.rs index 87dbfafc47..c2a3430c19 100644 --- a/crates/perry-runtime/src/object/mod.rs +++ b/crates/perry-runtime/src/object/mod.rs @@ -262,6 +262,8 @@ pub use class_meta_registry::{ js_register_class_extends_error, js_register_class_generic_origin, js_register_class_has_instance, js_register_class_to_string_tag, }; +#[cfg(test)] +pub(crate) use descriptor_state::test_may_have_descriptor_entry; pub use descriptor_state::PERRY_CLASS_FIELD_INLINE_GUARD_DISABLED; pub(crate) use descriptor_state::{ accessor_descriptor_keys_for_obj, class_field_inline_guard_enabled, diff --git a/crates/perry-runtime/src/proxy/reflect.rs b/crates/perry-runtime/src/proxy/reflect.rs index cecf2299cd..38ed6001dc 100644 --- a/crates/perry-runtime/src/proxy/reflect.rs +++ b/crates/perry-runtime/src/proxy/reflect.rs @@ -236,10 +236,11 @@ fn descriptor_key(name: &[u8]) -> (*const crate::StringHeader, f64) { } pub(super) unsafe fn descriptor_field_present(desc: f64, name: &[u8]) -> bool { + let scope = crate::gc::RuntimeHandleScope::new(); + let desc_handle = scope.root_nanbox_f64(desc); let (key, key_value) = descriptor_key(name); + let desc = desc_handle.get_nanbox_f64(); if lookup(desc).is_some() { - let scope = crate::gc::RuntimeHandleScope::new(); - let desc_handle = scope.root_nanbox_f64(desc); let key_handle = scope.root_nanbox_f64(key_value); return crate::value::js_is_truthy(js_proxy_has( desc_handle.get_nanbox_f64(), @@ -251,10 +252,11 @@ pub(super) unsafe fn descriptor_field_present(desc: f64, name: &[u8]) -> bool { } unsafe fn descriptor_field(desc: f64, name: &[u8]) -> f64 { + let scope = crate::gc::RuntimeHandleScope::new(); + let desc_handle = scope.root_nanbox_f64(desc); let (key, key_value) = descriptor_key(name); + let desc = desc_handle.get_nanbox_f64(); if lookup(desc).is_some() { - let scope = crate::gc::RuntimeHandleScope::new(); - let desc_handle = scope.root_nanbox_f64(desc); let key_handle = scope.root_nanbox_f64(key_value); return js_proxy_get(desc_handle.get_nanbox_f64(), key_handle.get_nanbox_f64()); } @@ -266,10 +268,12 @@ unsafe fn descriptor_field(desc: f64, name: &[u8]) -> f64 { } pub(super) unsafe fn descriptor_bool_field(desc: f64, name: &[u8]) -> Option { - if !descriptor_field_present(desc, name) { + let scope = crate::gc::RuntimeHandleScope::new(); + let desc_handle = scope.root_nanbox_f64(desc); + if !descriptor_field_present(desc_handle.get_nanbox_f64(), name) { return None; } - Some(crate::value::js_is_truthy(descriptor_field(desc, name)) != 0) + Some(crate::value::js_is_truthy(descriptor_field(desc_handle.get_nanbox_f64(), name)) != 0) } unsafe fn complete_proxy_descriptor_result(desc: f64) -> f64 { @@ -278,36 +282,42 @@ unsafe fn complete_proxy_descriptor_result(desc: f64) -> f64 { } let scope = crate::gc::RuntimeHandleScope::new(); let desc_handle = scope.root_nanbox_f64(desc); - let desc = desc_handle.get_nanbox_f64(); - let has_enumerable = descriptor_field_present(desc, b"enumerable"); - let has_configurable = descriptor_field_present(desc, b"configurable"); - let has_value = descriptor_field_present(desc, b"value"); - let has_writable = descriptor_field_present(desc, b"writable"); - let has_get = descriptor_field_present(desc, b"get"); - let has_set = descriptor_field_present(desc, b"set"); + let has_enumerable = descriptor_field_present(desc_handle.get_nanbox_f64(), b"enumerable"); + let has_configurable = descriptor_field_present(desc_handle.get_nanbox_f64(), b"configurable"); + let has_value = descriptor_field_present(desc_handle.get_nanbox_f64(), b"value"); + let has_writable = descriptor_field_present(desc_handle.get_nanbox_f64(), b"writable"); + let has_get = descriptor_field_present(desc_handle.get_nanbox_f64(), b"get"); + let has_set = descriptor_field_present(desc_handle.get_nanbox_f64(), b"set"); - let enumerable = - has_enumerable && crate::value::js_is_truthy(descriptor_field(desc, b"enumerable")) != 0; + let enumerable = has_enumerable + && crate::value::js_is_truthy(descriptor_field( + desc_handle.get_nanbox_f64(), + b"enumerable", + )) != 0; let configurable = has_configurable - && crate::value::js_is_truthy(descriptor_field(desc, b"configurable")) != 0; + && crate::value::js_is_truthy(descriptor_field( + desc_handle.get_nanbox_f64(), + b"configurable", + )) != 0; let value = if has_value { - descriptor_field(desc, b"value") + descriptor_field(desc_handle.get_nanbox_f64(), b"value") } else { f64::from_bits(TAG_UNDEFINED) }; let value_handle = scope.root_nanbox_f64(value); - let writable = - has_writable && crate::value::js_is_truthy(descriptor_field(desc, b"writable")) != 0; + let writable = has_writable + && crate::value::js_is_truthy(descriptor_field(desc_handle.get_nanbox_f64(), b"writable")) + != 0; let getter = if has_get { - descriptor_field(desc, b"get") + descriptor_field(desc_handle.get_nanbox_f64(), b"get") } else { f64::from_bits(TAG_UNDEFINED) }; let getter_handle = scope.root_nanbox_f64(getter); let setter = if has_set { - descriptor_field(desc, b"set") + descriptor_field(desc_handle.get_nanbox_f64(), b"set") } else { f64::from_bits(TAG_UNDEFINED) }; @@ -360,40 +370,61 @@ pub extern "C" fn js_reflect_get_own_property_descriptor(target: f64, key: f64) return revoked_return(); } - let trap = handler_trap(handler, "getOwnPropertyDescriptor"); + // The handler lookup, trap, and descriptor field getters can all run JS. + // A root is useful only if each post-callback read reloads its current value. + let inner_handle = scope.root_nanbox_f64(inner); + let handler_handle = scope.root_nanbox_f64(handler); + let trap = handler_trap(handler_handle.get_nanbox_f64(), "getOwnPropertyDescriptor"); let trap_bits = trap.to_bits(); if trap_bits == TAG_UNDEFINED || trap_bits == TAG_NULL { // No trap — forward to the target's [[GetOwnProperty]]. When the target // is itself a Proxy, recurse through the Reflect entry point rather than // the ordinary object path, which would deref the fake proxy pointer. - if lookup(inner).is_some() { - return js_reflect_get_own_property_descriptor(inner, property_key); + if lookup(inner_handle.get_nanbox_f64()).is_some() { + return js_reflect_get_own_property_descriptor( + inner_handle.get_nanbox_f64(), + property_key_handle.get_nanbox_f64(), + ); } - return crate::object::js_object_get_own_property_descriptor(inner, property_key); + return crate::object::js_object_get_own_property_descriptor( + inner_handle.get_nanbox_f64(), + property_key_handle.get_nanbox_f64(), + ); } if !is_callable_function(trap) { return throw_type_error("proxy getOwnPropertyDescriptor trap is not a function"); } - let rebound = crate::closure::clone_closure_rebind_this(trap_bits, handler); + let rebound = + crate::closure::clone_closure_rebind_this(trap_bits, handler_handle.get_nanbox_f64()); let closure = closure_from(f64::from_bits(rebound)); if closure.is_null() { return throw_type_error("proxy getOwnPropertyDescriptor trap is not a function"); } let this_scope = crate::gc::RuntimeHandleScope::new(); // #9445 - let prev = this_scope.root_nanbox_f64(crate::object::js_implicit_this_set(handler)); - let result = js_closure_call2(closure, inner, property_key); + let prev = this_scope.root_nanbox_f64(crate::object::js_implicit_this_set( + handler_handle.get_nanbox_f64(), + )); + let result = js_closure_call2( + closure, + inner_handle.get_nanbox_f64(), + property_key_handle.get_nanbox_f64(), + ); crate::object::js_implicit_this_set(prev.get_nanbox_f64()); let result_handle = scope.root_nanbox_f64(result); - let target_desc = crate::object::js_object_get_own_property_descriptor(inner, property_key); + let target_desc = crate::object::js_object_get_own_property_descriptor( + inner_handle.get_nanbox_f64(), + property_key_handle.get_nanbox_f64(), + ); let target_desc_handle = scope.root_nanbox_f64(target_desc); let result = result_handle.get_nanbox_f64(); - let target_desc = target_desc_handle.get_nanbox_f64(); if result.to_bits() == TAG_UNDEFINED { - if target_desc.to_bits() != TAG_UNDEFINED - && (crate::object::obj_value_no_extend(inner) - || unsafe { descriptor_bool_field(target_desc, b"configurable") } == Some(false)) + if target_desc_handle.get_nanbox_u64() != TAG_UNDEFINED + && (crate::object::obj_value_no_extend(inner_handle.get_nanbox_f64()) + || unsafe { + descriptor_bool_field(target_desc_handle.get_nanbox_f64(), b"configurable") + } == Some(false)) { return throw_type_error( "proxy getOwnPropertyDescriptor trap cannot hide target property", @@ -407,16 +438,17 @@ pub extern "C" fn js_reflect_get_own_property_descriptor(target: f64, key: f64) } let result = unsafe { complete_proxy_descriptor_result(result) }; let result_handle = scope.root_nanbox_f64(result); - let result = result_handle.get_nanbox_f64(); - if target_desc.to_bits() == TAG_UNDEFINED { - if crate::object::obj_value_no_extend(inner) { + if target_desc_handle.get_nanbox_u64() == TAG_UNDEFINED { + if crate::object::obj_value_no_extend(inner_handle.get_nanbox_f64()) { return throw_type_error( "proxy getOwnPropertyDescriptor trap reports new property on non-extensible target", ); } - } else if unsafe { descriptor_bool_field(target_desc, b"configurable") } == Some(false) - && unsafe { descriptor_bool_field(result, b"configurable") } == Some(true) + } else if unsafe { descriptor_bool_field(target_desc_handle.get_nanbox_f64(), b"configurable") } + == Some(false) + && unsafe { descriptor_bool_field(result_handle.get_nanbox_f64(), b"configurable") } + == Some(true) { return throw_type_error( "proxy getOwnPropertyDescriptor trap reports incompatible descriptor", @@ -425,9 +457,13 @@ pub extern "C" fn js_reflect_get_own_property_descriptor(target: f64, key: f64) // [[GetOwnProperty]] step 21.a: a non-configurable result descriptor is only // valid for a non-configurable existing target property. - if unsafe { descriptor_bool_field(result, b"configurable") } == Some(false) { - let target_configurable = target_desc.to_bits() == TAG_UNDEFINED - || unsafe { descriptor_bool_field(target_desc, b"configurable") } != Some(false); + if unsafe { descriptor_bool_field(result_handle.get_nanbox_f64(), b"configurable") } + == Some(false) + { + let target_configurable = target_desc_handle.get_nanbox_u64() == TAG_UNDEFINED + || unsafe { + descriptor_bool_field(target_desc_handle.get_nanbox_f64(), b"configurable") + } != Some(false); if target_configurable { return throw_type_error( "proxy getOwnPropertyDescriptor trap reports a non-configurable descriptor for a configurable or absent target property", @@ -435,5 +471,5 @@ pub extern "C" fn js_reflect_get_own_property_descriptor(target: f64, key: f64) } } - result + result_handle.get_nanbox_f64() } diff --git a/crates/perry-runtime/src/regex.rs b/crates/perry-runtime/src/regex.rs index 683af8750a..81d25639f2 100644 --- a/crates/perry-runtime/src/regex.rs +++ b/crates/perry-runtime/src/regex.rs @@ -262,6 +262,120 @@ pub(crate) unsafe fn regex_header_finalize_for_gc(re: *mut RegExpHeader) { regex_header_clear_dead_for_gc(re as usize); } +/// Finalize the RegExp headers that died in from-space during a copied minor. +/// +/// The copying minor's from-space flip runs no per-object finalize hooks, so +/// a nursery header that was neither evacuated nor pinned would otherwise keep +/// its `Arc` programs and its `REGEX_POINTERS` / `REGEX_SOURCE_TABLE` / expando +/// entries forever. Same shape as `map::finalize_dead_copied_minor_from_space_maps`: +/// walk the registry after the flip, collect the provably-dead addresses, then +/// finalize each (the finalizer removes its own registry entries, which is why +/// the walk and the removal are two passes). +/// +/// Cost: O(registry) = O(live headers + headers allocated since the last +/// minor) — the same order as the malloc sweep this replaces, and +/// proportional to allocation, not to program history. +pub(crate) fn finalize_dead_copied_minor_from_space_regexps() -> usize { + let dead: Vec = REGEX_POINTERS.with(|table| { + table + .borrow() + .iter() + .copied() + .filter(|&addr| { + crate::gc::owner_is_dead_copied_minor_from_space_of_type( + addr, + crate::gc::GC_TYPE_REGEXP, + ) + }) + .collect() + }); + let count = dead.len(); + for addr in dead { + unsafe { regex_header_finalize_for_gc(addr as *mut RegExpHeader) }; + } + count +} + +/// Sweep-entry twin of the above for the non-copying cycle kinds (fallback +/// minor / full mark-sweep): a dead header in the ACTIVE nursery allocation +/// block is never object-walked by any sweeper, so it is collected from the +/// registry right after trace instead (#6010, mirroring Map/Set/Buffer). +/// Deadness: unmarked ∧ not pinned ∧ not forwarded, and for a minor trace also +/// not tenured and physically in the nursery. +pub(crate) fn collect_dead_registered_regexps_post_trace(full_trace: bool) -> Vec { + REGEX_POINTERS.with(|table| { + table + .borrow() + .iter() + .copied() + .filter(|&addr| unsafe { registered_regexp_is_dead_post_trace(addr, full_trace) }) + .collect() + }) +} + +/// Finalize one collected-dead RegExp (budget-chunked by the sweep state). +pub(crate) fn finalize_collected_dead_regexp(addr: usize) { + unsafe { regex_header_finalize_for_gc(addr as *mut RegExpHeader) }; +} + +unsafe fn registered_regexp_is_dead_post_trace(addr: usize, full_trace: bool) -> bool { + let Some(header) = crate::value::addr_class::try_read_gc_header(addr) else { + return false; + }; + if header.obj_type != crate::gc::GC_TYPE_REGEXP { + return false; + } + let flags = header.gc_flags; + if flags + & (crate::gc::GC_FLAG_MARKED | crate::gc::GC_FLAG_PINNED | crate::gc::GC_FLAG_FORWARDED) + != 0 + { + return false; + } + if full_trace { + return true; + } + if flags & crate::gc::GC_FLAG_TENURED != 0 { + return false; + } + matches!( + crate::arena::classify_heap_generation(addr), + crate::arena::HeapGeneration::Nursery + ) +} + +/// Test support: construct a RegExp through the PRODUCTION path +/// (`js_regexp_new`), run one `test()` so the compiled programs are installed +/// on the header, and hand the header back unrooted. +#[cfg(all(test, feature = "regex-engine"))] +pub(crate) fn test_construct_regexp_and_exec_once(pattern: &str, flags: &str) -> *mut RegExpHeader { + let scope = crate::gc::RuntimeHandleScope::new(); + let p = scope.root_string_ptr(js_string_from_str(pattern)); + let f = scope.root_string_ptr(js_string_from_str(flags)); + let re = p.with_mut_ptr::(|p| { + f.with_mut_ptr::(|f| js_regexp_new(p, f)) + }); + let subject = scope.root_string_ptr(js_string_from_str("abc")); + subject.with_const_ptr::(|s| { + let _ = js_regexp_test(re, s); + }); + re +} + +/// Test support: strong count of the standard program a header holds (the +/// observer clone taken here is released before returning). +#[cfg(all(test, feature = "regex-engine"))] +pub(crate) fn test_regexp_std_program_strong_count(re: *const RegExpHeader) -> usize { + unsafe { + let raw = (*re).regex_ptr as *const Regex; + assert!(!raw.is_null(), "program must be installed"); + let arc = Arc::from_raw(raw); + let n = Arc::strong_count(&arc); + std::mem::forget(arc); + n + } +} + #[cfg(test)] pub(crate) fn test_regex_pointer_entry_exists(addr: usize) -> bool { REGEX_POINTERS.with(|table| table.borrow().contains(&addr)) @@ -532,7 +646,7 @@ fn shared_never_match_program() -> Arc { } NEVER_MATCH.with(|slot| { slot.borrow_mut() - .get_or_insert_with(|| Arc::new(Regex::new(NEVER_MATCH_SOURCE).unwrap())) + .get_or_insert_with(|| Arc::new(Regex::new(NEVER_MATCH_PATTERN).unwrap())) .clone() }) } @@ -846,7 +960,7 @@ pub extern "C" fn js_regexp_new( ) -> *mut RegExpHeader { // ★ `pattern` is a raw `StringHeader*` in a Rust local, and this function // allocates twice below (`js_string_from_str` for the canonical flags, then - // `gc_malloc` for the header). Either can drive an evacuating minor that + // `arena_alloc_gc` for the header). Either can drive an evacuating minor that // relocates the pattern string, after which this argument names retired // from-space — and it is then *stored into the header* as `pattern_ptr`, // so the damage is permanent rather than transient. @@ -1055,10 +1169,35 @@ pub extern "C" fn js_regexp_new( #[allow(unused_variables)] let pattern_str: () = (); - // Allocate the header via gc_malloc so it's tracked by the GC and gets - // freed when no longer referenced. Previously this used raw alloc() and - // leaked every header, which was a 64-byte-per-call leak on top of the - // (now-fixed) regex object leak. + // ★ The header is NURSERY-allocated, like an ordinary object. + // + // It used to be `gc_malloc`'d: raw `alloc()` at first (a 64-byte leak per + // construction), then the tracked malloc arm so the sweep could free it. + // That arm costs, PER CONSTRUCTION, a mimalloc allocation, a push onto + // `MALLOC_STATE.objects`, an insert into the malloc-registry `PtrHashSet` + // (which rehashes as it grows), two old→young remembered-set entries for + // `pattern_ptr`/`flags_ptr`, and — at death — a malloc-sweep visit, the + // finalizer and a free. A JS regex literal constructs a fresh object every + // time it is evaluated, so on the claude-code TUI `PERRY_GC_TRACE` counted + // **199,873 RegExp headers malloc'd per 400-character reply — 100.0 % of + // all malloc allocations** — with the registry swinging 26,690 → 1,689 + // across one minor: ~94 % of them die young and were paying + // old-generation prices to do it. + // + // `GC_TYPE_REGEXP` has been movable (`GcMoveHookKind::RegExpSideTables` + // rekeys `REGEX_POINTERS`, `REGEX_SOURCE_TABLE` and the expando owner + // after evacuation; `GcLayoutSlotKind::RegExpFields` traces the two string + // edges and `meta`) since the copying collector landed, and + // `test_movable_regexp_evacuation_migrates_all_address_owned_state` has + // exercised the arena arm all along. What kept production on malloc was + // finalization: the copying minor's from-space flip runs no per-object + // finalize hooks (`gc::copying`), so a nursery header that dies young + // would leak its three `Arc` programs and its registry entries. That is + // now handled the way Map/Set/Error handle theirs — + // `finalize_dead_copied_minor_from_space_regexps` after a copied minor and + // `collect_dead_registered_regexps_post_trace` at sweep entry for the + // non-copying cycle kinds — and a tenured header is finalized by the + // old-generation sweep's ordinary `gc_type_finalize_unmarked_payload`. let header_size = std::mem::size_of::(); // `flags_ptr` must hold the CANONICAL form, so that `flags_ptr`-keyed // lookups (FANCY_CACHE, lookup_fancy_regex) and the GC-survivable source @@ -1075,12 +1214,12 @@ pub extern "C" fn js_regexp_new( scope.root_string_ptr(js_string_from_str(flags_str)) } }; - // ★ #7341: root the canonical flags string too. The `gc_malloc` below is an - // allocation and therefore a collection point, exactly as the comment above - // `pattern_root` says — but only the PATTERN was rooted and re-read. The - // flags string is created here and stored into the header AFTER that - // allocation, so an evacuating minor in `gc_malloc` moved it and the header - // kept the pre-collection address. `flags_ptr` is then permanently stale in + // ★ #7341: root the canonical flags string too. The header allocation below + // is an allocation and therefore a collection point, exactly as the comment + // above `pattern_root` says — but only the PATTERN was rooted and re-read. + // The flags string is created here and stored into the header AFTER that + // allocation, so an evacuating minor in the header allocation moved it and + // the header kept the pre-collection address. `flags_ptr` is then permanently stale in // a live header: `lookup_fancy_regex` reads it through `string_as_str` and // faults on retired from-space, which is 5 of the 31 catches in #7341 // (four different callers, all reaching that one read). @@ -1089,7 +1228,11 @@ pub extern "C" fn js_regexp_new( // missing is that the value written had to survive the allocation first. unsafe { - let raw = crate::gc::gc_malloc(header_size, crate::gc::GC_TYPE_REGEXP); + let raw = crate::arena::arena_alloc_gc( + header_size, + std::mem::align_of::(), + crate::gc::GC_TYPE_REGEXP, + ); if raw.is_null() { // #5067 — catchable RangeError instead of aborting on OOM. crate::error::throw_allocation_failed(); @@ -1114,19 +1257,24 @@ pub extern "C" fn js_regexp_new( (*ptr).pattern_ptr = pattern; (*ptr).flags_ptr = canonical_flags_ptr; // `pattern_ptr` / `flags_ptr` are GC-managed StringHeaders — the GC scans - // this 2-slot payload range via the magic-tagged RegExp layout, and - // `canonical_flags_ptr` (js_string_from_str above) is a freshly-allocated - // YOUNG string. They are stored into this malloc'd (old-generation) header - // by raw writes; without a write barrier the old→young edge is never - // remembered, so a copying minor GC sweeps the string while the retained - // RegExp still points at it. The evacuation verifier reports this as an - // uncovered object→string edge, and it crashes for real when the freed - // slot is later scanned/read (a heavy regex workload — e.g. ANSI/emoji - // parsing in a terminal UI — hits it within seconds). Remember both edges, - // mirroring every other native-header pointer store (closure captures, - // object prototype slots, array headers). `runtime_write_barrier_gc_slot` - // detects the malloc parent and only remembers genuinely-young children, - // so an already-old/interned `pattern` is a harmless no-op. + // this 2-slot payload range via the magic-tagged RegExp layout. + // + // The header is young now, so for a young child the barrier records + // nothing; it still has to run, because a header born while a budgeted + // cycle is marking is allocated black, and because a header that has + // been promoted and then reassigned (`RegExp.prototype.compile`) is a + // genuine old→young store. Historically the header was malloc'd, i.e. + // old, and this store was THE old→young edge a copying minor would + // otherwise miss: the evacuation verifier reported it as an uncovered + // object→string edge, and it crashed for real when the freed slot was + // later scanned/read (a heavy regex workload — e.g. ANSI/emoji parsing + // in a terminal UI — hit it within seconds). + // + // Remember both edges, mirroring every other native-header pointer + // store (closure captures, object prototype slots, array headers). + // `runtime_write_barrier_gc_slot` classifies the parent and only + // remembers genuinely-young children, so an already-old/interned + // `pattern` is a harmless no-op. let regexp_parent_addr = ptr as usize; if !pattern.is_null() { crate::gc::runtime_write_barrier_gc_slot( @@ -1484,7 +1632,7 @@ fn linear_rules_out_match(re: *const RegExpHeader, subject: &str, start: usize) return false; } let program: &Regex = &*program; - if program.as_str() == NEVER_MATCH_SOURCE { + if program.as_str() == NEVER_MATCH_PATTERN { // The `regex` crate refused this pattern (lookaround / // backreference); it has no opinion about the subject. return false; @@ -1493,13 +1641,6 @@ fn linear_rules_out_match(re: *const RegExpHeader, subject: &str, start: usize) } } -/// The source of the never-match program `compile_and_cache_regex_checked` -/// installs for a pattern only another engine can serve. Compared by TEXT -/// rather than by `Arc` identity so this stays independent of how the -/// placeholder is allocated. -#[cfg(feature = "regex-engine")] -const NEVER_MATCH_SOURCE: &str = r"[^\s\S]"; - /// [`lookup_repeat_matcher`] with the linear pre-check applied: `None` also /// when the linear program proves no match at or after `start`, so the /// backtracker is never entered on a subject that cannot match. Every diff --git a/crates/perry-runtime/src/regex/lazy.rs b/crates/perry-runtime/src/regex/lazy.rs index a982a34875..f5492c317f 100644 --- a/crates/perry-runtime/src/regex/lazy.rs +++ b/crates/perry-runtime/src/regex/lazy.rs @@ -38,8 +38,8 @@ //! engines refuse); //! * `.source` / `.flags` / `.global` / `.sticky` / `lastIndex` are header //! and side-table reads that never touched the compiled program; -//! * identity is untouched — `js_regexp_new` still `gc_malloc`s a fresh -//! header per evaluation. +//! * identity is untouched — `js_regexp_new` still allocates a fresh header +//! per evaluation. //! //! The build itself happens on the first operation that needs a matcher, //! through [`ensure_regex_compiled`], and installs exactly the pointers diff --git a/crates/perry-runtime/src/regex/tests.rs b/crates/perry-runtime/src/regex/tests.rs index 2c571153c4..cc5908b41e 100644 --- a/crates/perry-runtime/src/regex/tests.rs +++ b/crates/perry-runtime/src/regex/tests.rs @@ -1884,3 +1884,67 @@ fn quantified_capture_pattern_does_not_backtrack_on_a_non_matching_subject() { 1 ); } + +/// #6759 phase 1 follow-up: a `RegExp` receiver can now answer the +/// descriptor-summary probe. Before the meta edge was wired for +/// `GC_TYPE_REGEXP`, `may_have_descriptor_entry` answered the conservative +/// `true` for every RegExp, so `set_last_index_throwing` built a `String` and +/// SipHashed `(usize, String)` on every global/sticky `test()`/`exec()`. +#[test] +fn a_fresh_regexp_proves_lastindex_absent_without_probing_the_tables() { + let _lock = crate::gc::global_side_table_test_lock(); + let scope = crate::gc::RuntimeHandleScope::new(); + let pattern = scope.root_string_ptr(make_string("x")); + let flags = scope.root_string_ptr(make_string("g")); + let re = pattern.with_mut_ptr::(|pattern| { + flags.with_mut_ptr::(|flags| js_regexp_new(pattern, flags)) + }); + // Premise: this really is the dedicated RegExp cell, not a shaped object + // that would have answered through the ordinary `GC_TYPE_OBJECT` path. + let gc = unsafe { crate::value::addr_class::try_read_gc_header(re as usize) } + .expect("RegExp must be a GC allocation"); + assert_eq!(gc.obj_type, crate::gc::GC_TYPE_REGEXP); + + assert!( + !crate::object::test_may_have_descriptor_entry(re as usize, "lastIndex", false), + "a fresh RegExp has no descriptors, so the meta summary must prove \ + `lastIndex` absent instead of sending the caller to the table" + ); + assert!( + crate::object::get_property_attrs(re as usize, "lastIndex").is_none(), + "and the answer the fast path skips must be the same one" + ); +} + +/// The other half, and the one that makes the fast negative safe: an owner +/// that DOES have a descriptor must still be found. Install and probe share +/// one predicate, so a probe widened without its install would answer +/// "proven absent" here and `set_last_index_throwing` would silently stop +/// throwing (test262 prototype/{exec,test}/y-fail-lastindex-no-write). +#[test] +fn a_regexp_with_a_non_writable_lastindex_is_still_found_by_the_probe() { + let _lock = crate::gc::global_side_table_test_lock(); + let scope = crate::gc::RuntimeHandleScope::new(); + let pattern = scope.root_string_ptr(make_string("x")); + let flags = scope.root_string_ptr(make_string("g")); + let re = pattern.with_mut_ptr::(|pattern| { + flags.with_mut_ptr::(|flags| js_regexp_new(pattern, flags)) + }); + let attrs = crate::object::PropertyAttrs::new(false, true, true); + crate::object::set_property_attrs(re as usize, "lastIndex".to_string(), attrs); + + assert!( + crate::object::test_may_have_descriptor_entry(re as usize, "lastIndex", false), + "the install set the key bit, so the probe must send the caller to the table" + ); + let found = crate::object::get_property_attrs(re as usize, "lastIndex") + .expect("the descriptor the test installed must be readable back"); + assert!(!found.writable(), "and it must still read as non-writable"); + + // A DIFFERENT key on the same owner stays proven-absent: the summary is + // per key, not per owner, so widening it must not blunt it. + assert!( + !crate::object::test_may_have_descriptor_entry(re as usize, "source", false), + "an unrelated key on the same RegExp must still take the fast negative" + ); +} diff --git a/crates/perry-runtime/src/tls_hot.rs b/crates/perry-runtime/src/tls_hot.rs index 217176590e..7a1c7a7e18 100644 --- a/crates/perry-runtime/src/tls_hot.rs +++ b/crates/perry-runtime/src/tls_hot.rs @@ -132,7 +132,7 @@ pub(crate) struct HotTls { // arena/block.rs pub(crate) arena: *mut u8, pub(crate) inline_state: *mut u8, - // arena/page_meta.rs + // arena/page_meta/mod.rs pub(crate) page_generation_cache: *mut u8, pub(crate) page_generations: *mut u8, // gc/malloc.rs diff --git a/crates/perry-stdlib/src/fetch/body_metadata.rs b/crates/perry-stdlib/src/fetch/body_metadata.rs index 4074a10e1c..0695ff11ac 100644 --- a/crates/perry-stdlib/src/fetch/body_metadata.rs +++ b/crates/perry-stdlib/src/fetch/body_metadata.rs @@ -157,6 +157,90 @@ fn file_last_modified_now() -> f64 { .unwrap_or(0.0) } +unsafe fn form_data_entry_from_js(value: f64, filename: f64) -> FormDataValue { + let value_id = handle_id(value); + let blob = JSValue::from_bits(value.to_bits()) + .is_pointer() + .then(|| BLOB_REGISTRY.lock().unwrap().get(&value_id).cloned()) + .flatten(); + let Some(mut blob) = blob else { + return FormDataValue::Text(form_data_value_string(value)); + }; + + let filename_override = + (filename.to_bits() != TAG_UNDEFINED).then(|| form_data_value_string(filename)); + if filename_override.is_none() && blob.file_name.is_some() { + return FormDataValue::File(value_id); + } + + blob.file_name = Some( + filename_override + .or(blob.file_name) + .unwrap_or_else(|| "blob".to_string()), + ); + blob.last_modified_ms = Some(file_last_modified_now()); + FormDataValue::File(alloc_blob(blob)) +} + +fn multipart_quoted(value: &str) -> String { + let mut escaped = String::with_capacity(value.len()); + for ch in value.chars() { + match ch { + '\r' => escaped.push_str("%0D"), + '\n' => escaped.push_str("%0A"), + '"' => escaped.push_str("%22"), + _ => escaped.push(ch), + } + } + escaped +} + +pub(super) fn serialize_form_data(handle: usize) -> Option<(Vec, String)> { + static NEXT_BOUNDARY: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(1); + + let entries = FORM_DATA_REGISTRY.lock().unwrap().get(&handle)?.clone(); + let serial = NEXT_BOUNDARY.fetch_add(1, std::sync::atomic::Ordering::Relaxed); + let boundary = format!("----PerryFormDataBoundary{handle:012x}{serial:016x}"); + let mut body = Vec::new(); + + for (name, value) in entries.entries { + body.extend_from_slice(format!("--{boundary}\r\n").as_bytes()); + match value { + FormDataValue::Text(value) => { + body.extend_from_slice( + format!( + "Content-Disposition: form-data; name=\"{}\"\r\n\r\n", + multipart_quoted(&name) + ) + .as_bytes(), + ); + body.extend_from_slice(value.as_bytes()); + } + FormDataValue::File(blob_id) => { + let blob = BLOB_REGISTRY.lock().unwrap().get(&blob_id)?.clone(); + let filename = blob.file_name.as_deref().unwrap_or("blob"); + let content_type = if blob.content_type.is_empty() { + "application/octet-stream" + } else { + &blob.content_type + }; + body.extend_from_slice( + format!( + "Content-Disposition: form-data; name=\"{}\"; filename=\"{}\"\r\nContent-Type: {content_type}\r\n\r\n", + multipart_quoted(&name), + multipart_quoted(filename), + ) + .as_bytes(), + ); + body.extend_from_slice(&blob.body); + } + } + body.extend_from_slice(b"\r\n"); + } + body.extend_from_slice(format!("--{boundary}--\r\n").as_bytes()); + Some((body, format!("multipart/form-data; boundary={boundary}"))) +} + fn form_data_from_multipart( body: &[u8], content_type: &str, @@ -509,23 +593,41 @@ pub extern "C" fn js_form_data_new() -> f64 { } #[no_mangle] -pub unsafe extern "C" fn js_form_data_append(handle: f64, name: f64, value: f64) -> f64 { +pub unsafe extern "C" fn js_form_data_append( + handle: f64, + name: f64, + value: f64, + filename: f64, +) -> f64 { let id = handle_id(handle); - let name = form_data_value_string(name); - let value = form_data_value_string(value); + let scope = perry_runtime::gc::RuntimeHandleScope::new(); + let name = scope.root_nanbox_f64(name); + let value = scope.root_nanbox_f64(value); + let filename = scope.root_nanbox_f64(filename); + let name = form_data_value_string(name.get_nanbox_f64()); + let value = form_data_entry_from_js(value.get_nanbox_f64(), filename.get_nanbox_f64()); if let Some(form) = FORM_DATA_REGISTRY.lock().unwrap().get_mut(&id) { - form.append(name, FormDataValue::Text(value)); + form.append(name, value); } f64::from_bits(TAG_UNDEFINED) } #[no_mangle] -pub unsafe extern "C" fn js_form_data_set(handle: f64, name: f64, value: f64) -> f64 { +pub unsafe extern "C" fn js_form_data_set( + handle: f64, + name: f64, + value: f64, + filename: f64, +) -> f64 { let id = handle_id(handle); - let name = form_data_value_string(name); - let value = form_data_value_string(value); + let scope = perry_runtime::gc::RuntimeHandleScope::new(); + let name = scope.root_nanbox_f64(name); + let value = scope.root_nanbox_f64(value); + let filename = scope.root_nanbox_f64(filename); + let name = form_data_value_string(name.get_nanbox_f64()); + let value = form_data_entry_from_js(value.get_nanbox_f64(), filename.get_nanbox_f64()); if let Some(form) = FORM_DATA_REGISTRY.lock().unwrap().get_mut(&id) { - form.set(name, FormDataValue::Text(value)); + form.set(name, value); } f64::from_bits(TAG_UNDEFINED) } @@ -695,6 +797,12 @@ pub fn form_data_contains_handle(handle: usize) -> bool { mod tests { use super::*; + unsafe fn string_value(value: &str) -> f64 { + f64::from_bits( + JSValue::string_ptr(js_string_from_bytes(value.as_ptr(), value.len() as u32)).bits(), + ) + } + #[test] fn selects_urlencoded_and_multipart_parsers_from_content_type() { let encoded = form_data_from_body( @@ -724,4 +832,111 @@ mod tests { assert!(form_data_from_body(b"{}", "application/json").is_err()); } + + #[test] + fn appended_blob_becomes_a_file_and_serializes_binary_multipart() { + let blob_id = alloc_blob(BlobData::blob( + vec![0, 0xff, b'\r', b'\n'], + "application/octet-stream".to_string(), + )); + let form = js_form_data_new(); + unsafe { + js_form_data_append( + form, + string_value("bin\r\nname"), + handle_to_f64(blob_id), + string_value("a\"b.bin"), + ); + } + + let form_id = handle_id(form); + let stored_entry = FORM_DATA_REGISTRY + .lock() + .unwrap() + .get(&form_id) + .unwrap() + .entries[0] + .1 + .clone(); + let stored_blob_id = match stored_entry { + FormDataValue::File(id) => id, + FormDataValue::Text(_) => panic!("Blob was stringified"), + }; + let stored_blob = BLOB_REGISTRY + .lock() + .unwrap() + .get(&stored_blob_id) + .unwrap() + .clone(); + assert_eq!(stored_blob.file_name.as_deref(), Some("a\"b.bin")); + + let (body, content_type) = serialize_form_data(form_id).unwrap(); + assert!(content_type.starts_with("multipart/form-data; boundary=")); + let wire = String::from_utf8_lossy(&body); + assert!(wire.contains("name=\"bin%0D%0Aname\"")); + assert!(wire.contains("filename=\"a%22b.bin\"")); + + let parsed = form_data_from_body(&body, &content_type).unwrap(); + let parsed_blob_id = match &parsed.entries[0].1 { + FormDataValue::File(id) => *id, + FormDataValue::Text(_) => panic!("serialized Blob parsed as text"), + }; + let parsed_blob = BLOB_REGISTRY + .lock() + .unwrap() + .get(&parsed_blob_id) + .unwrap() + .clone(); + assert_eq!(parsed_blob.body, [0, 0xff, b'\r', b'\n']); + assert_eq!(parsed_blob.file_name.as_deref(), Some("a%22b.bin")); + assert_eq!(parsed_blob.content_type, "application/octet-stream"); + } + + #[test] + fn request_owns_serialized_form_data_and_default_content_type() { + let form = js_form_data_new(); + unsafe { + js_form_data_append( + form, + string_value("caption"), + string_value("hello"), + f64::from_bits(TAG_UNDEFINED), + ); + } + let scope = perry_runtime::gc::RuntimeHandleScope::new(); + let url = scope.root_string_ptr(js_string_from_bytes(b"http://example.test/".as_ptr(), 20)); + let method = scope.root_string_ptr(js_string_from_bytes(b"POST".as_ptr(), 4)); + let request = unsafe { + js_request_new( + url.get_raw_const_ptr(), + method.get_raw_const_ptr(), + handle_id(form) as *const StringHeader, + 0.0, + std::ptr::null(), + std::ptr::null(), + std::ptr::null(), + std::ptr::null(), + std::ptr::null(), + std::ptr::null(), + std::ptr::null(), + f64::from_bits(TAG_FALSE), + std::ptr::null(), + f64::from_bits(TAG_UNDEFINED), + ) + }; + let request_id = handle_id(request); + let request = REQUEST_REGISTRY + .lock() + .unwrap() + .get(&request_id) + .unwrap() + .clone(); + let content_type = request.headers.get("content-type").unwrap(); + assert!(content_type.starts_with("multipart/form-data; boundary=")); + let parsed = form_data_from_body(request.body.as_deref().unwrap(), &content_type).unwrap(); + assert!(matches!( + &parsed.entries[0], + (name, FormDataValue::Text(value)) if name == "caption" && value == "hello" + )); + } } diff --git a/crates/perry-stdlib/src/fetch/dispatch.rs b/crates/perry-stdlib/src/fetch/dispatch.rs index 5aea55446f..2fc06a1a7c 100644 --- a/crates/perry-stdlib/src/fetch/dispatch.rs +++ b/crates/perry-stdlib/src/fetch/dispatch.rs @@ -686,6 +686,9 @@ pub fn dispatch_form_data_method(form_id: usize, method: &str, args: &[f64]) -> args.get(1) .copied() .unwrap_or(f64::from_bits(TAG_UNDEFINED)), + args.get(2) + .copied() + .unwrap_or(f64::from_bits(TAG_UNDEFINED)), )), "set" => Some(js_form_data_set( form_f64, @@ -695,6 +698,9 @@ pub fn dispatch_form_data_method(form_id: usize, method: &str, args: &[f64]) -> args.get(1) .copied() .unwrap_or(f64::from_bits(TAG_UNDEFINED)), + args.get(2) + .copied() + .unwrap_or(f64::from_bits(TAG_UNDEFINED)), )), "delete" => Some(js_form_data_delete(form_f64, str_arg(0))), "get" => Some(js_form_data_get(form_f64, str_arg(0))), diff --git a/crates/perry-stdlib/src/fetch/mod.rs b/crates/perry-stdlib/src/fetch/mod.rs index c80dd3cfad..a467c21ffb 100644 --- a/crates/perry-stdlib/src/fetch/mod.rs +++ b/crates/perry-stdlib/src/fetch/mod.rs @@ -678,9 +678,17 @@ pub unsafe extern "C" fn js_fetch_post( // so a binary body (Buffer / Uint8Array / typed array / ArrayBuffer) is sent // byte-for-byte instead of being shifted left 12 bytes by the StringHeader // data offset (#5757). `reqwest::Body` accepts `Vec` directly. - let body = fetch_request_body_bytes(body_ptr).unwrap_or_default(); - let content_type = - string_from_header(content_type_ptr).unwrap_or_else(|| "application/json".to_string()); + let form_data_body = body_metadata::serialize_form_data(body_ptr as usize); + let form_data_content_type = form_data_body + .as_ref() + .map(|(_, content_type)| content_type.clone()); + let body = form_data_body + .map(|(body, _)| body) + .or_else(|| fetch_request_body_bytes(body_ptr)) + .unwrap_or_default(); + let content_type = string_from_header(content_type_ptr) + .or(form_data_content_type) + .unwrap_or_else(|| "application/json".to_string()); spawn(async move { let client = fetch_client(); @@ -763,13 +771,20 @@ pub unsafe extern "C" fn js_fetch_with_options( // `Request` object and call `fetch(request, init)`; its handle id lands in // the `url_ptr` slot. Recover url/method/body/headers from the Request // registry so the request is dispatched (`init` members override). - let inputs = match request_handle::resolve_fetch_inputs( + let form_data_body = body_metadata::serialize_form_data(body_ptr as usize); + let form_data_content_type = form_data_body + .as_ref() + .map(|(_, content_type)| content_type.clone()); + let body_bytes = form_data_body + .map(|(body, _)| body) + .or_else(|| fetch_request_body_bytes(body_ptr)); + let mut inputs = match request_handle::resolve_fetch_inputs( string_from_header(url_ptr), string_from_header(method_ptr), // Read the body as raw bytes (binary bodies probe the buffer/typed-array // registry first) so a Buffer/Uint8Array body isn't corrupted by a lossy // StringHeader read (#5757). - fetch_request_body_bytes(body_ptr), + body_bytes, string_from_header(headers_json_ptr), url_ptr as usize, ) { @@ -779,6 +794,12 @@ pub unsafe extern "C" fn js_fetch_with_options( return promise; } }; + if let Some(content_type) = form_data_content_type { + inputs + .custom_headers + .entry("content-type".to_string()) + .or_insert(content_type); + } // Dispatch + abort handling live in `abort_bridge::run_request` (keeps this // file under the line-size lint gate). diff --git a/crates/perry-stdlib/src/fetch/request_ctor.rs b/crates/perry-stdlib/src/fetch/request_ctor.rs index cb3d526c55..a2354caad9 100644 --- a/crates/perry-stdlib/src/fetch/request_ctor.rs +++ b/crates/perry-stdlib/src/fetch/request_ctor.rs @@ -62,7 +62,11 @@ pub unsafe extern "C" fn js_request_new( // in `js_response_body_init_ptr` (the Response twin), which falls through via // `or_else` rather than if/else. let pending_stream_id = take_pending_fetch_body_stream_id(); - let non_stream_body: Option> = + let form_data_body = body_metadata::serialize_form_data(body_ptr as usize); + let form_data_content_type = form_data_body + .as_ref() + .map(|(_, content_type)| content_type.clone()); + let non_stream_body: Option> = form_data_body.map(|(body, _)| body).or_else(|| { if perry_runtime::value::addr_class::is_handle_band(body_ptr as usize) { crate::fetch::blob_bytes_clone(body_ptr as usize) .or_else(|| dispatch::incoming_message_raw_body_bytes(body_ptr as usize)) @@ -72,7 +76,8 @@ pub unsafe extern "C" fn js_request_new( // misread the handle id as a string pointer. .or_else(|| dispatch::incoming_message_raw_body_bytes(body_ptr as usize)) .or_else(|| dispatch::body_bytes_from_header(body_ptr)) - }; + } + }); // GET/HEAD requests may not carry a body (WHATWG fetch). Refs #2643. if (pending_stream_id.is_some() || non_stream_body.is_some()) && (method == "GET" || method == "HEAD") @@ -83,7 +88,7 @@ pub unsafe extern "C" fn js_request_new( .map(crate::streams::drain_readable_into_bytes) .or(non_stream_body); let headers_id_in = handle_id(headers_handle); - let headers = if headers_id_in != 0 { + let mut headers = if headers_id_in != 0 { HEADERS_REGISTRY .lock() .unwrap() @@ -93,6 +98,11 @@ pub unsafe extern "C" fn js_request_new( } else { HeadersStore::default() }; + if let Some(content_type) = form_data_content_type { + if !headers.has("content-type") { + headers.set("content-type", &content_type); + } + } // `signal` is a heap value the registry keeps (and the GC scanner in // `super::gc` roots), and defaulting it ALLOCATES an `AbortController` — // so resolve it, and build the whole record, before taking the registry diff --git a/crates/perry/src/commands/compile/collect_modules.rs b/crates/perry/src/commands/compile/collect_modules.rs index ddaaece7a3..4b29ebdca6 100644 --- a/crates/perry/src/commands/compile/collect_modules.rs +++ b/crates/perry/src/commands/compile/collect_modules.rs @@ -661,8 +661,14 @@ fn collect_module_one( collected: Some(ctx.native_modules.len() + ctx.js_modules.len()), ..Default::default() }); + // Expand only in the selected mode. Ordinary accessor/closure lowering then + // owns captures, source-order semantics and the generated renderer imports. + let solid_module = ctx + .solid_jsx + .then(|| perry_hir::solid_jsx::lower_solid_jsx(ast_module, "perry-solid")) + .flatten(); let lower_result = perry_hir::lower_module_full_with_platform_globals( - ast_module, + solid_module.as_ref().unwrap_or(ast_module), &module_name, &source_file_path, *next_class_id, diff --git a/crates/perry/src/commands/compile/host_config.rs b/crates/perry/src/commands/compile/host_config.rs index 01da27af50..6cd0cb613b 100644 --- a/crates/perry/src/commands/compile/host_config.rs +++ b/crates/perry/src/commands/compile/host_config.rs @@ -89,6 +89,14 @@ fn parse_boolean_switch(value: &str) -> Option { } } +fn solid_jsx_mode(value: Option<&str>) -> Result { + match value { + Some("solid") => Ok(true), + Some("default") => Ok(false), + _ => anyhow::bail!("perry.jsx must be \"solid\" or \"default\""), + } +} + fn should_auto_grant_compile_allow( has_universal_route: bool, allow_was_explicit: bool, @@ -166,6 +174,9 @@ pub(super) fn apply_pkg_and_toml_config( if let Some(pkg_json_path) = pkg_json_path.clone() { if let Ok(content) = fs::read_to_string(&pkg_json_path) { if let Ok(pkg) = serde_json::from_str::(&content) { + if let Some(mode) = pkg.get("perry").and_then(|perry| perry.get("jsx")) { + ctx.solid_jsx = solid_jsx_mode(mode.as_str())?; + } if let Some(aliases) = pkg .get("perry") .and_then(|p| p.get("packageAliases")) @@ -778,6 +789,9 @@ pub(super) fn apply_pkg_and_toml_config( .and_then(|s| s.parse::().ok()) { if let Some(perry_tbl) = table.get("perry").and_then(|v| v.as_table()) { + if let Some(mode) = perry_tbl.get("jsx") { + ctx.solid_jsx = solid_jsx_mode(mode.as_str())?; + } if let Some(strict) = perry_tbl.get("strict").and_then(|v| v.as_bool()) { ctx.strict_eval = strict; // #5230: broad `perry.strict` covers dynamic imports too. diff --git a/crates/perry/src/commands/compile/types.rs b/crates/perry/src/commands/compile/types.rs index daaffb002a..5d7f27ea1a 100644 --- a/crates/perry/src/commands/compile/types.rs +++ b/crates/perry/src/commands/compile/types.rs @@ -690,6 +690,8 @@ pub struct CompilationContext { pub native_addon_paths: BTreeMap, /// Package aliases: maps npm package name → replacement package name (from perry.packageAliases) pub package_aliases: HashMap, + /// Opt-in Solid universal JSX expansion; ordinary JSX remains the default. + pub solid_jsx: bool, /// Packages to compile natively instead of routing to V8 (from perry.compilePackages) pub compile_packages: HashSet, /// Node native-addon packages omitted from wildcard/automatic whole-package @@ -1219,6 +1221,7 @@ impl CompilationContext { native_addons: BTreeMap::new(), native_addon_paths: BTreeMap::new(), package_aliases: HashMap::new(), + solid_jsx: false, compile_packages: HashSet::new(), auto_skipped_node_addon_packages: HashSet::new(), aot_discovered_modules: HashSet::new(), diff --git a/crates/perry/src/main.rs b/crates/perry/src/main.rs index f900d5ee4b..db8da35bee 100644 --- a/crates/perry/src/main.rs +++ b/crates/perry/src/main.rs @@ -212,7 +212,11 @@ fn is_legacy_invocation(args: &[String]) -> bool { continue; } // Check if it looks like a TypeScript file (and not a subcommand) - if arg.ends_with(".ts") || arg.ends_with(".mts") || arg.ends_with(".cts") { + if arg.ends_with(".ts") + || arg.ends_with(".tsx") + || arg.ends_with(".mts") + || arg.ends_with(".cts") + { return true; } // If it's a known subcommand, not legacy diff --git a/crates/perry/tests/solid_jsx_config.rs b/crates/perry/tests/solid_jsx_config.rs new file mode 100644 index 0000000000..1f8a37cb24 --- /dev/null +++ b/crates/perry/tests/solid_jsx_config.rs @@ -0,0 +1,179 @@ +//! Mode selection and cache isolation complement the executable Solid fixture. + +use std::path::Path; +use std::process::{Command, Output}; + +fn fixture() -> tempfile::TempDir { + let directory = tempfile::tempdir().expect("temporary project"); + std::fs::write( + directory.path().join("main.tsx"), + "console.log(Hello);", + ) + .expect("JSX entry"); + std::fs::write( + directory.path().join("host.ts"), + "export function createElement(name: string) { return { name }; }\n\ + export function spread(node: any, props: any) { node.props = props; }\n", + ) + .expect("universal host"); + directory +} + +fn package(directory: &Path, mode: serde_json::Value) { + std::fs::write( + directory.join("package.json"), + serde_json::json!({ + "type": "module", + "perry": { "jsx": mode, "packageAliases": { "perry-solid": "./host.ts" } } + }) + .to_string(), + ) + .expect("project configuration"); +} + +fn compile(directory: &Path, name: &str) -> Output { + Command::new(env!("CARGO_BIN_EXE_perry")) + .current_dir(directory) + .args([ + "compile", + "main.tsx", + "--no-link", + "-o", + &format!("{name}/output.o"), + ]) + .env("PERRY_NO_AUTO_OPTIMIZE", "1") + .env_remove("PERRY_NO_CACHE") + .env_remove("PERRY_DISABLE_BUILD_CACHE") + .output() + .expect("run Perry") +} + +fn objects(path: &Path, output: &mut Vec>) { + if path.is_file() { + output.push(std::fs::read(path).expect("object bytes")); + } else { + for entry in std::fs::read_dir(path).expect("object directory") { + let path = entry.expect("object entry").path(); + if path.extension().is_some_and(|extension| extension == "o") { + output.push(std::fs::read(path).expect("object bytes")); + } + } + } +} + +fn assert_mode(directory: &Path, name: &str, solid: bool) { + let result = compile(directory, name); + assert!( + result.status.success(), + "compile failed: {}\n{}", + String::from_utf8_lossy(&result.stdout), + String::from_utf8_lossy(&result.stderr) + ); + let mut bytes = Vec::new(); + objects(&directory.join(name), &mut bytes); + assert!(!bytes.is_empty(), "the compiler must produce object files"); + let ordinary_jsx = bytes.iter().any(|object| { + object + .windows(b"js_jsx".len()) + .any(|window| window == b"js_jsx") + }); + assert_eq!( + ordinary_jsx, !solid, + "the selected mode must reach the correct runtime" + ); + let stdout = String::from_utf8_lossy(&result.stdout); + assert!( + stdout.contains(if solid { + "2 native, 0 JavaScript" + } else { + "1 native, 0 JavaScript" + }), + "only Solid mode imports the universal host: {stdout}" + ); +} + +#[test] +fn mode_switches_preserve_default_jsx_and_do_not_reuse_the_other_object() { + let directory = fixture(); + package(directory.path(), "default".into()); + assert_mode(directory.path(), "default-first.o", false); + package(directory.path(), "solid".into()); + assert_mode(directory.path(), "solid-objects", true); + package(directory.path(), "default".into()); + assert_mode(directory.path(), "default-again.o", false); + assert_eq!( + std::fs::read(directory.path().join("default-first.o/output.o")).unwrap(), + std::fs::read(directory.path().join("default-again.o/output.o")).unwrap(), + "changing back to default must recover its original object" + ); +} + +#[test] +fn toml_mode_overrides_package_mode() { + let directory = fixture(); + package(directory.path(), "solid".into()); + std::fs::write( + directory.path().join("perry.toml"), + "[perry]\njsx = 'default'\n", + ) + .unwrap(); + assert_mode(directory.path(), "toml-default.o", false); + package(directory.path(), "default".into()); + std::fs::write( + directory.path().join("perry.toml"), + "[perry]\njsx = 'solid'\n", + ) + .unwrap(); + assert_mode(directory.path(), "toml-solid", true); +} + +#[test] +fn invalid_mode_is_diagnosed_before_codegen() { + let directory = fixture(); + for mode in [ + serde_json::json!("soldi"), + serde_json::json!(true), + serde_json::Value::Null, + ] { + package(directory.path(), mode); + let result = compile(directory.path(), "invalid.o"); + assert!(!result.status.success()); + assert!(String::from_utf8_lossy(&result.stderr).contains("perry.jsx must be")); + } +} + +#[test] +fn tsx_shorthand_compiles_without_an_explicit_subcommand() { + let directory = fixture(); + package(directory.path(), "solid".into()); + let output = Command::new(env!("CARGO_BIN_EXE_perry")) + .current_dir(directory.path()) + .args(["main.tsx", "--no-link", "-o", "shorthand"]) + .env("PERRY_NO_AUTO_OPTIMIZE", "1") + .output() + .expect("Perry JSX shorthand"); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert!(String::from_utf8_lossy(&output.stdout).contains("2 native, 0 JavaScript")); +} + +#[test] +fn a_fragment_of_literals_does_not_import_an_unused_renderer() { + let directory = fixture(); + package(directory.path(), "solid".into()); + std::fs::write( + directory.path().join("main.tsx"), + "console.log(<>{42}{true});", + ) + .unwrap(); + let result = compile(directory.path(), "fragment"); + assert!( + result.status.success(), + "{}", + String::from_utf8_lossy(&result.stderr) + ); + assert!(String::from_utf8_lossy(&result.stdout).contains("1 native, 0 JavaScript")); +} diff --git a/packages/perry-solid/README.md b/packages/perry-solid/README.md index fae27497f6..7221198481 100644 --- a/packages/perry-solid/README.md +++ b/packages/perry-solid/README.md @@ -6,7 +6,7 @@ and sibling information in TypeScript so keyed lists can move native widgets without recreating them. This is the runtime bridge from [#4644](https://github.com/PerryTS/perry/issues/4644). -It provides native hyperscript; Solid JSX compilation remains a separate stage. +It provides native hyperscript and an opt-in Solid JSX compiler mode. Solid's bundled `solid-js/h` and `solid-js/html` use its web renderer and are not substitutes for this package's `h`. @@ -66,6 +66,44 @@ perry examples/counter.ts -o counter ## Components and properties +### JSX + +Set `"jsx": "solid"` inside your application's `perry` configuration alongside +the Solid client aliases above. The equivalent TOML setting is `[perry]` with +`jsx = "solid"`. An omitted setting, or `"default"`, keeps Perry's existing JSX +behavior. Perry performs the transform in the compiler; Babel is only used as +an independent test oracle for this package. + +```tsx +import { createSignal } from "solid-js"; +import { For } from "perry-solid"; + +function Counter() { + const [count, setCount] = createSignal(0); + return + Count: {count()} + + ; +} +``` + +Native intrinsic tags are `vstack`, `hstack`, `text`, `button`, `spacer`, and +`divider`. Capitalized and member names refer to your components. Use +`{item => {item.name}}` for keyed lists. +Signal reads in properties and children stay reactive; a signal write updates +the affected widgets without rerunning the component. References support a +callback or an assignable variable/member, and reference callbacks run untracked. +Spreads keep property precedence, and fragments group native children. + +For TypeScript checking, use `"jsx": "preserve"` and +`"jsxImportSource": "perry-solid"` in `tsconfig.json`. Perry consumes the `.tsx` +source directly. [examples/counter.tsx](examples/counter.tsx) is a complete app; +copy it into the application project where you installed `perry-solid`, then run +`perry counter.tsx -o counter` there. Generated JSX imports resolve the installed +`perry-solid` package just like handwritten imports. + +### Hyperscript + Use `h(Component, props)` for functions returning native children. Reactive children are accessors (`() => count()`); reactive properties are getters: @@ -110,7 +148,7 @@ idempotent, runs Solid cleanup, releases stored user callbacks, and detaches the mounted nodes. Native widget allocation and reclamation otherwise follow Perry's widget registry. -The low-level universal helpers (`createElement`, `createTextNode`, `insert`, +The low-level universal helpers (`createElement`, `createTextNode`, `insertNode`, `insert`, `spread`, `setProp`, `createComponent`, `effect`, `memo`, `mergeProps`, and `use`) are also exported. `perry-solid/renderer` exposes `createNativeRenderer` and its `NativeDriver` interface for testing host behavior without a display server. @@ -120,6 +158,7 @@ are also exported. `perry-solid/renderer` exposes `createNativeRenderer` and its ```sh npm ci --ignore-scripts npm test +npm run test:jsx:oracle npm run typecheck PERRY_BIN=/absolute/path/to/perry ../../tests/release/packages/_harness.sh --filter perry-solid ``` @@ -128,7 +167,10 @@ The release fixture copies the actual package sources and pinned dependencies, then checks the same assertions in Node's browser condition and Perry. It covers reactive properties/text, callback replacement, keyed identity/order, reparenting, invalid tree operations, and disposal; it also requires zero -JavaScript modules in the native build. +JavaScript modules in the native build. The JSX fixture runs the same assertions +through Solid's pinned official Babel universal transform in Node and through +Perry's own JSX compiler. It also checks conditional identity, refs, fragments, +component execution counts, and generated-helper name collisions. `test/native-smoke.ts` is a real widget app for Geisterhand checks. The macOS backend's `native_widget_order` Cargo target runs on the main thread and checks @@ -147,6 +189,10 @@ The runner checks updates to the same native Text handles, button callbacks, keyed row order with retained widget identities, and stopped effects after disposal. It saves screenshots and widget snapshots, then exits the app cleanly. GC scheduling and verifier environment variables are inherited by the app. +To exercise JSX against the same assertions, copy `test/native-smoke.tsx` into +your application project with `perry-solid` installed and `perry.jsx` set to +`"solid"`. Compile that copy with the same Geisterhand flag, then run the Python +runner against the resulting binary. The client runtime's separate GC verifier correction is in [#9822](https://github.com/PerryTS/perry/pull/9822). Use that correction for diff --git a/packages/perry-solid/examples/counter.tsx b/packages/perry-solid/examples/counter.tsx new file mode 100644 index 0000000000..69ead49f62 --- /dev/null +++ b/packages/perry-solid/examples/counter.tsx @@ -0,0 +1,21 @@ +import { App, VStack } from "perry/ui"; +import { createSignal } from "solid-js"; +import { For, render } from "perry-solid"; + +function Counter() { + const [count, setCount] = createSignal(0); + const [items, setItems] = createSignal(["Alpha", "Beta", "Gamma"]); + return + Count: {count()} + + + + + + {item => {item}} + ; +} + +const body = VStack([]); +render(() => , body); +App({ title: "Solid JSX + Perry", width: 420, height: 300, body }); diff --git a/packages/perry-solid/package-lock.json b/packages/perry-solid/package-lock.json index 7b6a3f49d8..ee53ab72fb 100644 --- a/packages/perry-solid/package-lock.json +++ b/packages/perry-solid/package-lock.json @@ -9,7 +9,9 @@ "version": "0.1.0", "license": "MIT", "devDependencies": { + "@babel/core": "7.29.7", "@types/node": "26.4.1", + "babel-preset-solid": "1.9.15", "solid-js": "1.9.15", "typescript": "5.9.3" }, @@ -17,6 +19,322 @@ "solid-js": "^1.9.15" } }, + "node_modules/@babel/code-frame": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/code-frame/-/code-frame-7.29.7.tgz", + "integrity": "sha512-Aup7aUOfpbAUg2ROOJN6Iw5f9DMBlzu0mIkm/malLQFN/YQgO48wCj0Kxa3sEHJvPVFg7siR+qRInwXd2qhQKw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/helper-validator-identifier": "^7.29.7", + "js-tokens": "^4.0.0", + "picocolors": "^1.1.1" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/compat-data": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/compat-data/-/compat-data-7.29.7.tgz", + "integrity": "sha512-locTkQyKvwIEgBzVrn8693ebc97F2U8ZHjbXwDXJ5Fn2TCpNwTlKcaKLkdHop5c/icOFE7qt7Q9JC5hnKNa6Gg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/core": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/core/-/core-7.29.7.tgz", + "integrity": "sha512-RgHBCvtjbOK2gXSNBNIkNoEc9qoVEtau3hj8gEqKQuL3HZAibKarWFEI3Lfm6EYKkLalOh8eSrj9b+ch9H/VBA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/code-frame": "^7.29.7", + "@babel/generator": "^7.29.7", + "@babel/helper-compilation-targets": "^7.29.7", + "@babel/helper-module-transforms": "^7.29.7", + "@babel/helpers": "^7.29.7", + "@babel/parser": "^7.29.7", + "@babel/template": "^7.29.7", + "@babel/traverse": "^7.29.7", + "@babel/types": "^7.29.7", + "@jridgewell/remapping": "^2.3.5", + "convert-source-map": "^2.0.0", + "debug": "^4.1.0", + "gensync": "^1.0.0-beta.2", + "json5": "^2.2.3", + "semver": "^6.3.1" + }, + "engines": { + "node": ">=6.9.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/babel" + } + }, + "node_modules/@babel/generator": { + "version": "7.29.8", + "resolved": "https://registry.npmjs.org/@babel/generator/-/generator-7.29.8.tgz", + "integrity": "sha512-gZbepsdh3WDtgZKWL+vTPh71LSBrm/Y4/QDZBVCcYfmeTEEuoOYwlSy+G1StfJg+/Zy550u/3TATbm7qDbbMtg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/parser": "^7.29.8", + "@babel/types": "^7.29.8", + "@jridgewell/gen-mapping": "^0.3.12", + "@jridgewell/trace-mapping": "^0.3.28", + "jsesc": "^3.0.2" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-compilation-targets": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/helper-compilation-targets/-/helper-compilation-targets-7.29.7.tgz", + "integrity": "sha512-wem6WaBj4NaVYVdNhLPPVacES6ZJ+KBBfSkTMD3YZxbP3rm3Di85tJU5ljaUNhaOynt+Aj0xruhYuzQBt8n71g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/compat-data": "^7.29.7", + "@babel/helper-validator-option": "^7.29.7", + "browserslist": "^4.24.0", + "lru-cache": "^5.1.1", + "semver": "^6.3.1" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-globals": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/helper-globals/-/helper-globals-7.29.7.tgz", + "integrity": "sha512-3nQVUAtvkKH9zahfWgw96Jc/uFOmjACE1kQz82E2lqWmHBgjzbNlsC22nuQTfahmWeQtTq5nQ/4Nnd2A1wj4zA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-module-imports": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/helper-module-imports/-/helper-module-imports-7.29.7.tgz", + "integrity": "sha512-ejHwrQQYcm9xnTivShn2IDOlIzInN34AXskvq9QicvCtEzq1Vzclu/tKF8Jq1Cg8JG2GL6/EmjgsCT7lXepE3g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/traverse": "^7.29.7", + "@babel/types": "^7.29.7" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-module-transforms": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/helper-module-transforms/-/helper-module-transforms-7.29.7.tgz", + "integrity": "sha512-UPUVSyXbOh627KiCIGQSgwWzGeBKLkaJ9PJEdrngIwMSzxLR4jS4+f1f1jb7VzBbg8nFLaYotvVPFCTqdrmTAg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/helper-module-imports": "^7.29.7", + "@babel/helper-validator-identifier": "^7.29.7", + "@babel/traverse": "^7.29.7" + }, + "engines": { + "node": ">=6.9.0" + }, + "peerDependencies": { + "@babel/core": "^7.0.0" + } + }, + "node_modules/@babel/helper-plugin-utils": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/helper-plugin-utils/-/helper-plugin-utils-7.29.7.tgz", + "integrity": "sha512-G7sHYigPY17oO5SYWnfD/0MTBwVR781S/JI643e/JhUYgVgWE/61SoW3NH9KWUKyKq5LVh3npif99Wkt6j86Jw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-string-parser": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/helper-string-parser/-/helper-string-parser-7.29.7.tgz", + "integrity": "sha512-Pb5ijPrZ89GDH8223L4UP8i6QApWxs04RbPQJTeWDV0/keR2E36MeKnyr6LYmUUvqRRI+Iv87SuF1W6ErINzYw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-validator-identifier": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/helper-validator-identifier/-/helper-validator-identifier-7.29.7.tgz", + "integrity": "sha512-qehxGkRj55h/ff8EMaJ+cYhyaKlHIxqYDn682wQD7RNp9UujOQsHog2uS0r2vzr4pW+sXf90NeeayjcNaX3fFg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-validator-option": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/helper-validator-option/-/helper-validator-option-7.29.7.tgz", + "integrity": "sha512-N9ZErrD+yW5geCDtBqnOoxmR8+tNKiGuxKlDpuJxfsqpa2dFcexaziGAE/qoHLiDDreVNMupxGmSoNlyvsA3gw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helpers": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/helpers/-/helpers-7.29.7.tgz", + "integrity": "sha512-1k2lAGRMfHTcwuNYcCNUmaUffmQv8KWMfh2iJUUeRlwlwH4FdNG7mfPI10NPfLHJFThE4Tyr4mv7kTNZOiPuBg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/template": "^7.29.7", + "@babel/types": "^7.29.7" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/parser": { + "version": "7.29.8", + "resolved": "https://registry.npmjs.org/@babel/parser/-/parser-7.29.8.tgz", + "integrity": "sha512-E8lTAYNB1KW+FH+VGJuZM1ioAx2E6oVlvQFRrf5P8ZZmsiJXYAD9vTFV7yyEURNzgh1dFqMZuO6tUwcARbqFCA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/types": "^7.29.8" + }, + "bin": { + "parser": "bin/babel-parser.js" + }, + "engines": { + "node": ">=6.0.0" + } + }, + "node_modules/@babel/plugin-syntax-jsx": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/plugin-syntax-jsx/-/plugin-syntax-jsx-7.29.7.tgz", + "integrity": "sha512-TSu8+mHCoEaaCDEZ0I3+6mvTBYR4PCxQwf2z9/r5Tbztv6NaLR3B9thGTTxX2WGuGHJqRiAbKPeGTJ5XWXVg6A==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/helper-plugin-utils": "^7.29.7" + }, + "engines": { + "node": ">=6.9.0" + }, + "peerDependencies": { + "@babel/core": "^7.0.0-0" + } + }, + "node_modules/@babel/template": { + "version": "7.29.7", + "resolved": "https://registry.npmjs.org/@babel/template/-/template-7.29.7.tgz", + "integrity": "sha512-puq+Gf35oI24FeN11LkoUQFqv9uwNeWpxXZi/Ji3rRIoKAzKnxRaZ+Gkj0vKS9ZCiTESfng1N9LyOyXvo+m+Gg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/code-frame": "^7.29.7", + "@babel/parser": "^7.29.7", + "@babel/types": "^7.29.7" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/traverse": { + "version": "7.29.8", + "resolved": "https://registry.npmjs.org/@babel/traverse/-/traverse-7.29.8.tgz", + "integrity": "sha512-I5z7H3bf/41ktsNVLtpN0wAa336HkqIHQ5BuPLEhTkt1jVSyZpeNKIzTgEWmlxjdg81R0IgUCcaE+Ok3NvrfZg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/code-frame": "^7.29.7", + "@babel/generator": "^7.29.8", + "@babel/helper-globals": "^7.29.7", + "@babel/parser": "^7.29.8", + "@babel/template": "^7.29.7", + "@babel/types": "^7.29.8", + "debug": "^4.3.1" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/types": { + "version": "7.29.8", + "resolved": "https://registry.npmjs.org/@babel/types/-/types-7.29.8.tgz", + "integrity": "sha512-Vj1jF3cPfxg7OAfoI7QnVKLoILlm2JF9pnVHrX8qx7AHMiYWT+NDAA7jChlNgRS4WTLc/fD1lXLmPixluj+3Gg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/helper-string-parser": "^7.29.7", + "@babel/helper-validator-identifier": "^7.29.7" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@jridgewell/gen-mapping": { + "version": "0.3.13", + "resolved": "https://registry.npmjs.org/@jridgewell/gen-mapping/-/gen-mapping-0.3.13.tgz", + "integrity": "sha512-2kkt/7niJ6MgEPxF0bYdQ6etZaA+fQvDcLKckhy1yIQOzaoKjBBjSj63/aLVjYE3qhRt5dvM+uUyfCg6UKCBbA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jridgewell/sourcemap-codec": "^1.5.0", + "@jridgewell/trace-mapping": "^0.3.24" + } + }, + "node_modules/@jridgewell/remapping": { + "version": "2.3.5", + "resolved": "https://registry.npmjs.org/@jridgewell/remapping/-/remapping-2.3.5.tgz", + "integrity": "sha512-LI9u/+laYG4Ds1TDKSJW2YPrIlcVYOwi2fUC6xB43lueCjgxV4lffOCZCtYFiH6TNOX+tQKXx97T4IKHbhyHEQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jridgewell/gen-mapping": "^0.3.5", + "@jridgewell/trace-mapping": "^0.3.24" + } + }, + "node_modules/@jridgewell/resolve-uri": { + "version": "3.1.2", + "resolved": "https://registry.npmjs.org/@jridgewell/resolve-uri/-/resolve-uri-3.1.2.tgz", + "integrity": "sha512-bRISgCIjP20/tbWSPWMEi54QVPRZExkuD9lJL+UIxUKtwVJA8wW1Trb1jMs1RFXo1CBTNZ/5hpC9QvmKWdopKw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.0.0" + } + }, + "node_modules/@jridgewell/sourcemap-codec": { + "version": "1.6.0", + "resolved": "https://registry.npmjs.org/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.6.0.tgz", + "integrity": "sha512-T7jf+5zgsZHwNJ4lvQ7/aezbyk0nNX+zJVWpmHA7VYsEx7a7qr5Rg5IbtJFqkgze5Y2sruq1RUY8Q837Od7iFw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@jridgewell/trace-mapping": { + "version": "0.3.31", + "resolved": "https://registry.npmjs.org/@jridgewell/trace-mapping/-/trace-mapping-0.3.31.tgz", + "integrity": "sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jridgewell/resolve-uri": "^3.1.0", + "@jridgewell/sourcemap-codec": "^1.4.14" + } + }, "node_modules/@types/node": { "version": "26.4.1", "resolved": "https://registry.npmjs.org/@types/node/-/node-26.4.1.tgz", @@ -27,6 +345,130 @@ "undici-types": "~8.3.0" } }, + "node_modules/babel-plugin-jsx-dom-expressions": { + "version": "0.40.10", + "resolved": "https://registry.npmjs.org/babel-plugin-jsx-dom-expressions/-/babel-plugin-jsx-dom-expressions-0.40.10.tgz", + "integrity": "sha512-lxve6Y02YiZTldB7efKpnbf1BH00XCFZNYYW235jSGsYaJNFtHrYlKV6/O+miHbjqpIr9FTe5+0no4hofAMbfA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/helper-module-imports": "7.18.6", + "@babel/plugin-syntax-jsx": "^7.18.6", + "@babel/types": "^7.20.7", + "html-entities": "2.3.3", + "parse5": "^7.1.2" + }, + "peerDependencies": { + "@babel/core": "^7.20.12" + } + }, + "node_modules/babel-plugin-jsx-dom-expressions/node_modules/@babel/helper-module-imports": { + "version": "7.18.6", + "resolved": "https://registry.npmjs.org/@babel/helper-module-imports/-/helper-module-imports-7.18.6.tgz", + "integrity": "sha512-0NFvs3VkuSYbFi1x2Vd6tKrywq+z/cLeYC/RJNFrIX/30Bf5aiGYbtvGXolEktzJH8o5E5KJ3tT+nkxuuZFVlA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/types": "^7.18.6" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/babel-preset-solid": { + "version": "1.9.15", + "resolved": "https://registry.npmjs.org/babel-preset-solid/-/babel-preset-solid-1.9.15.tgz", + "integrity": "sha512-GBmg1OiPb+OwcH51XbDAKPtvrPfQW7rCJTJxcp8+yhtWwN+kqnbEJk2SgVybd+uhTxTKAvjaFyiQSr/eUZBwzg==", + "dev": true, + "license": "MIT", + "dependencies": { + "babel-plugin-jsx-dom-expressions": "^0.40.10" + }, + "peerDependencies": { + "@babel/core": "^7.0.0", + "solid-js": "^1.9.15" + }, + "peerDependenciesMeta": { + "solid-js": { + "optional": true + } + } + }, + "node_modules/baseline-browser-mapping": { + "version": "2.11.21", + "resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.11.21.tgz", + "integrity": "sha512-uh8vpY/1/YyFkunIDFH/12p7/7VdPKA1hejMVEbdkEaWnUz0Hesvx5EbiU6XxjyHZIOju+ZMbQJkRh+es3/spQ==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "baseline-browser-mapping": "dist/cli.cjs" + }, + "engines": { + "node": ">=6.0.0" + } + }, + "node_modules/browserslist": { + "version": "4.28.9", + "resolved": "https://registry.npmjs.org/browserslist/-/browserslist-4.28.9.tgz", + "integrity": "sha512-EWazOblFYUvlGZcfGhPUPmYh3nikUxBVb+y9MJun5f3hBi812X+8MSQTujLBtgK3cf51fJWbWfOjyeO954d+Eg==", + "dev": true, + "funding": [ + { + "type": "opencollective", + "url": "https://opencollective.com/browserslist" + }, + { + "type": "tidelift", + "url": "https://tidelift.com/funding/github/npm/browserslist" + }, + { + "type": "github", + "url": "https://github.com/sponsors/ai" + } + ], + "license": "MIT", + "dependencies": { + "baseline-browser-mapping": "^2.11.20", + "caniuse-lite": "^1.0.30001810", + "electron-to-chromium": "^1.5.420", + "node-releases": "^2.0.54", + "update-browserslist-db": "^1.3.2" + }, + "bin": { + "browserslist": "cli.js" + }, + "engines": { + "node": "^6 || ^7 || ^8 || ^9 || ^10 || ^11 || ^12 || >=13.7" + } + }, + "node_modules/caniuse-lite": { + "version": "1.0.30001810", + "resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001810.tgz", + "integrity": "sha512-TITQPUkaz+aVk5GL6NhOdwk1aEaNTSDPsGFWrTuhKGtjTF70jL/Oht2W4c6rXUe5fu7Ie19VIahAXHIIiWWNeg==", + "dev": true, + "funding": [ + { + "type": "opencollective", + "url": "https://opencollective.com/browserslist" + }, + { + "type": "tidelift", + "url": "https://tidelift.com/funding/github/npm/caniuse-lite" + }, + { + "type": "github", + "url": "https://github.com/sponsors/ai" + } + ], + "license": "CC-BY-4.0" + }, + "node_modules/convert-source-map": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/convert-source-map/-/convert-source-map-2.0.0.tgz", + "integrity": "sha512-Kvp459HrV2FEJ1CAsi1Ku+MY3kasH19TFykTz2xWmMeq6bk2NU3XXvfJ+Q61m0xktWwt+1HSYf3JZsTms3aRJg==", + "dev": true, + "license": "MIT" + }, "node_modules/csstype": { "version": "3.2.3", "resolved": "https://registry.npmjs.org/csstype/-/csstype-3.2.3.tgz", @@ -34,6 +476,161 @@ "dev": true, "license": "MIT" }, + "node_modules/debug": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", + "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==", + "dev": true, + "license": "MIT", + "dependencies": { + "ms": "^2.1.3" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/electron-to-chromium": { + "version": "1.5.422", + "resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.5.422.tgz", + "integrity": "sha512-UvA/32XqrLDdZSn7Jllo1AYNcWji/G0d5M0GTViE7KoGBiMunw3a34Sb2KO4ZZyrSEhqsxFoVhWWJshdyfKqJA==", + "dev": true, + "license": "ISC" + }, + "node_modules/entities": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/entities/-/entities-6.0.1.tgz", + "integrity": "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==", + "dev": true, + "license": "BSD-2-Clause", + "engines": { + "node": ">=0.12" + }, + "funding": { + "url": "https://github.com/fb55/entities?sponsor=1" + } + }, + "node_modules/escalade": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/escalade/-/escalade-3.2.0.tgz", + "integrity": "sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6" + } + }, + "node_modules/gensync": { + "version": "1.0.0-beta.2", + "resolved": "https://registry.npmjs.org/gensync/-/gensync-1.0.0-beta.2.tgz", + "integrity": "sha512-3hN7NaskYvMDLQY55gnW3NQ+mesEAepTqlg+VEbj7zzqEMBVNhzcGYYeqFo/TlYz6eQiFcp1HcsCZO+nGgS8zg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/html-entities": { + "version": "2.3.3", + "resolved": "https://registry.npmjs.org/html-entities/-/html-entities-2.3.3.tgz", + "integrity": "sha512-DV5Ln36z34NNTDgnz0EWGBLZENelNAtkiFA4kyNOG2tDI6Mz1uSWiq1wAKdyjnJwyDiDO7Fa2SO1CTxPXL8VxA==", + "dev": true, + "license": "MIT" + }, + "node_modules/js-tokens": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-4.0.0.tgz", + "integrity": "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/jsesc": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/jsesc/-/jsesc-3.1.0.tgz", + "integrity": "sha512-/sM3dO2FOzXjKQhJuo0Q173wf2KOo8t4I8vHy6lF9poUp7bKT0/NHE8fPX23PwfhnykfqnC2xRxOnVw5XuGIaA==", + "dev": true, + "license": "MIT", + "bin": { + "jsesc": "bin/jsesc" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/json5": { + "version": "2.2.3", + "resolved": "https://registry.npmjs.org/json5/-/json5-2.2.3.tgz", + "integrity": "sha512-XmOWe7eyHYH14cLdVPoyg+GOH3rYX++KpzrylJwSW98t3Nk+U8XOl8FWKOgwtzdb8lXGf6zYwDUzeHMWfxasyg==", + "dev": true, + "license": "MIT", + "bin": { + "json5": "lib/cli.js" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/lru-cache": { + "version": "5.1.1", + "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-5.1.1.tgz", + "integrity": "sha512-KpNARQA3Iwv+jTA0utUVVbrh+Jlrr1Fv0e56GGzAFOXN7dk/FviaDW8LHmK52DlcH4WP2n6gI8vN1aesBFgo9w==", + "dev": true, + "license": "ISC", + "dependencies": { + "yallist": "^3.0.2" + } + }, + "node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "dev": true, + "license": "MIT" + }, + "node_modules/node-releases": { + "version": "2.0.54", + "resolved": "https://registry.npmjs.org/node-releases/-/node-releases-2.0.54.tgz", + "integrity": "sha512-YHs7BmmcsdAI5Ozuf8JZo6PT0mv2GIWC9vMfvUC3dp65M8hn7Ux8CPL+2oBI7juNuj9d0ndhTcznq2ODBps9cQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=18" + } + }, + "node_modules/parse5": { + "version": "7.3.0", + "resolved": "https://registry.npmjs.org/parse5/-/parse5-7.3.0.tgz", + "integrity": "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==", + "dev": true, + "license": "MIT", + "dependencies": { + "entities": "^6.0.0" + }, + "funding": { + "url": "https://github.com/inikulin/parse5?sponsor=1" + } + }, + "node_modules/picocolors": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz", + "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==", + "dev": true, + "license": "ISC" + }, + "node_modules/semver": { + "version": "6.3.1", + "resolved": "https://registry.npmjs.org/semver/-/semver-6.3.1.tgz", + "integrity": "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA==", + "dev": true, + "license": "ISC", + "bin": { + "semver": "bin/semver.js" + } + }, "node_modules/seroval": { "version": "1.5.6", "resolved": "https://registry.npmjs.org/seroval/-/seroval-1.5.6.tgz", @@ -89,6 +686,44 @@ "integrity": "sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ==", "dev": true, "license": "MIT" + }, + "node_modules/update-browserslist-db": { + "version": "1.3.2", + "resolved": "https://registry.npmjs.org/update-browserslist-db/-/update-browserslist-db-1.3.2.tgz", + "integrity": "sha512-UQ+MSxlhRm1bzjhU+DcuXfjFO1FzNtqhK5+9Yvlp90ItDLk5vT932A0rFu619nf7RVS+Y/VeaUW1jaRDqZ8VJw==", + "dev": true, + "funding": [ + { + "type": "opencollective", + "url": "https://opencollective.com/browserslist" + }, + { + "type": "tidelift", + "url": "https://tidelift.com/funding/github/npm/browserslist" + }, + { + "type": "github", + "url": "https://github.com/sponsors/ai" + } + ], + "license": "MIT", + "dependencies": { + "escalade": "^3.2.0", + "picocolors": "^1.1.1" + }, + "bin": { + "update-browserslist-db": "cli.js" + }, + "peerDependencies": { + "browserslist": ">= 4.21.0" + } + }, + "node_modules/yallist": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/yallist/-/yallist-3.1.1.tgz", + "integrity": "sha512-a4UGQaWPH59mOXUYnAG2ewncQS4i4F43Tv3JoAM+s2VDAmS9NsK8GpDMLrCHPksFT7h3K6TOoUNn2pb7RoXx4g==", + "dev": true, + "license": "ISC" } } } diff --git a/packages/perry-solid/package.json b/packages/perry-solid/package.json index 76f86c56b6..ccf5bd1064 100644 --- a/packages/perry-solid/package.json +++ b/packages/perry-solid/package.json @@ -7,7 +7,8 @@ "types": "./src/index.ts", "exports": { ".": "./src/index.ts", - "./renderer": "./src/renderer.ts" + "./renderer": "./src/renderer.ts", + "./jsx-runtime": "./src/jsx-runtime.ts" }, "files": [ "src", @@ -18,13 +19,16 @@ "solid-js": "^1.9.15" }, "devDependencies": { + "@babel/core": "7.29.7", "@types/node": "26.4.1", + "babel-preset-solid": "1.9.15", "solid-js": "1.9.15", "typescript": "5.9.3" }, "scripts": { "test": "node --conditions=browser test/renderer.test.ts", - "typecheck": "tsc --noEmit" + "typecheck": "tsc --noEmit", + "test:jsx:oracle": "node test/jsx/oracle.cjs && node --conditions=browser test/jsx/generated.ts" }, "perry": { "nativeModule": true, @@ -38,6 +42,7 @@ }, "packageAliases": { "solid-js": "solid-js/dist/solid.js" - } + }, + "jsx": "solid" } } diff --git a/packages/perry-solid/src/index.ts b/packages/perry-solid/src/index.ts index 9757460a30..78246c85ec 100644 --- a/packages/perry-solid/src/index.ts +++ b/packages/perry-solid/src/index.ts @@ -82,6 +82,7 @@ export const render = native.render; export const createElement = native.createElement; export const createTextNode = native.createTextNode; export const insert = native.insert; +export const insertNode = native.insertNode; export const spread = native.spread; export const setProp = native.setProp; export const createComponent = native.createComponent; diff --git a/packages/perry-solid/src/jsx-runtime.ts b/packages/perry-solid/src/jsx-runtime.ts new file mode 100644 index 0000000000..eacad348d5 --- /dev/null +++ b/packages/perry-solid/src/jsx-runtime.ts @@ -0,0 +1,32 @@ +import type { Child, NativeNode, Props } from "./renderer.ts"; + +/** Types for JSX preserved for Perry's Solid compiler mode. */ +export namespace JSX { + export type Element = Child; + export type ElementType = keyof IntrinsicElements | ((props: any) => Child); + export interface ElementChildrenAttribute { children: {}; } + export interface NativeProps extends Props { + children?: Child; + ref?: NativeNode | ((node: NativeNode) => void); + text?: string; + onPress?: () => void; + width?: number; + height?: number; + opacity?: number; + hidden?: boolean; + disabled?: boolean; + tooltip?: string; + padding?: number; + cornerRadius?: number; + fontSize?: number; + backgroundColor?: [number, number, number, number]; + } + export interface IntrinsicElements { + vstack: NativeProps; + hstack: NativeProps; + text: NativeProps; + button: NativeProps; + spacer: NativeProps; + divider: NativeProps; + } +} diff --git a/packages/perry-solid/src/renderer.ts b/packages/perry-solid/src/renderer.ts index a1e8cdac0f..1499f890d6 100644 --- a/packages/perry-solid/src/renderer.ts +++ b/packages/perry-solid/src/renderer.ts @@ -47,6 +47,15 @@ export function createNativeRenderer(driver: NativeDriver) { } function createElement(name: string): NativeNode { + // JSX intrinsic names are lowercase; the native driver uses Perry names. + switch (name) { + case "vstack": name = "VStack"; break; + case "hstack": name = "HStack"; break; + case "text": name = "Text"; break; + case "button": name = "Button"; break; + case "spacer": name = "Spacer"; break; + case "divider": name = "Divider"; break; + } if (!["VStack", "HStack", "Text", "Button", "Spacer", "Divider"].includes(name)) { throw new Error(`Unsupported Perry Solid element: ${name}`); } diff --git a/packages/perry-solid/test/jsx/.gitignore b/packages/perry-solid/test/jsx/.gitignore new file mode 100644 index 0000000000..6c1fa8aa50 --- /dev/null +++ b/packages/perry-solid/test/jsx/.gitignore @@ -0,0 +1,3 @@ +generated.ts +out +*.log diff --git a/packages/perry-solid/test/jsx/host.ts b/packages/perry-solid/test/jsx/host.ts new file mode 100644 index 0000000000..4d8313c895 --- /dev/null +++ b/packages/perry-solid/test/jsx/host.ts @@ -0,0 +1,45 @@ +import { createNativeRenderer, type NativeDriver, type NativeNode, type ElementName } from "../../src/renderer.ts"; + +export const widgets: { + kind: ElementName; + children: number[]; + props: Record; + press: () => void; +}[] = []; + +const driver: NativeDriver = { + create(kind, press) { + widgets.push({ kind, children: [], props: {}, press }); + return widgets.length; + }, + setProperty(handle, _kind, name, value) { widgets[handle - 1].props[name] = value; }, + insert(parent, child, index, previousParent) { + if (previousParent !== null) { + const old = widgets[previousParent - 1].children; + old.splice(old.indexOf(child), 1); + } + widgets[parent - 1].children.splice(index, 0, child); + }, + move(parent, from, to) { + const children = widgets[parent - 1].children; + const child = children.splice(from, 1)[0]; + children.splice(to, 0, child); + }, + remove(parent, child) { + const children = widgets[parent - 1].children; + children.splice(children.indexOf(child), 1); + }, +}; + +export const root = driver.create("VStack", () => {}); +export const { + render, h, createElement, createTextNode, createComponent, insert, insertNode, + spread, setProp, effect, memo, mergeProps, use, +} = createNativeRenderer(driver); + +export function props(node: NativeNode): Record { + return widgets[node.handle - 1].props; +} +export function children(node: NativeNode): number[] { + return widgets[node.handle - 1].children; +} diff --git a/packages/perry-solid/test/jsx/main.tsx b/packages/perry-solid/test/jsx/main.tsx new file mode 100644 index 0000000000..899006d3d6 --- /dev/null +++ b/packages/perry-solid/test/jsx/main.tsx @@ -0,0 +1,91 @@ +import assert from "node:assert/strict"; +import { createSignal, onCleanup } from "solid-js"; +import { For, type NativeNode, type Child } from "../../src/renderer.ts"; +import { render, root, widgets, props, children } from "./host.ts"; + +// Deliberately collide with the transform's first candidate helper prefix. +const __perry_solid_0_createElement = "user binding"; +const [count, setCount] = createSignal(0); +const [rows, setRows] = createSignal(["Alpha", "Beta", "Gamma"]); +const [shown, setShown] = createSignal(1); +const [spreadProps, setSpreadProps] = createSignal({ text: "Spread 0", width: 150 }); +const [handler, setHandler] = createSignal<() => void>(() => setCount(value => value + 1)); +let label!: NativeNode; +let button!: NativeNode; +let list!: NativeNode; +let raw!: NativeNode; +let conditional!: NativeNode; +let spreadOnly!: NativeNode; +let precedence!: NativeNode; +const memberRef: { current?: NativeNode } = {}; +let callbackRef!: NativeNode; +let refCalls = 0; +const capture = (node: NativeNode) => { count(); refCalls++; callbackRef = node; }; +let componentRuns = 0; +let cleanups = 0; + +function Panel(properties: { children?: Child; title: string }) { + componentRuns++; + return {properties.children}; +} +const UI = { Panel }; +const dispose = render(() => { + onCleanup(() => cleanups++); + return + Count: {count()} + + {"Raw: " + count()} + + {item => {item}} + + {shown() > 0 && Kept} + + + Member + Callback + <>Fragment + ; +}, root); + +assert.equal(__perry_solid_0_createElement, "user binding"); +assert.equal(props(label!).text, "Count: 0"); +assert.equal(props(label!).width, 100); +assert.equal(props(button!).text, "Increment"); +assert.equal(props(spreadOnly!).text, "Spread 0"); +assert.equal(props(precedence!).width, 200); +assert.equal(props(memberRef.current!).text, "Member"); +assert.equal(props(callbackRef!).text, "Callback"); +const firstRaw = children(raw!)[0]; +const firstConditional = conditional!; +const firstRows = [...children(list!)]; +widgets[button!.handle - 1].press(); +assert.equal(props(label!).text, "Count: 1"); +assert.equal(props(label!).width, 101); +assert.equal(children(raw!)[0], firstRaw); +assert.equal(widgets[firstRaw - 1].props.text, "Raw: 1"); +assert.equal(componentRuns, 1, "signal writes do not rerun the component"); +setHandler(() => () => setCount(value => value + 10)); +widgets[button!.handle - 1].press(); +assert.equal(props(label!).text, "Count: 11"); +assert.equal(refCalls, 1, "ref callbacks do not subscribe to signals they read"); +setRows(items => [items[2], items[0], items[1]]); +assert.deepEqual(children(list!), [firstRows[2], firstRows[0], firstRows[1]]); +setShown(2); +assert.equal(conditional!, firstConditional, "truthy condition updates preserve the native branch"); +setShown(0); +assert.equal(firstConditional.parent, null); +setShown(1); +assert.notEqual(conditional!, firstConditional); +setSpreadProps({ text: "Spread 1", width: 160 }); +assert.equal(props(spreadOnly!).text, "Spread 1"); +assert.equal(props(spreadOnly!).width, 160); +assert.equal(props(precedence!).width, 211, "later attributes retain precedence over a changing spread"); +const beforeDispose = props(label!).text; +dispose(); +setCount(99); +widgets[button!.handle - 1].press(); +assert.equal(count(), 99); +assert.equal(props(label!).text, beforeDispose); +assert.equal(cleanups, 1); +assert.deepEqual(widgets[root - 1].children, []); +console.log("PASS Solid JSX: native updates, components, keyed identity, conditionals, spreads, refs, fragments, disposal"); diff --git a/packages/perry-solid/test/jsx/oracle.cjs b/packages/perry-solid/test/jsx/oracle.cjs new file mode 100644 index 0000000000..a95053681f --- /dev/null +++ b/packages/perry-solid/test/jsx/oracle.cjs @@ -0,0 +1,14 @@ +const { readFileSync, writeFileSync } = require("node:fs"); +const { join } = require("node:path"); +const { transformSync } = require("@babel/core"); +const preset = require("babel-preset-solid"); + +const input = join(__dirname, "main.tsx"); +const output = transformSync(readFileSync(input, "utf8"), { + filename: input, + configFile: false, + babelrc: false, + parserOpts: { plugins: ["typescript", "jsx"] }, + presets: [[preset, { generate: "universal", moduleName: "./host.ts", builtIns: [] }]], +}); +writeFileSync(join(__dirname, "generated.ts"), output.code + "\n"); diff --git a/packages/perry-solid/test/jsx/package.json b/packages/perry-solid/test/jsx/package.json new file mode 100644 index 0000000000..a592634bf1 --- /dev/null +++ b/packages/perry-solid/test/jsx/package.json @@ -0,0 +1,13 @@ +{ + "private": true, + "type": "module", + "perry": { + "jsx": "solid", + "compilePackages": ["solid-js"], + "allow": { "compilePackages": ["solid-js"] }, + "packageAliases": { + "solid-js": "solid-js/dist/solid.js", + "perry-solid": "./host.ts" + } + } +} diff --git a/packages/perry-solid/test/native-smoke.tsx b/packages/perry-solid/test/native-smoke.tsx new file mode 100644 index 0000000000..064076fe6c --- /dev/null +++ b/packages/perry-solid/test/native-smoke.tsx @@ -0,0 +1,17 @@ +import { App, VStack, Button, widgetAddChild } from "perry/ui"; +import { createSignal } from "solid-js"; +import { render, For } from "perry-solid"; + +const [count, setCount] = createSignal(0); +const [items, setItems] = createSignal(["Alpha", "Beta", "Gamma"]); +const body = VStack([]); +const dispose = render(() => + Count: {count()} + + + {"Raw: " + count()} + {item => {item}} +, body); +widgetAddChild(body, Button("Dispose", () => { dispose(); setCount(99); })); +widgetAddChild(body, Button("Exit", () => process.exit(0))); +App({ title: "Solid native smoke", width: 420, height: 360, body }); diff --git a/packages/perry-solid/tsconfig.json b/packages/perry-solid/tsconfig.json index 9c101c5918..3be8c425b0 100644 --- a/packages/perry-solid/tsconfig.json +++ b/packages/perry-solid/tsconfig.json @@ -10,12 +10,20 @@ "paths": { "perry/ui": [ "../../types/perry/ui/index.d.ts" + ], + "perry-solid/jsx-runtime": [ + "./src/jsx-runtime.ts" ] - } + }, + "jsx": "preserve", + "jsxImportSource": "perry-solid" }, "include": [ - "src/**/*.ts", - "test/**/*.ts", - "examples/**/*.ts" + "src/**/*", + "test/**/*", + "examples/**/*" + ], + "exclude": [ + "test/jsx/generated.ts" ] } diff --git a/scripts/addr_class_allowlist.txt b/scripts/addr_class_allowlist.txt index 5f0a030f52..0b74a93015 100644 --- a/scripts/addr_class_allowlist.txt +++ b/scripts/addr_class_allowlist.txt @@ -166,9 +166,9 @@ crates/perry-runtime/src/array/collection_tag_tests.rs | * | unit tests for the crates/perry-ext-events/src/lib.rs | const EVENT_EMITTER_HANDLE_ID_END | #7272: the crate's own handle-id range end, a registry bound rather than a heap-address band test crates/perry-ext-net/src/jsvalue.rs | * | #7272: socket/server handle-vs-pointer discrimination before dereference; re-types the handle band instead of calling addr_class::is_handle_band, which perry-runtime does not export crates/perry-ext-ratelimit/src/lib.rs | (obj as usize) >= 0x100000 | #7272: same handle-vs-pointer guard before an ObjectHeader read -crates/perry-runtime/src/arena/page_meta.rs | let header = addr as *const GcHeader; | promoted-page-run expansion: `addr` starts at a `first_header` recorded by arena/promote.rs's linear block iteration (its grandfathered sibling entry above) and advances by `GcHeader::size` from there, so every address is a block-interior header, never a NaN-box payload; the parse stops at the first implausible size exactly as the arena walkers do +crates/perry-runtime/src/arena/page_meta/mod.rs | let header = addr as *const GcHeader; | promoted-page-run expansion: `addr` starts at a `first_header` recorded by arena/promote.rs's linear block iteration (its grandfathered sibling entry above) and advances by `GcHeader::size` from there, so every address is a block-interior header, never a NaN-box payload; the parse stops at the first implausible size exactly as the arena walkers do crates/perry-runtime/src/arena/tests_promoted_runs.rs | * | arena promoted-run tests: header addresses are offsets into a buffer the test itself allocated and initialised, never NaN-box payloads -- same discipline as the arena/tests.rs entry above crates/perry-runtime/src/box/release_tests.rs | * | async-box release tests: the closure whose GcHeader is read is allocated by the test itself, and the test needs a MUTABLE header to toggle GC_FLAG_MARKED and restore it -- try_read_gc_header yields a shared ref, so it cannot express this. Same discipline as the arena/tests_promoted_runs.rs entry above. crates/perry-ext-typescript/src/bun.rs | const HANDLE_BAND_MAX: usize = 0x100000; | #9219: raw_heap_address must reject the fetch/zlib/proxy handle bands (real addresses on Linux; macOS hides it), but this crate links only perry-ffi and cannot import value::addr_class::HANDLE_BAND_MAX. The literal is a documented mirror of that constant, used solely as the >= floor — no other band arithmetic here. Delete it if perry-ffi ever re-exports the predicate. -crates/perry-runtime/src/arena/page_meta.rs | (0x1000_0000, 0x1010_0000, 7, 0x10_0000), | #9779 test fixture, not classification: two synthetic 1 MiB block ranges with a 1 MiB hole between them, inside `#[test] fn block_range_lookup_respects_gaps_and_ends`. They are inputs to `old_arena_block_range_index`, asserting a gap is not attributed to the block below it — no runtime address is classified against them. -crates/perry-runtime/src/arena/page_meta.rs | (0x1020_0000, 0x1030_0000, 9, 0x10_0000), | #9779 test fixture — the second of the two synthetic block ranges above. +crates/perry-runtime/src/arena/page_meta/tests.rs | (0x1000_0000, 0x1010_0000, 7, 0x10_0000), | #9779 test fixture, not classification: two synthetic 1 MiB block ranges with a 1 MiB hole between them, inside `#[test] fn block_range_lookup_respects_gaps_and_ends`. They are inputs to `old_arena_block_range_index`, asserting a gap is not attributed to the block below it — no runtime address is classified against them. +crates/perry-runtime/src/arena/page_meta/tests.rs | (0x1020_0000, 0x1030_0000, 9, 0x10_0000), | #9779 test fixture — the second of the two synthetic block ranges above. diff --git a/scripts/gc_runtime_root_holders.json b/scripts/gc_runtime_root_holders.json index bca48d8f59..940d8738b7 100644 --- a/scripts/gc_runtime_root_holders.json +++ b/scripts/gc_runtime_root_holders.json @@ -276,7 +276,7 @@ "file": "crates/perry-runtime/src/gc/census.rs", "name": "PASS1_MARKED", "verdict": "non_moving_snapshot", - "why": "Real GC header addresses, deliberately untraced so the diagnostic does not keep its observed objects alive. Populated only at the end of mark propagation of a synchronous full cycle; consumed at sweep entry in the same run_to_completion invocation. The intervening full-cycle phases do not relocate or run JS callbacks. The Vec is used for membership comparisons and dropped with the census before sweep. Budgeted and minor cycles skip both boundaries. Pin re-audited 2026-09-05 after #9760 touched `gc/mod.rs`: that change is `mod heap_stats;` plus a `pub(crate) use` re-export and alters no mark/sweep control flow. `heap_stats()` is reached only from `js_bun_jsc_heap_stats` (the JS-facing `bun:jsc.heapStats()`), i.e. from mutator code, never inside a cycle, and its own module contract forbids allocation or collection during its walk. The mark-complete \u2192 sweep-entry window is unchanged. Re-audited 2026-09-05 (train125) after #9769 and #9771 touched pinned files. #9769 adds one `reg_scanner!` registration to `gc/mod.rs`; #9771 adds a feature-gated `alloc_census_init()` there and a feature-gated Rust-heap dump inside `take_census`. `alloc-census` is not in the default feature set, and decisively: `census_take_if_armed_at_full_sweep_start` does `PASS1_MARKED.with(|p| p.borrow_mut().take())` BEFORE calling `take_census`, so the snapshot has already left the thread-local by the time #9771's code runs \u2014 it cannot affect the window. Neither change alters mark/sweep control flow. Re-audited 2026-09-05 (train126) after #9755 restructured `gc/cycle.rs`. Its hunks are all root-scan machinery (`RootScanSubphase`, `RootScanCycleState`, the mutable-scanner iteration state), which runs BEFORE mark propagation completes; `gc/mod.rs` gains only a `mod young_log;` declaration. The bracketing is unchanged \u2014 `census_pass1_if_armed` is still inside `step_mark_propagation` and `census_take_if_armed_at_full_sweep_start` inside `step_sweep` \u2014 and a synchronous full mark-sweep still moves nothing between them. Re-audited 2026-09-06 after #9831 touched `gc/policy.rs`. Its hunks are (a) the tiny-parse pressure guard's pricing (`tiny_parse_pressure_headroom_bytes`, `tiny_parse_pressure_due*`, a `Cell` byte-count base) consulted from JSON.parse's mutator-side boundaries (`gc_bump_malloc_trigger`, `gc_collect_pending_suppressed_parse`, `gc_schedule_parse_boundary_collection_if_pressure`), none of which is reachable from inside a cycle, and (b) one extra `Cell` store in `note_collection_finished_arena_occupancy`, which runs from `publish_reclaim_outcome` in the Publish subphase \u2014 after `step_sweep` has already consumed the snapshot. Mark/sweep control flow between `census_pass1_if_armed` and `census_take_if_armed_at_full_sweep_start` is untouched.", + "why": "Real GC header addresses, deliberately untraced so the diagnostic does not keep its observed objects alive. Populated only at the end of mark propagation of a synchronous full cycle; consumed at sweep entry in the same run_to_completion invocation. The intervening full-cycle phases do not relocate or run JS callbacks. The Vec is used for membership comparisons and dropped with the census before sweep. Budgeted and minor cycles skip both boundaries. Pin re-audited 2026-09-05 after #9760 touched `gc/mod.rs`: that change is `mod heap_stats;` plus a `pub(crate) use` re-export and alters no mark/sweep control flow. `heap_stats()` is reached only from `js_bun_jsc_heap_stats` (the JS-facing `bun:jsc.heapStats()`), i.e. from mutator code, never inside a cycle, and its own module contract forbids allocation or collection during its walk. The mark-complete \u2192 sweep-entry window is unchanged. Re-audited 2026-09-05 (train125) after #9769 and #9771 touched pinned files. #9769 adds one `reg_scanner!` registration to `gc/mod.rs`; #9771 adds a feature-gated `alloc_census_init()` there and a feature-gated Rust-heap dump inside `take_census`. `alloc-census` is not in the default feature set, and decisively: `census_take_if_armed_at_full_sweep_start` does `PASS1_MARKED.with(|p| p.borrow_mut().take())` BEFORE calling `take_census`, so the snapshot has already left the thread-local by the time #9771's code runs \u2014 it cannot affect the window. Neither change alters mark/sweep control flow. Re-audited 2026-09-06 after #9831 touched `gc/policy.rs`. Its hunks are (a) the tiny-parse pressure guard's pricing (`tiny_parse_pressure_headroom_bytes`, `tiny_parse_pressure_due*`, a `Cell` byte-count base) consulted from JSON.parse's mutator-side boundaries (`gc_bump_malloc_trigger`, `gc_collect_pending_suppressed_parse`, `gc_schedule_parse_boundary_collection_if_pressure`), none of which is reachable from inside a cycle, and (b) one extra `Cell` store in `note_collection_finished_arena_occupancy`, which runs from `publish_reclaim_outcome` in the Publish subphase \u2014 after `step_sweep` has already consumed the snapshot. Mark/sweep control flow between `census_pass1_if_armed` and `census_take_if_armed_at_full_sweep_start` is untouched. Re-audited 2026-09-05 (train126) after #9755 restructured `gc/cycle.rs`. Its hunks are all root-scan machinery (`RootScanSubphase`, `RootScanCycleState`, the mutable-scanner iteration state), which runs BEFORE mark propagation completes; `gc/mod.rs` gains only a `mod young_log;` declaration. The bracketing is unchanged \u2014 `census_pass1_if_armed` is still inside `step_mark_propagation` and `census_take_if_armed_at_full_sweep_start` inside `step_sweep` \u2014 and a synchronous full mark-sweep still moves nothing between them. Re-pinned 2026-09-05 for the #9740 hot-TLS conversion of this file: the sole change is `thread_local!` \u2192 `crate::perry_thread_local!`, a macro-name swap with identical declaration syntax and `.with()` call sites. No control flow, no phase boundary, and no storage semantics change. Re-audited 2026-09-06 (train128) after #9794's GC diagnostics touched `gc/mod.rs` and `gc/policy.rs`: both gain diagnostic module declarations and counters only \u2014 no mark/sweep control flow, and the census bracketing in `step_mark_propagation` / `step_sweep` is unchanged. Re-audited for #9794's GC diagnostics: `gc/mod.rs` gains `mod diag_sites;` / `mod survival_diag;`, a re-export, a `diag_sites::full_started(...)` call at TRIGGER time (before mark propagation begins), and exit-time reporting. Nothing executes between mark-complete and sweep-entry, so the window is unchanged. Re-audited 2026-09-06 for the retained array-growth verifier fix: the cycle.rs change passes the existing non-copying evacuation verifier an explicit all-forwarded policy. That call remains in minor finalization, outside the synchronous full-cycle census window; its root and heap reads do not allocate GC objects, move objects, or invoke JS callbacks. The mark-complete and sweep-entry boundaries are unchanged. Re-audited 2026-09-05 after #9830 touched `gc/policy.rs`. That change is (a) six `thread_local! {` blocks rewritten as `crate::perry_thread_local! {` and (b) one `#[cfg(test)]` accessor listing the trigger path's hot-slot indices. The macro keeps the same storage, the same `.with()` at every read and write, and the same destructor registration (the teardown guard exists exactly when `needs_drop` holds, which is what `std::thread_local!` already decided); no value, predicate or branch in the file changes, so no mark or sweep control flow does. The one new behaviour is on a declaration's FIRST read: `HotKey::resolve_and_cache` takes a mutex and allocates a key through the GLOBAL allocator. Even if a first read landed inside this window it would be sound \u2014 the window's contract is that nothing relocates and no JS callback runs, and a mimalloc allocation does neither. `census_pass1_if_armed` is still inside `step_mark_propagation` and `census_take_if_armed_at_full_sweep_start` inside `step_sweep`; the bracketing is untouched. Re-audited 2026-09-06 (train132) after #9860 and #9845 touched `gc/mod.rs`. Both hunks are re-export lists and nothing else: #9860 adds `idle_reclaim_elapsed_starts` / `IDLE_RECLAIM_REARM_MS`, and #9845 adds `owner_is_dead_copied_minor_from_space_of_type`. No mark or sweep control flow changes. #9845's substantive work sits in `gc/oldgen.rs` and `gc/copying.rs`, neither pinned: the copying-minor arm (`finalize_dead_copied_minor_from_space_regexps`) runs on a MINOR, which skips both census boundaries; the full-cycle arm (`collect_dead_registered_regexps_post_trace`, from `with_dead_collection_finalize`) walks the RegExp registry building a Vec of addresses \u2014 no GC allocation, no JS callback, so it cannot relocate the snapshot's subjects \u2014 and it is reached from the sweep body, i.e. AFTER `census_take_if_armed_at_full_sweep_start` has already `take()`n the snapshot out of the thread-local. The mark-complete -> sweep-entry window is unchanged.", "window": { "start": { "file": "crates/perry-runtime/src/gc/census.rs", @@ -293,7 +293,7 @@ "sources": { "crates/perry-runtime/src/gc/census.rs": "388414f9629f196e84673e91bebd04bdcdcabdaa180252d2dfe4b82d1b49ca5a", "crates/perry-runtime/src/gc/cycle.rs": "2e2f5adca2229f74409e01a1cb571e2147cd8a33f58d0976711fce98d4777309", - "crates/perry-runtime/src/gc/mod.rs": "7dd42b9506a97e6844fd3225dc53dfd59512631784750f58ff72208d68595481", + "crates/perry-runtime/src/gc/mod.rs": "43523b66595c61516ef6fcd4139d3ec5b4768a13c46ae1470c1d45481eacfdd9", "crates/perry-runtime/src/gc/policy.rs": "dc9242ed40c0aa9c411d1ec0235c0219c6716dd82d56eb4d46578f7e889825d2", "crates/perry-runtime/src/gc/progress.rs": "a5ad3971bbe4047229ca57325234780daa85921dbc778e1c08dff4ad07ccfb96" } @@ -2784,27 +2784,27 @@ "name": "SURVIVOR_ARENA_1" }, { - "file": "crates/perry-runtime/src/arena/page_meta.rs", + "file": "crates/perry-runtime/src/arena/page_meta/mod.rs", "name": "OLD_GEN_PAGE_DIRTY_EPOCH" }, { - "file": "crates/perry-runtime/src/arena/page_meta.rs", + "file": "crates/perry-runtime/src/arena/page_meta/mod.rs", "name": "OLD_GEN_RECLAIM_POOLED_BYTES" }, { - "file": "crates/perry-runtime/src/arena/page_meta.rs", + "file": "crates/perry-runtime/src/arena/page_meta/mod.rs", "name": "OLD_GEN_RECLAIM_RETURNED_BYTES" }, { - "file": "crates/perry-runtime/src/arena/page_meta.rs", + "file": "crates/perry-runtime/src/arena/page_meta/mod.rs", "name": "OLD_GEN_RECLAIM_REUSABLE_BYTES" }, { - "file": "crates/perry-runtime/src/arena/page_meta.rs", + "file": "crates/perry-runtime/src/arena/page_meta/mod.rs", "name": "OLD_PAGE_META_SNAPSHOT_CALLS" }, { - "file": "crates/perry-runtime/src/arena/page_meta.rs", + "file": "crates/perry-runtime/src/arena/page_meta/mod.rs", "name": "PAGE_GENERATION_CACHE" }, { diff --git a/scripts/shape_descriptor_census.py b/scripts/shape_descriptor_census.py index 927bc0d098..0f03ff37b4 100644 --- a/scripts/shape_descriptor_census.py +++ b/scripts/shape_descriptor_census.py @@ -569,10 +569,15 @@ def assert_authority_surfaces(sources: dict[str, str]) -> None: require_code(body, r"obj_type\s*==\s*crate::gc::GC_TYPE_OBJECT", f"{name} GC kind") if re.search(r"regex_header_has_magic|object_type", body): raise CensusError(f"{name} reintroduced an old payload discriminator") + # #9845 moved the header off the malloc arm into the nursery, so the birth + # site is now `arena_alloc_gc`. What this asserts is unchanged and is the + # point of the check: whichever allocator RegExp is born from, it is born + # with its OWN GcHeader kind, never as a generic object that something later + # has to re-identify by payload magic. regexp_alloc = function_body(regex_runtime, "js_regexp_new") require_code( regexp_alloc, - r"gc_malloc\s*\([^;]*crate::gc::GC_TYPE_REGEXP", + r"(?:gc_malloc|arena_alloc_gc)\s*\([^;]*crate::gc::GC_TYPE_REGEXP", "RegExp dedicated GC birth kind", ) expando_kind = function_body(exotic_expando, "exotic_expando_kind") diff --git a/scripts/thread_local_cold_allowlist.json b/scripts/thread_local_cold_allowlist.json index 9686b6e210..5884403015 100644 --- a/scripts/thread_local_cold_allowlist.json +++ b/scripts/thread_local_cold_allowlist.json @@ -4,7 +4,6 @@ "files": { "crates/perry-runtime/src/agent.rs": 1, "crates/perry-runtime/src/arena/block.rs": 5, - "crates/perry-runtime/src/arena/page_meta.rs": 2, "crates/perry-runtime/src/async_context.rs": 3, "crates/perry-runtime/src/async_hooks.rs": 7, "crates/perry-runtime/src/builtins/console.rs": 4, @@ -84,6 +83,7 @@ "crates/perry-runtime/src/v8.rs": 2, "crates/perry-runtime/src/wasi.rs": 1, "crates/perry-runtime/src/weakref.rs": 2, - "crates/perry-runtime/src/web_storage.rs": 2 + "crates/perry-runtime/src/web_storage.rs": 2, + "crates/perry-runtime/src/arena/page_meta/mod.rs": 2 } } diff --git a/test-files/test_gap_gc_for_in_proxy_callback_roots.ts b/test-files/test_gap_gc_for_in_proxy_callback_roots.ts new file mode 100644 index 0000000000..cc2d14d882 --- /dev/null +++ b/test-files/test_gap_gc_for_in_proxy_callback_roots.ts @@ -0,0 +1,46 @@ +// for...in retains its output and current receiver while Proxy traps run JS. +// The inherited proxy fires after own keys have already grown the output past +// its initial capacity. Churn in each trap exposes stale locals under the GC +// schedule/protection matrix without relying on a Node-only gc() function. +let trapCalls = 0; +function churn() { + for (let i = 0; i < 24; i++) { + const garbage = { value: ["temporary", i, trapCalls] }; + if (garbage.value.length !== 3) throw new Error("allocation witness"); + } + trapCalls++; +} +function wrapped(target: any): any { + return new Proxy(target, { + ownKeys(value) { churn(); return Reflect.ownKeys(value); }, + getOwnPropertyDescriptor(value, key) { + churn(); return Reflect.getOwnPropertyDescriptor(value, key); + }, + getPrototypeOf(value) { churn(); return Reflect.getPrototypeOf(value); }, + }); +} +const inherited = wrapped({ inheritedA: 1, inheritedB: 2, hidden: 3 }); +const target: any = Object.create(inherited); +for (let i = 0; i < 14; i++) target["own" + i] = i; +Object.defineProperty(target, "hidden", { value: 4, enumerable: false, configurable: true }); +for (const receiver of [target, wrapped(target)]) { + const keys: string[] = []; + for (const key in receiver) keys.push(key); + const expected = Array.from({ length: 14 }, (_, i) => "own" + i).concat(["inheritedA", "inheritedB"]); + if (keys.join(",") !== expected.join(",")) throw new Error("enumeration lost keys: " + keys.join(",")); + console.log(keys.join(",")); +} +if (trapCalls === 0) throw new Error("traps were not invoked"); +const descriptorProxy = new Proxy({ property_name: 23 }, { + getOwnPropertyDescriptor(value, key) { + return { + enumerable: true, configurable: true, writable: true, + get value() { churn(); return value[key]; }, + }; + }, +}); +const descriptor = Reflect.getOwnPropertyDescriptor(descriptorProxy, "property_name")!; +if (descriptor.value !== 23 || !descriptor.writable || !descriptor.enumerable || !descriptor.configurable) { + throw new Error("descriptor fields lost across collection"); +} +console.log("PASS for-in callback roots"); diff --git a/test-files/test_issue_9842_form_data_blob_upload.ts b/test-files/test_issue_9842_form_data_blob_upload.ts new file mode 100644 index 0000000000..e80c71b2a6 --- /dev/null +++ b/test-files/test_issue_9842_form_data_blob_upload.ts @@ -0,0 +1,55 @@ +// Regression for #9842: FormData.append/set must preserve Blob and File +// values, and Request must serialize FormData as a non-empty multipart body. + +const form = new FormData(); +const original = new File( + [new Uint8Array([0, 255, 13, 10, 65])], + "original.bin", + { type: "application/octet-stream", lastModified: 1234 }, +); +form.append("caption", "Perry upload"); +form.append("original", original); +form.append("renamed", original, "renamed.bin"); +form.set("blob", new Blob(["blob payload"], { type: "text/plain" })); + +const originalEntry = form.get("original") as File; +const renamedEntry = form.get("renamed") as File; +const blobEntry = form.get("blob") as File; +console.log( + `entries=${originalEntry instanceof File}/${originalEntry.name}/${originalEntry.lastModified};` + + `${renamedEntry instanceof File}/${renamedEntry.name};` + + `${blobEntry instanceof File}/${blobEntry.name}/${blobEntry.type}`, +); + +const request = new Request("https://example.test/upload", { + method: "POST", + body: form, +}); +const contentType = request.headers.get("content-type") || ""; +console.log(`multipart=${contentType.startsWith("multipart/form-data; boundary=")}`); + +const bytes = new Uint8Array(await request.arrayBuffer()); +console.log(`body=${bytes.byteLength > 5}/${bytes.includes(255)}`); + +const parsedRequest = new Request("https://example.test/upload", { + method: "POST", + body: form, +}); +const parsed = await parsedRequest.formData(); +const parsedOriginal = parsed.get("original") as File; +const parsedRenamed = parsed.get("renamed") as File; +const parsedBlob = parsed.get("blob") as File; +console.log( + `parsed=${parsed.get("caption")};${parsedOriginal.name}/${parsedOriginal.type}/${parsedOriginal.size};` + + `${parsedRenamed.name};${parsedBlob.name}/${await parsedBlob.text()}`, +); +console.log( + `binary=${[...new Uint8Array(await parsedOriginal.arrayBuffer())].join(",")}`, +); + +const explicit = new Request("https://example.test/upload", { + method: "POST", + headers: { "content-type": "application/custom" }, + body: form, +}); +console.log(`explicit=${explicit.headers.get("content-type")}`); diff --git a/test-parity/gc_repsel_corpus.txt b/test-parity/gc_repsel_corpus.txt index 7f76d12528..80310c49f9 100644 --- a/test-parity/gc_repsel_corpus.txt +++ b/test-parity/gc_repsel_corpus.txt @@ -856,3 +856,6 @@ test_gap_gc_string_repeat_reentrant_count # Pre-fix: SIGBUS after the first copying minor under # PERRY_GC_PROTECT_FROMSPACE=1; post-fix: `600000 0.35 0.25 0.1 100000 0.35`. test_gap_gc_coalesce_local_root + +# for-in output/receiver custody across Proxy callbacks (#4644) +test_gap_gc_for_in_proxy_callback_roots diff --git a/tests/release/packages/perry-solid/expected-jsx.txt b/tests/release/packages/perry-solid/expected-jsx.txt new file mode 100644 index 0000000000..1f02a19fa2 --- /dev/null +++ b/tests/release/packages/perry-solid/expected-jsx.txt @@ -0,0 +1 @@ +PASS Solid JSX: native updates, components, keyed identity, conditionals, spreads, refs, fragments, disposal diff --git a/tests/release/packages/perry-solid/fixture.sh b/tests/release/packages/perry-solid/fixture.sh index 84da2669d4..a2ee1d826e 100755 --- a/tests/release/packages/perry-solid/fixture.sh +++ b/tests/release/packages/perry-solid/fixture.sh @@ -18,3 +18,12 @@ if ! grep -Eq 'Found [0-9]+ module\(s\): [1-9][0-9]* native, 0 JavaScript' perry echo 'FAIL perry-solid — expected every module to compile natively' exit 1 fi +cp perry-compile.log perry-renderer-compile.log +node test/jsx/oracle.cjs +node --conditions=browser test/jsx/generated.ts > jsx-node-out.txt +diff -u "$fixture_dir/expected-jsx.txt" jsx-node-out.txt +PERRY_DISABLE_BUILD_CACHE=1 fixture_compile_run_diff perry-solid-jsx test/jsx/main.tsx "$fixture_dir/expected-jsx.txt" +if ! grep -Eq 'Found [0-9]+ module\(s\): [1-9][0-9]* native, 0 JavaScript' perry-compile.log; then + echo 'FAIL perry-solid-jsx — expected every module to compile natively' + exit 1 +fi