Merge train: #9794, #9780, #9774, #9814, #9756, #9796, #9775 (all previously conflicting) - #9863
Merged
Conversation
added 27 commits
September 6, 2026 05:53
…l origins, allocation-site sampling Three instruments for the cc-perf campaign, all inert unless asked for. `PERRY_GC_DIAG=1` gains the lines that say WHY the collector ran: `[gc-trigger]` (every predicate input at each decision site), `[gc-full]` (the arm behind each synchronous full mark-sweep, counted per site), `[gc-budgeted] start/done` (steps, per-phase step time, root-scan share), `[gc-charge]` (mutator-assist / synchronous-full time per calling site, resolved to JS display names) and `[gc-survival]` (per copying minor, the root that first reached each surviving byte — shadow stack, native stack map, named scanner, remembered set by old-parent type — with transitive reach charged to the originating root through a parallel worklist origin vector). `PERRY_ALLOC_SITE_SAMPLE=<bytes>` samples the arena allocation sites byte- proportionally across the runtime allocators AND the codegen inline bump path (the mirrored inline block limit is capped at one interval while sampling, so the fast path returns to the runtime once per interval). The survival test is sabotage-checked: disabling the drain propagation charges the 40 elements to `worklist_drain` and the test fails on that row. The knob's OFF state and magnitude parse are pinned next to the other GC knobs. `gc_diag_enabled()` gets the per-thread test override the census already has, so the diag paths are testable without touching the process environment. Claude-Session: https://claude.ai/code/session_01YPfnmWZmSpSWpmnoXvH8z2
…iptors on the primitive-string path
Three allocations a JS program can never observe, found with the
allocation-site sampler (`PERRY_ALLOC_SITE_SAMPLE`) on the compiled
claude-code TUI, where they are the largest attributed source of garbage in
both the streaming turn and the render pass that follows it.
1. A one-ASCII-character string is now the canonical per-thread header.
`js_string_char_at` minted a fresh 32-byte heap string per character read,
and everything that walks a string a character at a time goes through it:
`s[i]`, `charAt`, string spread, the String-wrapper index installer. There
are 128 possible contents. The table has the same residency contract as the
small-integer string table next to it (longlived arena, `refcount = 0` so it
is never mutated in place, pinned out of the young generation) and rides
that table's existing root scanner rather than registering a 96th one.
2. Runtime-internal constant property names resolve through the intern table.
The `globalThis` builtin lookup, `x.constructor`, `toString` resolution and
primitive-method dispatch each built a fresh heap string for a literal name
on every call; `js_get_global_this_builtin_value` alone accounted for 133 MB
of the 990 MB one 3300-character reply allocates. `string::canonical_key`
routes them through the content-keyed per-thread table that
`js_string_materialize_to_heap` already uses, which is also what the
property read/write fast paths require of a key.
3. A `String` wrapper no longer stores a property descriptor per character.
ECMA-262 §10.4.3 gives every in-range index of a String exotic object
`{ writable: false, enumerable: true, configurable: false }` — a fact of the
class and the boxed length, not per-object state — so `get_property_attrs`
answers it from the wrapper's payload. Storing it cost, per boxed character,
a Rust `String`, a `PROPERTY_DESCRIPTORS` entry only a full collection's
dead-owner prune could reclaim, an owner-index entry, and one program-wide
`prop_plan_epoch_bump()`. A sloppy method call on a string primitive boxes
its receiver, so the TUI paid all of it per rendered line. A real stored
descriptor still wins, so `Object.freeze`/`defineProperty` on a wrapper are
unchanged.
`PERRY_GC_DIAG=1` also gains `[gc-primitive-dispatch]`: which
`<Builtin>.prototype.<method>` names reach the primitive-method fallback, how
often, and how many wrapper index properties they cost — the counter that says
whether a boxing fix ran.
Claude-Session: https://claude.ai/code/session_01YPfnmWZmSpSWpmnoXvH8z2
… % more The five tables #9754 converted were valued individually with a measurement-only `PERRY_YOUNG_LOG=0` gate on `RuntimeRootVisitor::young_scope()` (all five scanners fall back to their full walk together inside one binary), plus a third arm — `cc_base_new`, main `1d63fa91f`, no logs at all. Three interleaved rounds, `stream_scale` len 3300, identical collection schedule in every arm (minors 196/194/196, budgeted steps 60/59/59), so these are scan costs: | scanner, ms per turn | main | log, full walk | log, minor walk | |------------------------------|-----------------|-----------------|-----------------| | all 95 scanners | 14761/14105/19411 | 23368/23286/26974 | 2667/2852/3674 | | scan_shape_table_rekey_mut | 10884/11077/14458 | 18893/18655/22125 | 1426/1567/1947 | | scan_descriptor_roots_mut | 1807/1192/2223 | 2257/2467/2548 | 127/136/145 | | scan_closure_dynamic_props | 1014/985/1347 | 890/902/959 | 224/236/209 | | transition_cache scanner | 121/123/159 | 254/221/246 | 85/94/145 | | shape_cache scanner | 89/89/113 | 159/153/167 | 121/122/151 | The shape cache is the one table where the log loses to the walk it replaced: +34 ms (+35 %) against main, having skipped **0.0 % of 3.85 M entry visits in every one of 107 collections**. The cause was already documented — the canonical keys arrays are allocated in the LONGLIVED arena, which `addr_is_minor_relevant` must answer `true` for because a longlived parent is not write-barriered, and a longlived object is never promoted, so no entry ever leaves the log. So it goes back to the plain `values_mut()` walk: the arm helper, its production and test-seam call sites, the thread-local log, the name constant and the `debug_assert_logged` re-derivation are all deleted. An inert log is not free — it is a permanent arming obligation on every future writer of that cache plus a suppression audit that has to keep proving each site — and it should not land on the promise of a longlived remembered set that does not exist yet. When that set exists and makes this table skip something, the log can come back with a measurement. The test is kept as a scanner test (a young entry reachable only through the cache still moves and is re-keyed in both the inline slot and the overflow map) and now asserts that NO `[gc-young-log]` row exists for the table, so re-adding a log here without re-measuring is a red test. Note for anyone repeating this on another table: the two-arm version of this experiment gives the wrong answer. With the log merely disabled, the full-walk arm still pays its upkeep — a `take_sorted()` whose sorted result is discarded and an `addr_is_minor_relevant` probe per entry to rebuild `kept` — so every "off" row above is worse than main, by +7.8 s on the shapes table alone. Only the third arm says whether a log should exist at all. Claude-Session: https://claude.ai/code/session_014UZWia6L37DpA93VLtNK9m
`ShapeIndex.slots` — the content-hash to slot accelerator built for every keys array past `KEYS_INDEX_THRESHOLD` — was a `PtrHashMap<u64, SlotList>` per shape: a 33-byte hashbrown bucket per key in a power-of-two table (2.1 KB for a 40-key object). The compiled claude-code TUI holds thousands of these. Every hit the index produces is re-validated against the key bytes (`shape_slot_lookup_verdict`), so a colliding answer is a miss and never a wrong property — which is what lets the stored hash be narrow. `SlotIndex` is an open-addressing table of (16-bit tag, 16-bit slot) cells (4 B; a 16-bit tag and 32-bit slot only past 65,535 keys); the tag is the top of a golden-ratio fold of the FNV-1a hash (FNV's own high bits barely move for short keys), the probe position is a function of the tag alone so cells re-place themselves on growth and after a delete's `retain_shift`, load kept at or below 7/8. 40 keys: 64 x 4 B = 256 B against the 2.1 KB hashbrown table. Same O(1) probe; a repeated note-hit no longer appends a duplicate.
…es, engine prototype switch Rebased onto main after #9764 landed as ddbe0b1; the site cache, header-authoritative lookups and find-only global `test` are main's now and are gone from here. The program-cache coherence fix moved to #9801. Three changes remain. * The capture-group cliff. `repeat_matcher::capture_layout` takes a pattern off the linear engine when ECMA-262's RepeatMatcher capture semantics are observable — a capture directly under a quantifier, or a capture inside a negative lookaround. That routing is a correctness requirement, but the engine it routes to is a classical backtracker with no step budget, so adding parentheses fell from linear time to exponential (`/^(a+)+$/.test("a"*28 + "!")`: 16,522 ms; node 4,798 ms). 6.3 % of 4,463 real literals take that route. Both engines accept the same LANGUAGE and differ only in capture ASSIGNMENT, so `linear_rules_out_match` asks the linear program first and a subject it rules out — which is what every ReDoS input is — never reaches the backtracker. This removes the reachable exponential case; it does not bound the worst case (that needs the step budget open upstream as ridiculousfish/regress#177). * Allocation-free cache probes. The three compiled-program caches were `HashMap<(String, String), _>`, so every probe allocated two Strings and copied the pattern text, once per RegExp OBJECT. `ProgramKey = (Arc<str>, Arc<str>)` makes a probe two refcount increments; the remaining materialisations are cold (`RegExp.prototype.compile`, the syntax-error fallback). * `PERRY_REGEX_ENGINE=regress`, off by default: routes every pattern through the ECMAScript backtracker and installs a shared never-match placeholder as the standard program, so the tier-0 engine architecture can be measured in a real binary. Not a supported configuration — the backtracker has no budget. Tests: quantified_capture_pattern_does_not_backtrack_on_a_non_matching_subject, plus the `capture_layout` predicate assertions rewritten around the `(layout, needed)` pair. Claude-Session: https://claude.ai/code/session_01YPfnmWZmSpSWpmnoXvH8z2
changelog.d/README.md asks for `<PR-number>-<slug>.md`; the three fragments landed unnumbered. Renames only — no entry text and no code changes, so the measured candidate binary is unaffected. Claude-Session: https://claude.ai/code/session_014UZWia6L37DpA93VLtNK9m
This was referenced Sep 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge train: #9794, #9780, #9774, #9814, #9756, #9796, #9775 — seven PRs that had been stuck on conflicts, all resolved here.
The conflicts, and what each actually needed
#9756 presented as 43 conflict hunks across 13 files of GC internals and needed zero hand-merging. Its description opens with "Stacked on #9755 (its commit is included; review only the last commit)", and #9755 landed in train #9817. Comparing commit subjects against
mainshowed 6 of its 8 commits already there; picking the 2 genuinely missing ones applied clean. Reading the PR body first would have saved the most work of anything today.#9814 was the one that needed real judgement.
boxed_primitives.rshad a mid-function conflict where neither side alone was correct: #9794 addsboxed_string_wrapper_utf16_len, #9814 makes string indices virtual and roots the length install — and #9814's owndescriptor_state.rscalls #9794's helper.--oursdropped the virtual indices (its test then crashed withTypeError: Cannot assign to read only property '0');--theirsdropped the helper (unresolved import). The merge keeps #9814's rooted install plus #9794's helper, and dropsinstall_string_wrapper_indices, which virtual indices make obsolete.That merge then created a real bug, caught by the suite rather than the compiler: it left two synthesize paths, and #9814's
string_wrapper::has_indexearly-return short-circuited above the stored-descriptor table probe, soObject.definePropertyon a wrapper index could never win. Synthesis now happens only in the fallback below the table. All fivestring_wrappertests pass.#9796 — union for two independent
regex.rsadditions, then #9801 (landed since) had addedlazy.rscall sites using the old(String, String)cache key against #9796's new(Arc<str>, Arc<str>). Four stale sites in total, each surfacing at a different stage: compile, test-compile, and gate.#9775 —
mainalready hastry_import_meta_require, so HEAD wins that file; itsinprocess.rswork is kept.#9794, #9780, #9774 — a
gc/mod.rsentry-list union, a two-comment concat, and the census hot-TLS conversion respectively.check_thread_localsis now green on every file for the first time.File-cap splits
Four files crossed the 2000-line gate:
descriptor_state.rs→descriptor_state/gc_scan.rs;regex.rs→regex/program_key.rsandregex/replace_expand_fancy.rs;inprocess.rs→inprocess/optimize_emit.rs;lower/tests.rs→lower/tests/ui_widget_add_child.rs. Each verified in both feature configurations — a--features regex-enginebuild passes while the default one fails when a moved item sits behind that gate.The
raw_handle_debt--no-raise-vsinvocation then correctly refused to let the newreplace_expand_fancy.rsstart with a ceiling: a brand-new file must have zero bare reads. Rather than grandfather it, all four moved sites becamewith_mut_ptr— preserving the deliberate per-iteration re-read after an allocatingjs_string_from_str— so total debt dropped by four instead of relocating.Gate verdicts
SHAPE_CACHE_YOUNG's verdict was deleted because #9756 removes that log — the inventory failing a stale entry is it working as designed.NEVER_MATCH(Arc<regex::Regex>, a Rust-allocator program) classified.PASS1_MARKED's window re-pinned after #9794's diagnostics touchedgc/mod.rsandgc/policy.rs: both gain declarations and counters only, and the census bracketing instep_mark_propagation/step_sweepis unchanged.Validation
64/64 lint gates;
perry-runtime,perry-codegen,perry-hir,perry-stdlib,perry-transform,perry-ext-http— run serially (RUST_TEST_THREADS=1).