perf(regex): drop the traced-source side table, share one program set per header (72→56 B), tag the matcher kind

[proggeramlug]

Stacked on #9892 (perf/regex-literal-site-key @ 91a7791). Three runtime-only commits, written by codex from issue #9908's design lead and the segmenter lane's matcher flag, not yet compiled (dev box out of disk at the time); gates and the cc-rig measurement run on perrymaster and will be appended here.

What changes

1. 0178825dd — remove REGEX_SOURCE_TABLE. It existed (RegExpRouter: trie.buildRegExp() returns non-RegExp object in slot 0; matchers[method] = arr corrupts keys_array #637) because the header's pattern_ptr/flags_ptr were raw pointers into strings the GC could free while the header was gc_malloc'd and untraced. Since perf(regex): allocate the RegExp header in the nursery, not the malloc arm #9845 those two slots are traced GC edges, so .source, .flags, lazy compilation and the RegExp-pattern constructor arm read them directly; every insert, lookup, move-rekey and death removal of the table is gone. Write barriers added where RegExp.prototype.compile replaces the two edges. EscapeRegExpPattern is WTF-8 byte-aware so .source preserves lone surrogates. REGEX_POINTERS is untouched (the copied-minor finaliser enumerates it).
2. 1469d8089 — one header-owned Arc<Programs> handle instead of three per-object matcher pointers; the content cache holds the same shared set, the literal-site cache one weak reference to the complete set. Header 72 → 56 B (−22 %); header_bytes was 72.3 MB per 3300-char reply.
3. 6f98d35d5 — MatcherKind tag in the former padding byte: regexp_test_str_bounded compiles once, branches once, borrows the selected matcher without cloning an Arc (previously two ensure_regex_compiled + Arc clone/drop per call, ~172 k calls per reply from the segment loop).

Expected counters (cc rig, one 3300 reply)

side_table_inserts / new: 2.00 → 1.00; regex_header_clear_dead_for_gc half its removals (was 2.63 % of the thread in the probe profile); header_bytes −22 %; compile counts and site_key_hit unchanged.

Tests (named; not yet executed)

regexp_construct_reads_source_and_flags_from_the_pattern_header, regexp_compile_replaces_the_header_source_and_flags, regexp_source_round_trips_wtf8_lone_surrogates_from_the_header, regexp_header_is_one_56_byte_per_object_record, bounded_test_matcher_tag_routes_fancy_patterns_to_fancy_regex (sabotage: a (?<=left)right pattern whose standard program is the never-match placeholder — a wrong Standard tag fails it across lazy build, born-built cache hit, and compile).

Gates

Not run (disk): cargo build --release -p perry default features, nm, the runtime suite, the regex filter, test262's RegExp subset. Draft until perrymaster's seam runs them.

Gate history

• cc98590a7: cargo build --release -p perry rc=0, but that builds the compiler's default-features = false runtime copy. The archive build (--features perry-runtime/wasm-host) and the lib test target failed: error[E0425]: cannot find value owned_flags in this scope at regex.rs:1351 (a binding present only in the other cfg's arm).
• 1158dff8c (head): "fix(regex): retain canonical flags through allocation" threads owned_flags through the match site_entry tuple on both arms and touches its five call sites. Re-gating under the archive feature set on perrymaster; rows follow.
• 1158dff8c re-gated on perrymaster (picked onto the site-key tree, stamp 0f867bbb4): cargo build --release -p perry rc=0; full-feature archives rc=0; runtime suite 3231 passed / 0 failed / 4 ignored with the five named tests green. test262 RegExp subset not run (no vendor/test262 on that host) — the remaining gate before un-draft.

Measured (perrymaster, cc 3300-char reply, 5-round alternating rotation, load 1.15–1.32; paired deltas only — see caveat)

arm	turn CPU s (min / mean)	peak RSS MB	settled RSS MB (120 s)
main (train base)	3.58 / 3.62	603–617	485
site key #9892	3.41 / 3.43	615–624	500
+ this PR	3.32 / 3.38	590–601	472

Per round this PR beats main by 0.19–0.36 s (5/5, ≈ −6 %) and the site key alone by 0.01–0.09 s: the CPU win is the site key's; this PR's contribution is memory. Peak −13…−24 MB vs the site key and −5…−13 vs main; settled −28 vs the site key and −13 vs main, i.e. it removes the site key's residual +15 MB and ends below main. 400-char turn: peak 516 vs 545 main (−29).

Counters on one reply (whole [regex-diag] line, new=1152724): side_table_inserts/new = 1.000 (was 2.000) with src_ins=0; header_bytes/new = 56.0 B (was 72.0, −22 %); site_key_hit 99.79 % of new; exec/test/match/replace counts unchanged; compiles std=208 vs 209 before (one fewer, to explain), fancy=88 and repeat=33 unchanged.

Caveat: every arm's absolute CPU on this host now reads ~35 % above the same rotation's afternoon values (a foreign 27 %-CPU service appeared on the box); the within-rotation pairs above hold, cross-rotation absolutes do not.

Follow-up 107d40adb — preserve live literal programs on eviction (uncompiled; perrymaster stage I6d)

The per-pattern diag diff between the site-key tree and this branch showed three patterns built more often per reply (the 12.8 KB emoji /g literal 1 → 2, \s+ 2 → 3, \[1m\]/i 1 → 2) with cache_clears=2 on both arms. Mechanism: the removed source table held only source/flags text, never programs. This branch's one Weak<Programs> per literal site (the site-key tree held three weak matcher references that the engine maps kept alive) could no longer upgrade after the 1,024-slot content table overwrote a colliding entry and young-header finalization released the last strong reference; independently, the four 512-entry engine/validation maps cleared wholesale on overflow.
Change: the content construction cache is a bounded fingerprint map with collision buckets and full byte verification; capacity eviction happens only on a distinct-content miss and never evicts an entry whose (pattern, flags) is still recorded in the literal-site table; a content-owned build publishes one weak bundle reference to every matching literal site (no strong site → program table); the engine maps evict one entry instead of clearing. [regex-diag] keeps cache_clears as a zero control and adds evictions=. Named sabotage test: literal_site_program_is_not_rebuilt_after_cache_overflow_and_young_collection.
Acceptance on the cc reply: every pattern builds ≤ 1, lazy_builds ≈ 126, compiles std ≤ 208, cache_clears=0, retention not above the census finding; paired CPU vs the previous head not slower. Open question the per-pattern table answers: cc's 1,062 live literal sites exceed the 1,024-entry table; if the overflow set still rebuilds, the table must be sized above the working set.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[jdalton]

Review of 107d40adb9881ff5f91f94124e24907fcfea5796 (2026-09-07).

The content cache is bounded in entry count, but its new miss path needs a distinct-content workload: make_room first sums every bucket in entry_count, then evict_one_dynamic can scan all entries and call site_key::references_content for each candidate. Literal hits do not exercise this cost. Please add a workload beyond MAX_ENTRIES with pinned literals mixed with changing dynamic patterns, reporting construction CPU and retained bytes with the cache on/off. Keep full byte-comparison coverage for fingerprint collisions. The header-size/literal-loop measurements alone cannot establish that the new cache has no dynamic-construction regression.

Validation scope: source/diff inspection; I have not run this PR's build or test suite locally.