diff --git a/cc-perf-campaign/codex/REPORT_regex_census_rows.md b/cc-perf-campaign/codex/REPORT_regex_census_rows.md new file mode 100644 index 0000000000..604cab6ec6 --- /dev/null +++ b/cc-perf-campaign/codex/REPORT_regex_census_rows.md @@ -0,0 +1,175 @@ +# RegExp heap-census rows + +## Branch and commits + +- Branch: `diag/regex-census-rows` +- Base: `a93908a6cc684d511a0af8561b65be4f0536fbd4` (`fork/perf/regex-literal-site-test`) +- Common implementation: `49f551f05` (`feat(diagnostics): account regex tables in heap census`) +- #9958 site-table integration: `5e8079138` (`feat(diagnostics): add literal-site regex census rows`) + +Both implementation commits are diagnostic-only. No construction, matching, +cache-maintenance, or collection path calls the new walkers. The walkers run +only while an explicitly requested heap census is being assembled. + +## Emitted rows and byte derivation + +Every row is a JSON object in the census's existing `side_tables` array: +`{"table":"regex.","entries":,"bytes":,...}`. + +- `regex.pointers`: `entries` is `REGEX_POINTERS.len()`; `bytes` is the + HashSet bucket/control estimate; `live_headers` is the marked-or-pinned + RegExp-header count at sweep entry. +- `regex.program_cache`: the 512-entry `REGEX_CACHE`; `bytes` is HashMap + storage plus one de-duplicated lower-bound estimate per compiled `regex` + program. `compiled_programs`, `opaque_program_bytes`, `cleared`, and + `evictions` are included. The event counters read the already-existing + `PERRY_REGEX_DIAG` state and are zero when that diagnostic is off. +- `regex.fancy_cache`: the 512-entry `FANCY_CACHE`; HashMap storage plus one + de-duplicated `fancy_regex::Regex` lower bound. +- `regex.repeat_cache`: the 512-entry `REPEAT_MATCHER_CACHE`; HashMap storage, + the public wrapper/Arc lower bound, and visible capture-name Vec/String + buffers. The opaque `regress::Regex` heap graph is not exposed. +- `regex.validated_patterns`: the 512-entry validation map, including map + storage and owned pattern/flag String capacities. +- `regex.content_cache`: the 1,024-entry content map, collision-bucket Vec + capacities, owned pattern/flag text, `Programs` Arc allocations, and matcher + lower bounds not already charged to an engine cache. It reports + `pinned_programs` and `opaque_program_bytes`. +- `regex.literal_sites`: the 1,024-slot literal-key Vec, including its full + `Option` capacity. Its Arc text allocations are shared with the + content table and are not charged twice. +- `regex.site_table`: the #9958 site-to-rooted-header map. `sites` and + `rooted_headers` are exact. `rooted_header_bytes` is reported but explicitly + outside `side_table_bytes`, because those 56-byte headers are already in the + GC live-heap census. `pinned_programs` and `pinned_program_bytes` are gross, + unique program-bundle retention measurements, including programs shared + with content/engine caches; that gross field is explicitly outside the + additive total to prevent double counting. `exclusively_attributed_programs` + and `attributed_program_bytes` are the de-duplicated subset owned nowhere + else and are inside this row's `bytes` and `side_table_bytes`. +- `regex.active_factory_sites`: the transient #9958 authorization-stack Vec + capacity and current entries (normally empty when a synchronous census + runs). +- `regex.expando_owners`: the RegExp owner share of the mixed exotic-expando + HashMap plus each RegExp-owned property Vec and key-string capacity; + `owners` and `properties` are included. +- `regex.matcher_kinds`: counts `unbuilt`, `standard`, `fancy`, and `repeat` + headers. `bytes=0` and `bytes_inside_side_table_bytes=false`, because the tag + resides in each already-counted `RegExpHeader` rather than a side table. + +The regex engines do not expose their complete compiled heap graphs. All +program rows therefore carry +`program_bytes_estimate="opaque_inline_lower_bound"`: Arc counters, the public +wrapper value, exposed source text, and exposed auxiliary buffers are counted; +unexposed automata/program allocations are not guessed. The program-count +fields remain exact and are the decisive signal if the RX2 delta is primarily +opaque engine storage. + +## Reconciliation + +`side_table_bytes` remains an explicit census estimate, not an allocator tag. +The assembly now computes: + +`side_table_bytes = non_regex_side_table_bytes + regex_side_table_bytes`. + +Legacy regex tuples from #9958 are removed from the ordinary row stream before +summing. The rich regex rows are serialized once, while +`regex_side_table_bytes` is built by an independent second diagnostic walk of +the same registered table inventory. Consequently the sum of every emitted +`regex.*` row's `bytes` must equal `regex_side_table_bytes`; omitting a row does +not silently shrink the attributed total. + +Gross `regex.site_table.rooted_header_bytes` and `pinned_program_bytes` are +labelled outside the additive total. The row's table bytes and +`attributed_program_bytes` are inside it. This preserves reconciliation while +still exposing the causal site-pinned working set side by side. + +## Tests and sabotage + +- `census_prints_regex_rows_that_reconcile_with_side_table_total` constructs + six distinct regex sources, executes them, evaluates one direct #9958 + literal site twice, serializes and parses the direct census document, and + asserts pointer entries >= 6, site/root/header/program counts >= 1, the full + row inventory, `sum(regex row bytes) == regex_side_table_bytes`, and + `side_table_bytes - non_regex_side_table_bytes == regex_side_table_bytes`. + Its sabotage assertion removes one non-zero row contribution and proves the + independent attribution total no longer reconciles. Deleting one row from + registration therefore fails the equality assertion. +- `census_regex_rows_are_zero_cost_when_not_requested` resets a cfg(test) + census-walk counter, constructs and matches a regex, proves the counter is + still zero, invokes the census directly, and proves it advances. Adding a + per-construction bookkeeping call makes its zero assertion fail. + +## Gates + +Every reported Cargo test/build invocation used `-j4` and the campaign build +lock, and launched only after `df -g /` reported at least 12 GiB. + +- `cargo test -p perry-runtime --release --lib -j4 -- --test-threads=1 census`: + final tree, 20 passed, 0 failed, 3,261 filtered out. +- `cargo test -p perry-runtime --release --lib -j4 -- --test-threads=1 regex`: + final tree, 130 passed, 0 failed, 3,151 filtered out. +- `cargo test -p perry-runtime --release --lib -j4 -- --test-threads=1`: + the pre-split implementation passed through Cargo (3,276 passed, 0 failed, + 4 ignored); the exact final Cargo-built executable was then run directly + after the source-only commit split and passed 3,277, 0 failed, 4 ignored. + A final no-op Cargo wrapper retry was prohibited because its guarded + precheck reported 10 GiB after another lane's build. +- `cargo build --release -p perry-runtime --features wasm-host -j4`: + the combined implementation before its source-only commit split passed in + 4m34s. A final-tree refresh was prohibited by the same 10 GiB precheck; the + final default-feature release test target compiled without warnings. +- Direct `rustfmt --edition 2021 --check` on every touched Rust file: passed. +- `git diff --check`: passed. +- `scripts/check_file_size.sh`: passed; no Rust source exceeds 2,000 lines. +- `python3 -m json.tool scripts/gc_runtime_root_holders.json`: passed. +- `python3 scripts/gc_runtime_root_holders.py`: passed: 1,372 declarations, + 596 scanner-reached, 357 classified, 414 frontier-pinned, 152 scanners. +- `python3 scripts/gc_runtime_root_holders.py --self-test`: passed: 90 planted + declarations and 357 inventory entries. + +One non-mutating `cargo fmt --all -- --check` attempt was mistakenly allowed +to continue after its chained precheck printed 8 GiB; it reported only the two +formatting changes then applied. The final formatting gate was rerun directly +with `rustfmt --check` on every touched Rust file and passed. No build/test was +started below the floor. + +The GC snapshot pin was re-audited because `gc/census.rs` and the module list +changed. `PASS1_MARKED` is still taken out of TLS before `take_census`; all new +regex walks occur afterward, and neither boundary nor intervening cycle flow +changed. No thread-local was added. + +One initial full-suite run under a PTY-like harness state had 3,275 passing and +one environment-dependent failure in +`tty::tests::columns_undefined_when_not_tty` (it read terminal width 80). The +exact test passed in a non-TTY process, and the subsequent non-TTY full rerun +passed 3,276 with 4 ignored. The final-tree result above supersedes that +diagnostic history. + +## Origin/main application check + +The implementation was split at the stack boundary. In a disposable checkout +of `origin/main` at `8b7dc3342b22fe6270739c8d51585c3d2cdfa618`, +`git cherry-pick --no-commit 49f551f05` completed with no conflicts. The +excluded `5e8079138` commit contains the #9958 rooted-site row plus the +#9918/#9958 content/literal cache-layout accessors and the branch-specific GC +snapshot pins. Thus the common regex-table census applies cleanly to main; +the site-table integration is intentionally the one omitted stack row. + +## PerryMaster request + +On `app-main7rx` and `app-main7` (the still-running RX2 arm and control), run +one 120-second idle interval followed by one SIGUSR2 heap census on each. Send +the complete `regex.*` rows and the three side-table totals side by side: +`side_table_bytes`, `regex_side_table_bytes`, and +`non_regex_side_table_bytes`. + +In particular compare `regex.site_table.sites`, `rooted_headers`, +`pinned_programs`, `pinned_program_bytes`, and `attributed_program_bytes`, plus +the 512-entry engine-cache program counts. The expected explanation for RX2's +88.3 MB versus 82.4 MB (+6 MB) is roughly 550 site-pinned programs beyond the +512-entry engine cache. If that population is present, the gross site-pinned +lower bound/count identifies it even when shared ownership assigns additive +bytes to another regex row. If it is not present, the side-by-side reconciled +rows identify which other regex table grew; if no regex row explains the +delta, that is the finding and the residual belongs outside regex attribution. diff --git a/cc-perf-campaign/codex/REPORT_regex_literal_site_test.md b/cc-perf-campaign/codex/REPORT_regex_literal_site_test.md new file mode 100644 index 0000000000..a600ec854a --- /dev/null +++ b/cc-perf-campaign/codex/REPORT_regex_literal_site_test.md @@ -0,0 +1,116 @@ +# Regex literal-site `.test` report + +## Branch and SHA + +- Branch: `perf/regex-literal-site-test` +- Base: `107d40adb9881ff5f91f94124e24907fcfea5796` +- Implementation commit: `2f799c7b0318b683bd0f320705ea6855f73fed85` +- Target remote: `fork/perf/regex-literal-site-test` + +## Map and mechanism + +- Ordinary regex literals still derive an identity from the address of a compiler-emitted private `i64` global, never from a hand-written constant (`crates/perry-codegen/src/expr/logical_collections.rs:61-79`). Ordinary lowering loads the interned pattern/flags handles and calls `js_regexp_new_site(pattern, flags, site)` (`logical_collections.rs:1382-1413`; runtime entry at `crates/perry-runtime/src/regex.rs:988-994`). +- A direct HIR `RegExpTest` whose receiver is exactly `Expr::RegExp` is the non-escaping shape: the literal node is consumed solely as this call's receiver, and only the boolean result is published. It lowers through `js_regexp_site_test_new`, captures `.test` before evaluating the argument, roots receiver and method across argument evaluation, then dispatches (`crates/perry-codegen/src/expr/instance_misc1.rs:1192-1226`). An escaping receiver such as `const r = /x/g; r.test(a); r.test(b)` remains on `js_regexp_new_site` plus the ordinary `js_regexp_test` route. +- `js_regexp_site_test_new` allocates and records one header on the cold evaluation, then reuses it on canonical hits (`crates/perry-runtime/src/regex/site_test.rs:157-200`). The site table owns a strong mutable raw root; its registered visitor marks and rewrites the header during evacuation (`site_test.rs:493-500`; registration at `crates/perry-runtime/src/gc/mod.rs:1003-1005`). The header therefore survives minors and cannot enter regex-death finalization while the site lives. +- Validation is performed on every evaluation. The realm's `RegExp.prototype`, canonical `test` closure, and own-slot index are recorded at intrinsic installation; the two heap values are representation-correct GC roots. The probe rejects a replaced/deleted/accessor `test` slot and any explicit receiver prototype (`crates/perry-runtime/src/object/regex_proto_thunks.rs:319-408`). A decline resolves the actual property and calls it generically (`site_test.rs:397-490`). Property Get remains before argument evaluation, so an argument that patches the prototype still invokes the method captured before that patch. +- Ordinary `js_regexp_test` implements stateful global/sticky `lastIndex` behavior (`crates/perry-runtime/src/regex.rs:1701 onward`). A fresh literal starts with zero on every evaluation, so the allocation-free dispatch resets the private cached header to zero immediately before its test (`site_test.rs:460-484`). The resulting write is unobservable: this exact receiver has no escaping reference, no `this` capture, and only the already-validated builtin sees it. +- The segment-view tier validates the same canonical prototype and calls `regexp_test_str_bounded` on a borrowed segment (`crates/perry-runtime/src/intl/segments_view.rs:350-394`; bounded matcher at `crates/perry-runtime/src/regex.rs:1661-1699`). It deliberately refuses global/sticky regexes and operates only after its receiver exists. Consequently it removed segment materialization but could not remove `g54.default()`'s per-grapheme RegExp construction; generic function-result dispatch is documented at `crates/perry-runtime/src/object/native_call_method/primitive_methods.rs:541-545`. +- For the real bundle shape, codegen recognizes `().test(arg)` and preserves the actual call and member lookup (`crates/perry-codegen/src/expr/calls.rs:77-101,879-950`). Functions/closures are eligible to claim an active caller site only when HIR proves zero parameters, non-async, non-generator, and exactly one `return ` statement (`crates/perry-codegen/src/codegen/function.rs:523-531`; `closure.rs:561-571`). The runtime resolves the actual callee's native entry on every call and pairs it with the site; only the exact factory identity may claim/reuse the header (`site_test.rs:202-252,321-395`). Reassignment, a rebound namespace member, a non-literal body, or a nested helper declines to the generic result path. Active identity frames have exception savepoints so caught throws cannot leave stale authorization. +- `[regex-diag]` now prints `site_test_no_alloc=` and `site_test_declined=(patched_prototype=...,callee_mismatch=...,non_literal=...)` in the whole line (`crates/perry-runtime/src/hot_diag.rs:196-203,374-424`). The no-allocation counter is incremented inside the construction entries that avoid the priced allocation. `new=` falls by the served count because a hit never enters `js_regexp_new_impl`. +- `PERRY_GC_CENSUS` now contains `regex.content_cache`, `regex.literal_sites`, and `regex.site_test_headers` (`crates/perry-runtime/src/gc/census.rs:565-604`; aggregation at `crates/perry-runtime/src/regex/site_test.rs:503-520`). + +## Named correctness and sabotage coverage + +- Codegen: `direct_literal_test_uses_the_site_header_and_post_get_dispatch`, `escaping_literal_is_not_transformed_and_keeps_one_stateful_receiver`, `direct_factory_call_records_function_identity_and_uses_the_caller_site`, and `namespace_member_factory_call_uses_the_member_wrapper` (`crates/perry-codegen/src/expr/regex_site_test_tests.rs:73-207`). +- Runtime: `direct_global_site_allocates_one_header_and_resets_last_index` compares a table with fresh generic `/x/g`; `direct_sticky_site_starts_each_evaluation_at_zero` checks `/x/y` anchoring and reset; `escaping_generic_global_header_carries_last_index_between_tests` proves the untransformed stateful case (`crates/perry-runtime/src/regex/site_test.rs:586-640`). +- Cross-function sabotage: `direct_factory_site_reuses_only_the_recorded_callee`, `nested_exact_factory_cannot_claim_a_different_callees_site`, and `namespace_member_factory_site_is_covered` cover direct `f()`, nested/non-literal decline, and namespace-member rebinding on the next call (`site_test.rs:699-813`). +- Prototype/rooting sabotage: `patched_regexp_prototype_test_declines_on_the_next_call`, `caught_throw_restores_an_orphaned_factory_site_frame`, and `site_header_root_is_rewritten_by_a_copying_minor` cover the next-call patch guard, exception cleanup, and copied-minor root rewriting (`site_test.rs:753-880`). The decline tests assert the individual reason buckets, not only the total. + +## Gates + +Completed after the final edits: + +- Direct `rustfmt` over touched Rust files. +- `git diff --check`. +- `scripts/check_file_size.sh`: passed; `regex.rs` is below the 2,000-line ceiling. +- `python3 -m json.tool scripts/gc_runtime_root_holders.json`: passed. +- `python3 scripts/gc_runtime_root_holders.py --self-test`: passed, 90 planted declarations and 347 inventory entries. +- `python3 scripts/gc_runtime_root_holders.py`: passed, 1,371 holders scanned, 594 reached by registered scanners, 358 classified, 414 frontier-pinned, 152 scanners. +- `python3 scripts/gc_rekeyed_key_tables.py`: passed, 42 rekey sites, 25 registered prunes, 0 gaps. + +Cargo history and disk stop: + +- `cargo test -j4 -p perry-codegen` through `measure_lock.sh --build` passed before the last guard-only codegen edit: 1,452 unit tests passed, 1 ignored, followed by all integration and doc tests passing. The final tree was not rerun. +- `cargo test -j4 -p perry-runtime --release --lib -- --test-threads=1` initially passed (3,265 passed, 4 ignored) before the exception/counter additions. The final-tree attempt compiled successfully and ran 3,266 passing tests plus one new synthetic nested-wrapper test failure; that fixture's trivial Rust wrapper had been release-folded with its callee. The fixture was made observably distinct with an atomic side effect and `inline(never)`, but was not rerun. +- Immediately after that invocation, `df -g /` reported 9 GB available. The binding rule prohibits every further Cargo invocation below 12 GB, so no wait or additional build was attempted. +- Not run on the final tree: the runtime lib gate rerun, the codegen gate rerun, `cargo build --release -p perry-runtime --features wasm-host`, and `cargo build --release -p perry`. +- A fresh archive and `nm` check were not produced because the archive build was prohibited. No local cc CPU/RSS measurement was run. + +## Predictions + +For the supplied I6d 3,300-character reply: + +- `new=`: 1,074,006 -> at most 10,000. +- `site_test_no_alloc=`: approximately 1,068,858 (one cold header means the exact value may be one lower for that site). +- `header_bytes`: 60,144,336 bytes -> approximately 0.3 MB. +- `ptr_ins` / `ptr_rm`: approximately zero at reply scale, apart from cold and unrelated regex objects. +- `test=`: unchanged at approximately 2,139,156. +- Turn CPU at 3,300 characters: -4% to -6%, from removing `js_regexp_new`, regex-death/finalization, and pointer-side-table work. Peak RSS should be lower; +1% to +10% remains acceptable under the campaign goal. + +## Exact perrymaster request + +This commit touches CODEGEN. Build the compiler from `2f799c7b0318b683bd0f320705ea6855f73fed85` on the I6d tree, or on the I7-view tree if all prerequisite picks apply, and perform a full cc bundle recompile; do not reuse the base bundle. From the resulting artefact, use `nm` to prove the new site-test runtime entry symbols are present and report the number of emitted call sites for `js_regexp_site_test_new`, `js_regexp_site_factory_call_value`, and `js_regexp_site_factory_call_method` (including the `g54.default().test(O)` site). + +Run one identical 3,300-character reply and provide the entire `[regex-diag]` line plus the per-pattern table. Confirm the 12,807-byte emoji `/.../g` row is constructed once, built once, and tested approximately 1,068,858 times; report `new`, `site_test_no_alloc`, the three decline buckets, `header_bytes`, `ptr_ins`, `ptr_rm`, `test`, `test_global`, and compile/cache counters. Expected: `new <= 10,000`, `site_test_no_alloc ~= 1,068,858`, `header_bytes ~= 0.3 MB`, `ptr_ins/ptr_rm ~= 0`, and unchanged `test`. + +Then run paired 5x3,300-character and 3x400-character comparisons against the base bundle with identical warmup, environment, inputs, and node-parity stop conditions. Report every turn's CPU and peak RSS. Expected 3,300-character turn CPU improvement is 4% to 6% and peak RSS is lower. Finally capture a perf draw and verify `js_regexp_new`, `regex_header_clear_dead_for_gc`, and the dead-owner regex path have disappeared from the top 25. + +## CI fixes 2026-09-07 + +### Fixed heads + +- #9918 `perf/regex-drop-source-table`: `dd1c5242d2ce87139d33436f347adb7245fe754d` (old head `ce9e12801e8d83fae471e06cc85429257ac10854`). +- #9958 fixed code head, before this final report-only commit: `54c9373c882fe7a2bb63cde806f563ce5799605d` (old head `abb0d907ff8dade12dfcf6b092cbd8f71bfc4233`). The final remote branch head is this report commit, whose hash is necessarily determined after the report contents are committed. + +### Triage items + +1. Formatting: direct `rustfmt` put the test modules in formatter order at `crates/perry-runtime/src/regex.rs:1760-1767`, moving `mod tests_part2;` after `tests_cache` and `tests_header`. Direct `rustfmt --check` passes. `cargo fmt --all --check` was not run: disk (8 GB available, below the binding 12 GB floor). +2. #9918 raw-handle debt: the two new bare reads in the nursery relocation fixture are now scoped `RuntimeHandle::with_const_ptr` stores at `crates/perry-runtime/src/regex.rs:390-395`; the two pre-existing production reads and the ceiling remain unchanged. `python3 scripts/raw_handle_debt.py` passes at 955 sites (baseline 963), and `--self-test` passes. +3. #9918 product warnings: the `Arc` import is feature-gated at `crates/perry-runtime/src/regex.rs:14-15`; `MatcherKind` carries a feature-off `dead_code` allow with the layout-only reason at `regex.rs:500-514`. The workflow's exact product command (`RUSTFLAGS='-D warnings' cargo check -p perry --bins`) was not run: disk. +4. #9918 all-target warnings: the unused `regex_has_repeat_program` import is gone at `crates/perry-runtime/src/regex/tests_part2.rs:5-7`, and the unnecessary `unsafe` block around the safe lazy-build/assertion calls is gone at `tests_part2.rs:600-607`. The workflow's host-compatible `cargo check --workspace --all-targets ...` command was not run: disk. +5. Main's benchmark-freshness, build-cache, and GC-ratchet reds were not touched. +6. #9958 root-holder custody/windows self-test: removed the three duplicate `REGEXP_PROTOTYPE_*_SLOT` entries that the stack re-added; the authoritative #9893 entries remain once each at `scripts/gc_runtime_root_holders.json:647-664`. `python3 scripts/gc_runtime_root_holders.py` passes (1,372 declarations, 596 scanner-reached, 357 inventory-classified, 414 frontier-pinned, 152 scanners), and `--self-test` passes (90 planted declarations, 357 inventory entries). +7. #9958 raw-handle debt: `canonical_rooted_header` now pairs the canonicality call with its post-call reload through `RuntimeHandle::across_mut` at `crates/perry-runtime/src/regex/site_test.rs:164-170`. No per-module ceiling was added; the same debt command and self-test in item 2 pass. +8. `async_hooks_constructors_expose_real_prototype_methods` is in `crates/perry/tests/issue_6764_async_hooks_prototype_metadata.rs`. Not run: disk (8 GB available). The hypothesis that duplicate inventory registration caused the runtime failure remains unverified locally; no codegen bisection or blind patch was performed. +9. Main's unrelated shard and GC reds were not touched. + +Other requested cargo gates were not run: disk: `cargo test -p perry-runtime --release --lib -j4 -- --test-threads=1 regex`, the full runtime lib gate, and the single compiled async-hooks test. `git diff --check`, JSON parsing, both Python audits, and both audit self-tests pass. + +### Range-diffs + +#9918, `git range-diff 616a2cb84..ce9e12801 616a2cb84..dd1c5242d`: + +```text +1: d8daa4fd4 = 1: d8daa4fd4 perf(regex): remove the traced-source side table +2: e2b0a9054 = 2: e2b0a9054 perf(regex): share one program-set handle per header +3: 7a44e5948 = 3: 7a44e5948 perf(regex): tag the selected matcher on each header +4: 6ea7ad9eb = 4: 6ea7ad9eb test(regex): isolate WTF-8 source from matcher parsing +5: c217a231c = 5: c217a231c refactor(regex): split header properties and tests +6: 883d334a6 = 6: 883d334a6 fix(regex): retain canonical flags through allocation +7: ce9e12801 = 7: ce9e12801 perf(regex): preserve live literal programs on eviction +-: --------- > 8: dd1c5242d fix(regex): clear branch-owned CI failures +``` + +All seven measured commits are byte-identical; only the new CI-fix commit is added. + +#9958, `git range-diff ce9e12801..abb0d907f dd1c5242d..54c9373c8`: + +```text +1: f5a2bdb7c = 1: 90e21317a perf(regex): reuse literal headers at test-only sites +2: abb0d907f ! 2: 47370d886 docs(perf): record regex literal-site handoff + The report commit no longer carries the inherited tests_part2 warning cleanup; + that exact hunk is now in #9918's dd1c5242d fix beneath the stack. +-: --------- > 3: 54c9373c8 fix(regex): deduplicate CI custody records +``` + +The measured #9958 implementation commit is patch-identical. The only movement in the report commit is the listed inherited warning cleanup moving to the fixed base; the only new code hunk is the item 6/7 CI-fix commit. diff --git a/changelog.d/9918-regex-cache-eviction.md b/changelog.d/9918-regex-cache-eviction.md new file mode 100644 index 0000000000..29fd1168f7 --- /dev/null +++ b/changelog.d/9918-regex-cache-eviction.md @@ -0,0 +1 @@ +Keep compiled programs for recorded regular-expression literal sites across bounded cache eviction, and replace whole-cache overflow clears with one-entry eviction. diff --git a/crates/perry-codegen/src/codegen/closure.rs b/crates/perry-codegen/src/codegen/closure.rs index e530881f23..1facefbe65 100644 --- a/crates/perry-codegen/src/codegen/closure.rs +++ b/crates/perry-codegen/src/codegen/closure.rs @@ -515,6 +515,7 @@ pub(super) fn compile_closure( captures_new_target, enclosing_class, is_async, + is_generator, is_strict, ) = match closure_expr { perry_hir::Expr::Closure { @@ -525,6 +526,7 @@ pub(super) fn compile_closure( captures_new_target, enclosing_class, is_async, + is_generator, is_strict, .. } => ( @@ -535,6 +537,7 @@ pub(super) fn compile_closure( *captures_new_target, enclosing_class.clone(), *is_async, + *is_generator, *is_strict, ), _ => return Err(anyhow!("compile_closure: expected Expr::Closure")), @@ -556,6 +559,16 @@ pub(super) fn compile_closure( closure_relevant_ids.extend(captures.iter().copied()); let public_llvm_name = format!("perry_closure_{}__{}", module_prefix, func_id); + let regex_factory_identity = (!is_async + && !is_generator + && params.is_empty() + && matches!( + body.as_slice(), + [perry_hir::Stmt::Return(Some( + perry_hir::Expr::RegExp { .. } + ))] + )) + .then(|| public_llvm_name.clone()); let typed_public_trampoline = if cross_module.typed_f64_closures.contains(&func_id) { Some(TypedFunctionTrampolineKind::F64) } else if cross_module.typed_i32_closures.contains(&func_id) { @@ -1053,6 +1066,7 @@ pub(super) fn compile_closure( module_slug: crate::expr::native_region_slug(strings.module_prefix()), source_function: format!("closure_{}", func_id), source_function_slug: crate::expr::native_region_slug(&format!("closure_{}", func_id)), + regex_factory_identity, active_region_id: None, native_facts: &native_facts, locals, diff --git a/crates/perry-codegen/src/codegen/entry.rs b/crates/perry-codegen/src/codegen/entry.rs index 4564eb9046..5d2f5cc498 100644 --- a/crates/perry-codegen/src/codegen/entry.rs +++ b/crates/perry-codegen/src/codegen/entry.rs @@ -850,6 +850,7 @@ pub(super) fn compile_module_entry( module_slug: crate::expr::native_region_slug(strings.module_prefix()), source_function: "module_init".to_string(), source_function_slug: crate::expr::native_region_slug("module_init"), + regex_factory_identity: None, active_region_id: None, native_facts: &main_native_facts, locals: HashMap::new(), @@ -1640,6 +1641,7 @@ pub(super) fn compile_module_entry( module_slug: crate::expr::native_region_slug(strings.module_prefix()), source_function: "module_init".to_string(), source_function_slug: crate::expr::native_region_slug("module_init"), + regex_factory_identity: None, active_region_id: None, native_facts: &init_native_facts, locals: HashMap::new(), diff --git a/crates/perry-codegen/src/codegen/function.rs b/crates/perry-codegen/src/codegen/function.rs index 21eb811fb3..0289c4b085 100644 --- a/crates/perry-codegen/src/codegen/function.rs +++ b/crates/perry-codegen/src/codegen/function.rs @@ -5,7 +5,7 @@ use std::collections::{HashMap, HashSet}; use anyhow::{anyhow, Context, Result}; -use perry_hir::Function; +use perry_hir::{Expr, Function, Stmt}; use crate::expr::FnCtx; use crate::module::LlModule; @@ -524,6 +524,11 @@ pub(super) fn compile_function( .get(&f.id) .cloned() .ok_or_else(|| anyhow!("function name not resolved for {}", f.name))?; + let regex_factory_identity = (!f.is_async + && !f.is_generator + && f.params.is_empty() + && matches!(f.body.as_slice(), [Stmt::Return(Some(Expr::RegExp { .. }))])) + .then(|| public_llvm_name.clone()); let guarded_public_plan = if typed_public_trampoline.is_none() && spec_entry.is_none() { cross_module .spec_abi_functions @@ -1023,6 +1028,7 @@ pub(super) fn compile_function( module_slug: crate::expr::native_region_slug(strings.module_prefix()), source_function: f.name.clone(), source_function_slug: crate::expr::native_region_slug(&f.name), + regex_factory_identity, active_region_id: None, native_facts: &native_facts, locals, diff --git a/crates/perry-codegen/src/codegen/method.rs b/crates/perry-codegen/src/codegen/method.rs index 4c1a1f061e..c562c1de88 100644 --- a/crates/perry-codegen/src/codegen/method.rs +++ b/crates/perry-codegen/src/codegen/method.rs @@ -469,6 +469,7 @@ pub(super) fn compile_method( "{}.{}", class.name, method.name )), + regex_factory_identity: None, active_region_id: None, native_facts: &native_facts, locals, @@ -1609,6 +1610,7 @@ pub(super) fn compile_static_method( "{}.{}", class.name, f.name )), + regex_factory_identity: None, active_region_id: None, native_facts: &native_facts, locals, diff --git a/crates/perry-codegen/src/expr/calls.rs b/crates/perry-codegen/src/expr/calls.rs index f110590367..d2975c8799 100644 --- a/crates/perry-codegen/src/expr/calls.rs +++ b/crates/perry-codegen/src/expr/calls.rs @@ -13,7 +13,8 @@ use perry_hir::Expr; use crate::lower_call::{lower_call, lower_native_method_call}; use crate::nanbox::double_literal; -use crate::types::DOUBLE; +use crate::rooting; +use crate::types::{DOUBLE, I64}; use super::{ emit_string_literal_global, lower_expr, nanbox_pointer_inline, nanbox_string_inline, @@ -73,6 +74,32 @@ pub(crate) fn lower(ctx: &mut FnCtx<'_>, expr: &Expr) -> Result { args, ), + // `().test(arg)`, including the bundled namespace form + // `ns.default().test(arg)`. The inner call still executes normally; + // a structurally proven zero-argument regex factory consumes the + // active site and may return its rooted header. Any reassignment or + // non-literal body therefore reaches the unchanged generic method + // path, rather than trusting a source-level binding assumption. + Expr::Call { callee, args, .. } + if args.len() == 1 + && matches!( + callee.as_ref(), + Expr::PropertyGet { object, property, .. } + if property == "test" + && matches!( + object.as_ref(), + Expr::Call { callee, args, .. } + if args.is_empty() + // A computed/`with` reference carries + // receiver-binding semantics that the + // site wrapper does not model. + && !matches!(callee.as_ref(), Expr::IndexGet { .. } | Expr::WithGet { .. }) + ) + ) => + { + arm_regexp_factory_site_test(ctx, callee.as_ref(), &args[0]) + } + // #1645: `ReadableStream.from(iterable)` (Node 20+). The HIR lowers // `(ReadableStream as any).from(x)` to a Call whose callee is // `PropertyGet { ExternFuncRef("ReadableStream"), "from" }`; route it to @@ -848,3 +875,76 @@ pub(crate) fn lower(ctx: &mut FnCtx<'_>, expr: &Expr) -> Result { _ => unreachable!("expr/mod.rs dispatched a variant not handled by this submodule"), } } + +fn arm_regexp_factory_site_test( + ctx: &mut FnCtx<'_>, + outer_callee: &Expr, + argument: &Expr, +) -> Result { + let Expr::PropertyGet { object, .. } = outer_callee else { + unreachable!("guarded by the caller") + }; + let Expr::Call { + callee: inner_callee, + args: inner_args, + .. + } = object.as_ref() + else { + unreachable!("guarded by the caller") + }; + debug_assert!(inner_args.is_empty()); + + let slot_ref = super::logical_collections::emit_regexp_site_key(ctx); + let site_key = ctx.block().ptrtoint(&slot_ref, I64); + let receiver = match inner_callee.as_ref() { + Expr::PropertyGet { + object, property, .. + } => { + let object = lower_expr(ctx, object)?; + let key_idx = ctx.strings.intern(property); + let entry = ctx.strings.entry(key_idx); + let key_global = format!("@{}", entry.handle_global); + let key = ctx.block().load(DOUBLE, &key_global); + ctx.block().call( + DOUBLE, + "js_regexp_site_factory_call_method", + &[(I64, &site_key), (DOUBLE, &object), (DOUBLE, &key)], + ) + } + callee => { + let callee = lower_expr(ctx, callee)?; + ctx.block().call( + DOUBLE, + "js_regexp_site_factory_call_value", + &[(I64, &site_key), (DOUBLE, &callee)], + ) + } + }; + + // Property Get for `.test` precedes argument evaluation in ECMAScript. + // A cached/canonical receiver records the builtin as an internal marker; + // a decline resolves the actual property now, so a getter or a patch has + // exactly the generic ordering. + let method = ctx.block().call( + DOUBLE, + "js_regexp_site_test_get_method", + &[(I64, &site_key), (DOUBLE, &receiver)], + ); + rooting::with_rooted_group(ctx, 2, |ctx, roots| { + let receiver = roots.adopt_emitted(ctx, rooting::Repr::Boxed, &receiver, true); + let method = roots.adopt_emitted(ctx, rooting::Repr::Boxed, &method, true); + let argument = lower_expr(ctx, argument)?; + let receiver = roots.reread_emitted(ctx, receiver); + let method = roots.reread_emitted(ctx, method); + Ok(ctx.block().call( + DOUBLE, + "js_regexp_site_test_dispatch", + &[ + (I64, &site_key), + (DOUBLE, &receiver), + (DOUBLE, &method), + (DOUBLE, &argument), + ], + )) + }) +} diff --git a/crates/perry-codegen/src/expr/instance_misc1.rs b/crates/perry-codegen/src/expr/instance_misc1.rs index 9f395cfc5e..d212aed233 100644 --- a/crates/perry-codegen/src/expr/instance_misc1.rs +++ b/crates/perry-codegen/src/expr/instance_misc1.rs @@ -1190,6 +1190,41 @@ pub(crate) fn lower(ctx: &mut FnCtx<'_>, expr: &Expr) -> Result { // Receiver is a NaN-tagged i64 RegExpHeader pointer; arg is // a NaN-tagged string. Both must be unboxed before the call. Expr::RegExpTest { regex, string } => { + // A literal used directly as this one receiver cannot escape: the + // HIR node owns the literal expression and publishes only the + // call result. Construct (or fetch) the site's rooted header + // before evaluating the argument, resolving `.test` at the same + // pre-argument point as an ordinary call. This ordering matters + // for `/x/.test(patchPrototype())`: it invokes the method value + // captured before the patch. + if let Expr::RegExp { pattern, flags } = regex.as_ref() { + let (receiver, site_key) = + super::logical_collections::lower_regexp_site_test_receiver( + ctx, pattern, flags, + ); + let method = ctx.block().call( + DOUBLE, + "js_regexp_site_test_get_method", + &[(I64, &site_key), (DOUBLE, &receiver)], + ); + return rooting::with_rooted_group(ctx, 2, |ctx, roots| { + let receiver = roots.adopt_emitted(ctx, rooting::Repr::Boxed, &receiver, true); + let method = roots.adopt_emitted(ctx, rooting::Repr::Boxed, &method, true); + let argument = lower_expr(ctx, string)?; + let receiver = roots.reread_emitted(ctx, receiver); + let method = roots.reread_emitted(ctx, method); + Ok(ctx.block().call( + DOUBLE, + "js_regexp_site_test_dispatch", + &[ + (I64, &site_key), + (DOUBLE, &receiver), + (DOUBLE, &method), + (DOUBLE, &argument), + ], + )) + }); + } // #7154: the receiver is live across BOTH the string operand's own // lowering and the `js_jsvalue_to_string_coerce` below it, and the // coerce is unconditional — it allocates, and on an object argument diff --git a/crates/perry-codegen/src/expr/logical_collections.rs b/crates/perry-codegen/src/expr/logical_collections.rs index 68b3be1a6e..27d46ae0c9 100644 --- a/crates/perry-codegen/src/expr/logical_collections.rs +++ b/crates/perry-codegen/src/expr/logical_collections.rs @@ -58,6 +58,58 @@ use super::{ record_collection_string_key_selected, unbox_str_handle, unbox_to_i64, FnCtx, }; +/// Emit one immortal identity slot for a regex optimization site. +/// +/// The value stored in the slot is irrelevant; only its linker-stable address +/// is used. Keeping this in one helper prevents the allocation-free `.test` +/// paths from inventing a second site-key scheme or hand-writing an ABI +/// constant that can drift from ordinary literal lowering. +pub(crate) fn emit_regexp_site_key(ctx: &mut FnCtx<'_>) -> String { + let site_id = ctx.ic_site_counter; + ctx.ic_site_counter += 1; + let prefix = ctx.strings.module_prefix(); + let slot_name = if prefix.is_empty() { + format!("perry_regexp_site_{site_id}") + } else { + format!("perry_regexp_site_{prefix}__{site_id}") + }; + ctx.typed_parse_rodata + .push(format!("@{slot_name} = private global i64 0")); + format!("@{slot_name}") +} + +/// Construct the receiver for the exact non-escaping `/literal/.test(arg)` +/// shape. The returned site key is also consumed by the post-argument +/// dispatch, which revalidates the builtin before it exposes the cached +/// receiver as `this`. +pub(crate) fn lower_regexp_site_test_receiver( + ctx: &mut FnCtx<'_>, + pattern: &str, + flags: &str, +) -> (String, String) { + let pattern_idx = ctx.strings.intern(pattern); + let flags_idx = ctx.strings.intern(flags); + let pattern_global = format!("@{}", ctx.strings.entry(pattern_idx).handle_global); + let flags_global = format!("@{}", ctx.strings.entry(flags_idx).handle_global); + let slot_ref = emit_regexp_site_key(ctx); + let blk = ctx.block(); + let pattern_box = blk.load(DOUBLE, &pattern_global); + let flags_box = blk.load(DOUBLE, &flags_global); + let pattern_handle = unbox_to_i64(blk, &pattern_box); + let flags_handle = unbox_to_i64(blk, &flags_box); + let site_key = blk.ptrtoint(&slot_ref, I64); + let result = blk.call( + I64, + "js_regexp_site_test_new", + &[ + (I64, &pattern_handle), + (I64, &flags_handle), + (I64, &site_key), + ], + ); + (nanbox_pointer_inline(blk, &result), site_key) +} + fn is_static_string_key_map(ctx: &FnCtx<'_>, map: &Expr) -> bool { matches!( map_static_type_args(ctx, map), @@ -1327,34 +1379,37 @@ pub(crate) fn lower(ctx: &mut FnCtx<'_>, expr: &Expr) -> Result { // and unenforced: a future early return that drops the artifacts // breaks this site, loudly, at the in-process LLVM parse (`use of // undefined value`) rather than at runtime. - let site_id = ctx.ic_site_counter; - ctx.ic_site_counter += 1; - let slot_name = { - let prefix = ctx.strings.module_prefix(); - if prefix.is_empty() { - format!("perry_regexp_site_{site_id}") - } else { - format!("perry_regexp_site_{prefix}__{site_id}") - } - }; - ctx.typed_parse_rodata - .push(format!("@{slot_name} = private global i64 0")); - let slot_ref = format!("@{slot_name}"); + let slot_ref = emit_regexp_site_key(ctx); + let factory_identity = ctx.regex_factory_identity.clone(); let blk = ctx.block(); let pattern_box = blk.load(DOUBLE, &pattern_global); let flags_box = blk.load(DOUBLE, &flags_global); let pattern_handle = unbox_to_i64(blk, &pattern_box); let flags_handle = unbox_to_i64(blk, &flags_box); let site_key = blk.ptrtoint(&slot_ref, I64); - let result = blk.call( - I64, - "js_regexp_new_site", - &[ - (I64, &pattern_handle), - (I64, &flags_handle), - (I64, &site_key), - ], - ); + let result = if let Some(identity) = factory_identity { + let identity = blk.ptrtoint(&format!("@{identity}"), I64); + blk.call( + I64, + "js_regexp_new_factory_site", + &[ + (I64, &pattern_handle), + (I64, &flags_handle), + (I64, &site_key), + (I64, &identity), + ], + ) + } else { + blk.call( + I64, + "js_regexp_new_site", + &[ + (I64, &pattern_handle), + (I64, &flags_handle), + (I64, &site_key), + ], + ) + }; Ok(nanbox_pointer_inline(blk, &result)) } diff --git a/crates/perry-codegen/src/expr/mod.rs b/crates/perry-codegen/src/expr/mod.rs index a360d47452..60b33c7168 100644 --- a/crates/perry-codegen/src/expr/mod.rs +++ b/crates/perry-codegen/src/expr/mod.rs @@ -179,6 +179,8 @@ mod call_spread_short_tests; mod issue7628_rooting_tests; #[cfg(test)] mod readonly_collection_tests; +#[cfg(test)] +mod regex_site_test_tests; pub(crate) mod shadow_slot; #[cfg(test)] mod slice7_rooting_tests; @@ -265,6 +267,13 @@ pub(crate) struct FnCtx<'a> { /// module code uses `module_init`. pub source_function: String, pub source_function_slug: String, + /// Public callable symbol when this body is proven to be exactly + /// `function () { return /literal/flags; }`. The proof is structural at + /// the HIR function boundary (zero parameters, one return statement, no + /// async/generator machinery). Regex literal lowering passes this + /// identity to the runtime only for that shape; ordinary literals retain + /// fresh-object semantics. + pub regex_factory_identity: Option, /// Stable id for the labeled loop currently being lowered. pub active_region_id: Option, /// Full native-region fact graph collected for this lowered HIR region. diff --git a/crates/perry-codegen/src/expr/regex_site_test_tests.rs b/crates/perry-codegen/src/expr/regex_site_test_tests.rs new file mode 100644 index 0000000000..02817ae442 --- /dev/null +++ b/crates/perry-codegen/src/expr/regex_site_test_tests.rs @@ -0,0 +1,206 @@ +//! Allocation-free regex `.test` site lowering. These are IR-shape tests so +//! deleting a specialization while leaving the runtime helpers behind fails. + +use perry_hir::types::Type; +use perry_hir::{Expr, Function, Module, ModuleInitKind, Param, Stmt}; + +fn function( + id: u32, + name: &str, + params: Vec, + body: Vec, + return_type: Type, +) -> Function { + Function { + id, + name: name.to_string(), + type_params: Vec::new(), + params, + return_type, + body, + is_async: false, + is_generator: false, + is_strict: false, + is_exported: false, + captures: Vec::new(), + decorators: Vec::new(), + was_plain_async: false, + was_unrolled: false, + } +} + +fn param(id: u32, name: &str) -> Param { + Param { + id, + name: name.to_string(), + ty: Type::String, + default: None, + decorators: Vec::new(), + is_rest: false, + arguments_object: None, + } +} + +fn call(callee: Expr, args: Vec) -> Expr { + Expr::Call { + callee: Box::new(callee), + args, + type_args: Vec::new(), + byte_offset: 0, + } +} + +fn property(object: Expr, property: &str) -> Expr { + Expr::PropertyGet { + object: Box::new(object), + property: property.to_string(), + byte_offset: 0, + } +} + +fn compile(functions: Vec) -> String { + let mut module = Module::new("regex_site_test.ts"); + module.functions = functions; + module.init_kind = ModuleInitKind::Eager; + String::from_utf8( + crate::compile_module(&module, super::class_field_barrier_tests::ir_opts()) + .expect("regex site fixture compiles"), + ) + .expect("LLVM IR is UTF-8") +} + +#[test] +fn direct_literal_test_uses_the_site_header_and_post_get_dispatch() { + let body = vec![Stmt::Return(Some(Expr::RegExpTest { + regex: Box::new(Expr::RegExp { + pattern: "x".to_string(), + flags: "g".to_string(), + }), + string: Box::new(Expr::LocalGet(10)), + }))]; + let ir = compile(vec![function( + 1, + "direct", + vec![param(10, "s")], + body, + Type::Boolean, + )]); + assert!(ir.contains("call i64 @js_regexp_site_test_new("), "{ir}"); + assert!( + ir.contains("call double @js_regexp_site_test_get_method("), + "{ir}" + ); + assert!( + ir.contains("call double @js_regexp_site_test_dispatch("), + "{ir}" + ); +} + +#[test] +fn escaping_literal_is_not_transformed_and_keeps_one_stateful_receiver() { + let body = vec![ + Stmt::Let { + id: 20, + name: "r".to_string(), + ty: Type::Named("RegExp".to_string()), + mutable: false, + init: Some(Expr::RegExp { + pattern: "x".to_string(), + flags: "g".to_string(), + }), + }, + Stmt::Expr(Expr::RegExpTest { + regex: Box::new(Expr::LocalGet(20)), + string: Box::new(Expr::LocalGet(21)), + }), + Stmt::Return(Some(Expr::RegExpTest { + regex: Box::new(Expr::LocalGet(20)), + string: Box::new(Expr::LocalGet(22)), + })), + ]; + let ir = compile(vec![function( + 1, + "escaping", + vec![param(21, "a"), param(22, "b")], + body, + Type::Boolean, + )]); + // The fixture can be emitted in more than one specialized clone. Every + // clone must retain one ordinary construction and two stateful tests. + let constructions = ir.matches("call i64 @js_regexp_new_site(").count(); + assert!(constructions >= 1, "{ir}"); + assert_eq!( + ir.matches("call i64 @js_regexp_site_test_new(").count(), + 0, + "{ir}" + ); + assert_eq!( + ir.matches("call i32 @js_regexp_test(").count(), + constructions * 2, + "{ir}" + ); +} + +fn exact_factory() -> Function { + function( + 1, + "factory", + Vec::new(), + vec![Stmt::Return(Some(Expr::RegExp { + pattern: "x".to_string(), + flags: "g".to_string(), + }))], + Type::Named("RegExp".to_string()), + ) +} + +#[test] +fn direct_factory_call_records_function_identity_and_uses_the_caller_site() { + let inner = call(Expr::FuncRef(1), Vec::new()); + let outer = call(property(inner, "test"), vec![Expr::LocalGet(30)]); + let caller = function( + 2, + "caller", + vec![param(30, "s")], + vec![Stmt::Return(Some(outer))], + Type::Boolean, + ); + let ir = compile(vec![exact_factory(), caller]); + assert!(ir.contains("call i64 @js_regexp_new_factory_site("), "{ir}"); + assert!( + ir.contains("ptrtoint ptr @perry_fn_"), + "factory identity missing: {ir}" + ); + assert!( + ir.contains("call double @js_regexp_site_factory_call_value("), + "{ir}" + ); + assert!( + ir.contains("call double @js_regexp_site_test_dispatch("), + "{ir}" + ); +} + +#[test] +fn namespace_member_factory_call_uses_the_member_wrapper() { + // The runtime wrapper resolves `default` first, then activates the site + // only while invoking the resolved function. `Undefined` is sufficient + // for an IR-shape fixture; runtime tests exercise a real namespace object. + let inner = call(property(Expr::Undefined, "default"), Vec::new()); + let outer = call(property(inner, "test"), vec![Expr::String("x".to_string())]); + let ir = compile(vec![function( + 1, + "member", + Vec::new(), + vec![Stmt::Return(Some(outer))], + Type::Any, + )]); + assert!( + ir.contains("call double @js_regexp_site_factory_call_method("), + "{ir}" + ); + assert!( + ir.contains("call double @js_regexp_site_test_dispatch("), + "{ir}" + ); +} diff --git a/crates/perry-codegen/src/runtime_decls/mod.rs b/crates/perry-codegen/src/runtime_decls/mod.rs index fab34b46eb..99973fc3e1 100644 --- a/crates/perry-codegen/src/runtime_decls/mod.rs +++ b/crates/perry-codegen/src/runtime_decls/mod.rs @@ -264,5 +264,42 @@ mod tests { plain.starts_with("declare i64 @js_regexp_new(i64, i64)"), "got: {plain}" ); + + for (name, signature) in [ + ( + "js_regexp_site_test_new", + "declare i64 @js_regexp_site_test_new(i64, i64, i64)", + ), + ( + "js_regexp_new_factory_site", + "declare i64 @js_regexp_new_factory_site(i64, i64, i64, i64)", + ), + ( + "js_regexp_site_factory_call_value", + "declare double @js_regexp_site_factory_call_value(i64, double)", + ), + ( + "js_regexp_site_factory_call_method", + "declare double @js_regexp_site_factory_call_method(i64, double, double)", + ), + ( + "js_regexp_site_test_get_method", + "declare double @js_regexp_site_test_get_method(i64, double)", + ), + ( + "js_regexp_site_test_dispatch", + "declare double @js_regexp_site_test_dispatch(i64, double, double, double)", + ), + ] { + let line = module + .declaration_lines() + .find(|(candidate, _)| *candidate == name) + .map(|(_, line)| line) + .unwrap_or_else(|| panic!("missing declaration for {name}")); + assert!( + line.starts_with(signature), + "wrong declaration for {name}: {line}" + ); + } } } diff --git a/crates/perry-codegen/src/runtime_decls/strings.rs b/crates/perry-codegen/src/runtime_decls/strings.rs index 911e318a47..2e99cd4db1 100644 --- a/crates/perry-codegen/src/runtime_decls/strings.rs +++ b/crates/perry-codegen/src/runtime_decls/strings.rs @@ -1319,6 +1319,20 @@ pub fn declare_phase_b_strings(module: &mut LlModule) { // unit tests. `runtime_decls::tests` asserts the name AND the arity: a // wrong arity parses and miscompiles. module.declare_function("js_regexp_new_site", I64, &[I64, I64, I64]); + module.declare_function("js_regexp_new_factory_site", I64, &[I64, I64, I64, I64]); + module.declare_function("js_regexp_site_test_new", I64, &[I64, I64, I64]); + module.declare_function("js_regexp_site_factory_call_value", DOUBLE, &[I64, DOUBLE]); + module.declare_function( + "js_regexp_site_factory_call_method", + DOUBLE, + &[I64, DOUBLE, DOUBLE], + ); + module.declare_function("js_regexp_site_test_get_method", DOUBLE, &[I64, DOUBLE]); + module.declare_function( + "js_regexp_site_test_dispatch", + DOUBLE, + &[I64, DOUBLE, DOUBLE, DOUBLE], + ); // Full ECMAScript RegExp constructor: NaN-boxed pattern + flags in, handles // RegExp/undefined/object patterns and ToString-coerced flags. module.declare_function("js_regexp_construct", I64, &[DOUBLE, DOUBLE]); diff --git a/crates/perry-runtime/src/exception.rs b/crates/perry-runtime/src/exception.rs index a617be1909..63298fd0a2 100644 --- a/crates/perry-runtime/src/exception.rs +++ b/crates/perry-runtime/src/exception.rs @@ -137,6 +137,11 @@ struct ExceptionState { /// evaluating the right-hand side of a guarded private write skips the /// normal consumer, so catch entry must discard the orphaned hint. private_member_access_hint_depths: Box<[usize]>, + /// Active allocation-free regex-factory sites at handler entry. A + /// non-literal replacement callee can throw before the wrapper's normal + /// pop, so catch entry discards the orphaned identity frame. + #[cfg(feature = "regex-engine")] + regex_factory_site_depths: Box<[usize]>, /// #6559: dyn-eval interpreter state (rooted-stack length + interpreter /// call depth, packed) captured when each `try` was pushed. A throw /// `longjmp`s past interpreter Rust frames without running their @@ -168,6 +173,8 @@ impl ExceptionState { private_lexical_brand_depths: vec![0usize; MAX_TRY_DEPTH].into_boxed_slice(), derived_super_binding_depths: vec![0usize; MAX_TRY_DEPTH].into_boxed_slice(), private_member_access_hint_depths: vec![0usize; MAX_TRY_DEPTH].into_boxed_slice(), + #[cfg(feature = "regex-engine")] + regex_factory_site_depths: vec![0usize; MAX_TRY_DEPTH].into_boxed_slice(), #[cfg(feature = "dyn-eval")] dyn_eval_savepoints: vec![0u64; MAX_TRY_DEPTH].into_boxed_slice(), try_depth: 0, @@ -244,6 +251,11 @@ fn try_push_with_kind(kind: HandlerKind) -> *mut i32 { crate::object::derived_super_binding_stack_savepoint(); (*s).private_member_access_hint_depths[depth] = crate::object::private_member_access_hints_savepoint(); + #[cfg(feature = "regex-engine")] + { + (*s).regex_factory_site_depths[depth] = + crate::regex::site_test::active_factory_stack_savepoint(); + } // #6559: capture the dyn-eval interpreter's rooted-stack length + // call depth, so a caught throw restores interpreter state exactly // like the shadow stack. @@ -479,6 +491,10 @@ pub extern "C-unwind" fn js_throw(value: f64) -> ! { crate::object::private_member_access_hints_restore( (*s).private_member_access_hint_depths[depth], ); + #[cfg(feature = "regex-engine")] + crate::regex::site_test::active_factory_stack_restore( + (*s).regex_factory_site_depths[depth], + ); // #6559: restore the dyn-eval interpreter's rooted stack + call depth // (interpreter Rust frames unwound by this longjmp never run their // truncate/decrement epilogues). @@ -845,6 +861,10 @@ pub(crate) fn test_unwind_innermost_shadow_restore() { crate::object::private_member_access_hints_restore( (*s).private_member_access_hint_depths[depth], ); + #[cfg(feature = "regex-engine")] + crate::regex::site_test::active_factory_stack_restore( + (*s).regex_factory_site_depths[depth], + ); }); } diff --git a/crates/perry-runtime/src/gc/census.rs b/crates/perry-runtime/src/gc/census.rs index 743c93fb69..8b7d53b679 100644 --- a/crates/perry-runtime/src/gc/census.rs +++ b/crates/perry-runtime/src/gc/census.rs @@ -557,7 +557,7 @@ pub(crate) fn vec_bytes(v: &Vec) -> usize { v.capacity() * std::mem::size_of::() } -fn side_tables() -> Vec { +pub(super) fn side_tables() -> Vec { let mut rows: Vec = Vec::new(); rows.extend(crate::builtins::function_registries_census()); rows.extend(crate::closure::closure_registry_census()); @@ -575,6 +575,8 @@ fn side_tables() -> Vec { rows.extend(crate::module_require::path_registry_census()); rows.extend(crate::timer::timer_tables_census()); rows.push(crate::symbol::symbol_registry_census()); + #[cfg(feature = "regex-engine")] + rows.extend(crate::regex::site_test::side_table_census()); let (masks, typed) = super::layout_tables::per_object_layout_table_sizes(); rows.push(("gc.layout_slot_masks", masks, masks * 24)); rows.push(("gc.typed_layouts", typed, typed * 24)); @@ -586,6 +588,23 @@ fn side_tables() -> Vec { rows } +#[cfg(test)] +mod regex_census_tests { + #[test] + fn regex_side_tables_are_registered_with_the_census_prefix() { + let names: Vec<_> = super::side_tables() + .into_iter() + .filter_map(|(name, _, _)| name.starts_with("regex.").then_some(name)) + .collect(); + assert!(names.contains(&"regex.content_cache"), "rows: {names:?}"); + assert!(names.contains(&"regex.literal_sites"), "rows: {names:?}"); + assert!( + names.contains(&"regex.site_test_headers"), + "rows: {names:?}" + ); + } +} + // --------------------------------------------------------------------------- // Process-level numbers // --------------------------------------------------------------------------- @@ -792,14 +811,15 @@ fn take_census(label: &str, pass1: Option>) { side_rows.extend(crate::object::shapes::shape_table_liveness_census( &c.live_shape_ids, )); - let side: Vec = side_rows - .into_iter() - .map(|(n, e, b)| serde_json::json!({"table": n, "entries": e, "bytes": b})) - .collect(); - let side_total: usize = side - .iter() - .map(|r| r["bytes"].as_u64().unwrap_or(0) as usize) - .sum(); + let side_snapshot = super::regex_census::side_table_document_from(side_rows); + let side = side_snapshot["rows"].clone(); + let side_total = side_snapshot["side_table_bytes"].as_u64().unwrap_or(0) as usize; + let regex_side_total = side_snapshot["regex_side_table_bytes"] + .as_u64() + .unwrap_or(0) as usize; + let non_regex_side_total = side_snapshot["non_regex_side_table_bytes"] + .as_u64() + .unwrap_or(0) as usize; let live_total: u64 = c.space_live.iter().map(|a| a.bytes).sum(); let dead_total: u64 = c.space_dead.iter().map(|a| a.bytes).sum(); @@ -838,6 +858,8 @@ fn take_census(label: &str, pass1: Option>) { "live_bytes": live_total, "dead_bytes": dead_total, "side_table_bytes": side_total, + "regex_side_table_bytes": regex_side_total, + "non_regex_side_table_bytes": non_regex_side_total, "live_objects": c.space_live.iter().map(|a| a.count).sum::(), "dead_objects": c.space_dead.iter().map(|a| a.count).sum::(), "late_marked_bytes": late_total, diff --git a/crates/perry-runtime/src/gc/mod.rs b/crates/perry-runtime/src/gc/mod.rs index 64409430cb..7475fb04c2 100644 --- a/crates/perry-runtime/src/gc/mod.rs +++ b/crates/perry-runtime/src/gc/mod.rs @@ -256,6 +256,7 @@ pub(crate) mod census; #[cfg(feature = "diagnostics")] mod heap_snapshot; mod heap_stats; +mod regex_census; pub use census::{census_poll_signal, gc_census_enabled}; #[cfg(feature = "diagnostics")] pub use heap_snapshot::gc_build_v8_heap_snapshot_json; @@ -1001,6 +1002,8 @@ pub fn gc_init() { reg_scanner!(async_hooks_mutable_root_scanner); reg_scanner!(shape_cache_mutable_root_scanner); reg_scanner!(crate::regex::scan_last_exec_groups_root_mut); + #[cfg(feature = "regex-engine")] + reg_scanner!(crate::regex::site_test::scan_roots_mut); // #7211: the eight interned `typeof` result strings, and JSON.rawJSON's // interned `"rawJSON"` key. Both are thread-local caches of a RAW // `StringHeader*` allocated in the nursery and referenced by nothing else, diff --git a/crates/perry-runtime/src/gc/regex_census.rs b/crates/perry-runtime/src/gc/regex_census.rs new file mode 100644 index 0000000000..ac6647b72c --- /dev/null +++ b/crates/perry-runtime/src/gc/regex_census.rs @@ -0,0 +1,184 @@ +//! RegExp side-table serialization for the requested heap census. + +use super::census::SideTableRow; + +/// Serialize ordinary and RegExp-owned registries through one path. The regex +/// attribution is computed independently of the emitted rows so omission is a +/// visible reconciliation failure rather than a silently smaller total. +pub(super) fn side_table_document_from(mut ordinary: Vec) -> serde_json::Value { + // Replace the legacy RegExp tuples with the rich, reconciled rows. + ordinary.retain(|(table, _, _)| !table.starts_with("regex.")); + let non_regex_total = ordinary.iter().map(|(_, _, bytes)| *bytes).sum::(); + let mut rows = ordinary + .drain(..) + .map(|(table, entries, bytes)| { + serde_json::json!({"table": table, "entries": entries, "bytes": bytes}) + }) + .collect::>(); + + #[cfg(feature = "regex-engine")] + let regex_total = { + let snapshot = crate::regex::census_snapshot(); + rows.extend(snapshot.rows.iter().map(crate::regex::RegexCensusRow::json)); + snapshot.attributed_bytes + }; + #[cfg(not(feature = "regex-engine"))] + let regex_total = 0usize; + + serde_json::json!({ + "rows": rows, + "side_table_bytes": non_regex_total + regex_total, + "regex_side_table_bytes": regex_total, + "non_regex_side_table_bytes": non_regex_total, + }) +} + +#[cfg(test)] +fn test_side_table_document() -> serde_json::Value { + let snapshot = side_table_document_from(super::census::side_tables()); + serde_json::json!({ + "totals": { + "side_table_bytes": snapshot["side_table_bytes"], + "regex_side_table_bytes": snapshot["regex_side_table_bytes"], + "non_regex_side_table_bytes": snapshot["non_regex_side_table_bytes"], + }, + "side_tables": snapshot["rows"], + }) +} + +#[cfg(all(test, feature = "regex-engine"))] +mod tests { + use crate::regex::site_test::js_regexp_site_test_new; + use crate::regex::{js_regexp_new, js_regexp_test}; + + fn string(value: &str) -> *mut crate::StringHeader { + crate::string::js_string_from_bytes(value.as_ptr(), value.len() as u32) + } + + fn regex_rows(doc: &serde_json::Value) -> Vec<&serde_json::Value> { + doc["side_tables"] + .as_array() + .expect("side-table array") + .iter() + .filter(|row| { + row["table"] + .as_str() + .is_some_and(|name| name.starts_with("regex.")) + }) + .collect() + } + + #[test] + fn census_prints_regex_rows_that_reconcile_with_side_table_total() { + let _lock = crate::gc::global_side_table_test_lock(); + crate::regex::census_rows::test_reset_tables(); + + const N: usize = 6; + for index in 0..N { + let source = format!("regex-census-{index}"); + let pattern = string(&source); + let header = js_regexp_new(pattern, string("")); + assert_ne!(js_regexp_test(header, string(&source)), 0); + } + + // Evaluate one direct literal site twice: the second call must reuse + // the first rooted header and its installed program bundle. + let prototype = crate::object::builtin_prototype_value("RegExp"); + assert!(crate::value::JSValue::from_bits(prototype.to_bits()).is_pointer()); + static SITE: u64 = 0; + let site = std::ptr::addr_of!(SITE) as i64; + let first = js_regexp_site_test_new(string("site-census"), string("g"), site); + assert_ne!(js_regexp_test(first, string("site-census")), 0); + let second = js_regexp_site_test_new(string("site-census"), string("g"), site); + assert_eq!( + first, second, + "the literal site must reuse its rooted header" + ); + + assert_eq!( + crate::regex::census_rows::test_walks(), + 0, + "regex construction and matching must not do census bookkeeping" + ); + let encoded = super::test_side_table_document().to_string(); + let doc: serde_json::Value = serde_json::from_str(&encoded).expect("valid census JSON"); + let rows = regex_rows(&doc); + + let names = rows + .iter() + .map(|row| row["table"].as_str().unwrap()) + .collect::>(); + for expected in [ + "regex.pointers", + "regex.program_cache", + "regex.fancy_cache", + "regex.repeat_cache", + "regex.validated_patterns", + "regex.content_cache", + "regex.literal_sites", + "regex.site_table", + "regex.active_factory_sites", + "regex.expando_owners", + "regex.matcher_kinds", + ] { + assert!(names.contains(expected), "missing census row {expected}"); + } + + let pointer = rows + .iter() + .find(|row| row["table"] == "regex.pointers") + .expect("regex.pointers row"); + assert!(pointer["entries"].as_u64().unwrap() >= N as u64); + let site = rows + .iter() + .find(|row| row["table"] == "regex.site_table") + .expect("regex.site_table row"); + assert!(site["sites"].as_u64().unwrap() >= 1); + assert!(site["rooted_headers"].as_u64().unwrap() >= 1); + assert!(site["pinned_programs"].as_u64().unwrap() >= 1); + assert!( + site["pinned_program_bytes"].as_u64().unwrap() + >= site["attributed_program_bytes"].as_u64().unwrap() + ); + assert_eq!(site["pinned_program_bytes_inside_side_table_bytes"], false); + let row_bytes = rows + .iter() + .map(|row| row["bytes"].as_u64().expect("numeric row bytes")) + .sum::(); + let regex_total = doc["totals"]["regex_side_table_bytes"] + .as_u64() + .expect("regex attribution total"); + let side_total = doc["totals"]["side_table_bytes"].as_u64().unwrap(); + let non_regex_total = doc["totals"]["non_regex_side_table_bytes"] + .as_u64() + .unwrap(); + assert_eq!(row_bytes, regex_total); + assert_eq!(side_total - non_regex_total, regex_total); + + // Sabotage proof: the attribution total is built independently of + // JSON row registration. Omitting any non-zero row from `side_tables` + // makes this exact reconciliation fail. + let omitted = rows + .iter() + .find(|row| row["bytes"].as_u64().unwrap_or(0) != 0) + .unwrap()["bytes"] + .as_u64() + .unwrap(); + assert_ne!(row_bytes - omitted, regex_total); + } + + #[test] + fn census_regex_rows_are_zero_cost_when_not_requested() { + let _lock = crate::gc::global_side_table_test_lock(); + crate::regex::census_rows::test_reset_tables(); + let header = js_regexp_new(string("zero-cost-census"), string("")); + assert_ne!(js_regexp_test(header, string("zero-cost-census")), 0); + assert_eq!( + crate::regex::census_rows::test_walks(), + 0, + "construction must not enter regex census row code" + ); + let _ = super::test_side_table_document(); + assert!(crate::regex::census_rows::test_walks() > 0); + } +} diff --git a/crates/perry-runtime/src/gc/tests/copying/survival_and_malloc.rs b/crates/perry-runtime/src/gc/tests/copying/survival_and_malloc.rs index aed0b6c681..fba293deca 100644 --- a/crates/perry-runtime/src/gc/tests/copying/survival_and_malloc.rs +++ b/crates/perry-runtime/src/gc/tests/copying/survival_and_malloc.rs @@ -924,7 +924,6 @@ fn test_movable_regexp_evacuation_migrates_all_address_owned_state() { let old_addr = re as usize; assert!(crate::arena::pointer_in_nursery(old_addr)); assert!(crate::regex::test_regex_pointer_entry_exists(old_addr)); - assert!(crate::regex::test_regex_source_entry_exists(old_addr)); crate::object::exotic_expando::test_seed_exotic_expando_entry( old_addr, @@ -942,8 +941,6 @@ fn test_movable_regexp_evacuation_migrates_all_address_owned_state() { assert!(crate::regex::test_regex_pointer_entry_exists(new_addr)); assert!(!crate::regex::test_regex_pointer_entry_exists(old_addr)); - assert!(crate::regex::test_regex_source_entry_exists(new_addr)); - assert!(!crate::regex::test_regex_source_entry_exists(old_addr)); assert!(crate::object::exotic_expando::test_exotic_expando_entry_exists(new_addr)); assert!(!crate::object::exotic_expando::test_exotic_expando_entry_exists(old_addr)); @@ -1037,9 +1034,8 @@ fn nursery_regexp_that_dies_young_is_finalized_by_the_copied_minor() { "the header must be nursery-allocated" ); assert!(crate::regex::test_regex_pointer_entry_exists(dead_addr)); - assert!(crate::regex::test_regex_source_entry_exists(dead_addr)); // Both headers share one program through the site cache. - let count_before = crate::regex::test_regexp_std_program_strong_count(live); + let count_before = crate::regex::test_regexp_program_set_strong_count(live); assert!(count_before >= 2); // Only `live` is rooted; `dead` is garbage. @@ -1051,15 +1047,13 @@ fn nursery_regexp_that_dies_young_is_finalized_by_the_copied_minor() { assert_ne!(live_new, live_addr, "the rooted RegExp must be evacuated"); assert!(crate::regex::regex_header_has_magic(live_new as *const _)); assert!(crate::regex::test_regex_pointer_entry_exists(live_new)); - assert!(crate::regex::test_regex_source_entry_exists(live_new)); assert!( !crate::regex::test_regex_pointer_entry_exists(dead_addr), "a nursery RegExp that died must be removed from REGEX_POINTERS by the copied minor" ); - assert!(!crate::regex::test_regex_source_entry_exists(dead_addr)); assert_eq!( - crate::regex::test_regexp_std_program_strong_count(live_new as *const _), + crate::regex::test_regexp_program_set_strong_count(live_new as *const _), count_before - 1, "the dead header's Arc clone of the shared program must have been dropped" ); diff --git a/crates/perry-runtime/src/gc/types.rs b/crates/perry-runtime/src/gc/types.rs index 08f9832cd5..97129c6088 100644 --- a/crates/perry-runtime/src/gc/types.rs +++ b/crates/perry-runtime/src/gc/types.rs @@ -224,9 +224,9 @@ pub(crate) enum GcMoveHookKind { /// live on the Error's traced `ObjectMeta` edge and need no side-table /// rekeying. ErrorSideTables, - /// Rekey RegExp identity/source registries plus its exotic expando owner - /// entry. `GC_TYPE_REGEXP` is movable, and all three tables use the - /// payload address as their key. + /// Rekey the RegExp identity registry plus its exotic expando owner entry. + /// `GC_TYPE_REGEXP` is movable, and both tables use the payload address as + /// their key. RegExpSideTables, } diff --git a/crates/perry-runtime/src/hot_diag.rs b/crates/perry-runtime/src/hot_diag.rs index 51615783ca..028d929726 100644 --- a/crates/perry-runtime/src/hot_diag.rs +++ b/crates/perry-runtime/src/hot_diag.rs @@ -126,6 +126,9 @@ pub struct RegexDiag { pub compiles_std: u64, pub compiles_fancy: u64, pub compiles_repeat: u64, + /// One-entry evictions after a regex cache reaches its bound. The former + /// wholesale-clear counter remains as a zeroed regression control. + pub cache_evictions: u64, pub cache_clears: u64, /// `lazy::build_and_install_programs` runs (one per header that is /// executed at least once). @@ -169,17 +172,39 @@ pub struct RegexDiag { /// or missed: this is the `memcmp` volume alone, which is what a 12 KB /// emoji pattern makes expensive and a 60-byte one does not. pub new_site_verify_bytes: u64, - /// Address-keyed side-table inserts performed per construction - /// (`REGEX_POINTERS` and `REGEX_SOURCE_TABLE`) — two per header, each a - /// `PtrHasher` hash plus a hashbrown insert, mirrored by two removals at - /// death and two rekeys per evacuation. + /// Address-keyed side-table inserts performed per construction. This was + /// two (`REGEX_POINTERS` plus the source table) before the header's string + /// slots became traced edges; only `REGEX_POINTERS` remains. pub new_side_table_inserts: u64, + /// Split of the above by table. The source counters are retained as zeroed + /// before/after controls for the #9908 measurement; `REGEX_POINTERS` is + /// still the registry the copied-minor finaliser enumerates. + pub pointer_table_inserts: u64, + pub source_table_inserts: u64, + /// The death side. `source_table_removals` is the zeroed after-control; + /// `regex_header_clear_dead_for_gc` now removes only `REGEX_POINTERS`. + pub pointer_table_removals: u64, + pub source_table_removals: u64, + /// Evacuation rekeys of the remaining pointer registry. + pub side_table_rekeys: u64, /// Constructions answered from the LITERAL-SITE table — identity by the /// compiler-emitted site global's address, so neither the pattern's /// fingerprint nor its byte compare ran. `site_hit` counts the /// CONTENT-keyed cache; a site hit never reaches it, so the two are /// disjoint and `site_key_hit + site_hit <= new`. pub new_site_key_hit: u64, + /// `.test` evaluations served by a site-rooted RegExp header instead of a + /// fresh header allocation. + pub site_test_no_alloc: u64, + /// Validation declines, split so a perf run proves which guard fired. + pub site_test_declined: u64, + pub site_test_declined_patched_prototype: u64, + pub site_test_declined_callee_mismatch: u64, + pub site_test_declined_non_literal: u64, + #[cfg(test)] + test_program_builds: u64, + #[cfg(test)] + test_cache_evictions: u64, per_pattern: HashMap, } @@ -242,6 +267,33 @@ pub fn regex_with(f: impl FnOnce(&mut RegexDiag)) { }); } +#[cfg(test)] +pub(crate) fn test_reset_regex_builds_and_evictions() { + REGEX_DIAG.with(|diag| { + let mut diag = diag.borrow_mut(); + diag.test_program_builds = 0; + diag.test_cache_evictions = 0; + }); +} + +#[cfg(test)] +pub(crate) fn test_note_regex_program_build() { + REGEX_DIAG.with(|diag| diag.borrow_mut().test_program_builds += 1); +} + +#[cfg(test)] +pub(crate) fn test_note_regex_cache_eviction() { + REGEX_DIAG.with(|diag| diag.borrow_mut().test_cache_evictions += 1); +} + +#[cfg(test)] +pub(crate) fn test_regex_builds_and_evictions() -> (u64, u64) { + REGEX_DIAG.with(|diag| { + let diag = diag.borrow(); + (diag.test_program_builds, diag.test_cache_evictions) + }) +} + impl RegexDiag { fn pat(&mut self, pattern_addr: usize, pattern: &[u8], flags: &str) -> &mut PatStat { let entry = self.per_pattern.entry(pattern_addr).or_default(); @@ -321,12 +373,14 @@ impl RegexDiag { let _ = writeln!( out, "[regex-diag] t={secs:.1}s new={} validated_hit={} site_hit={} pattern_bytes={} \ - compiles std={} fancy={} repeat={} cache_clears={} lazy_builds={} lazy_cache_hits={} \ + compiles std={} fancy={} repeat={} cache_clears={} evictions={} lazy_builds={} lazy_cache_hits={} \ exec={} exec_matched={} capture_slots={} capture_bytes={} test={} test_global={} \ match={} replace={} replace_matches={} split={} flags_alloc={} \ desc_regexp_probes={} desc_regexp_meta_negative={} \ barrier_taken={} barrier_gated={} header_bytes={} site_verify_bytes={} \ - side_table_inserts={} site_key_hit={}", + side_table_inserts={} site_key_hit={} ptr_ins={} src_ins={} \ + ptr_rm={} src_rm={} rekeys={} site_test_no_alloc={} \ + site_test_declined={}(patched_prototype={},callee_mismatch={},non_literal={})", self.new_calls, self.new_validated_hit, self.new_site_hit, @@ -335,6 +389,7 @@ impl RegexDiag { self.compiles_fancy, self.compiles_repeat, self.cache_clears, + self.cache_evictions, self.lazy_builds, self.lazy_cache_hits, self.exec_calls, @@ -356,6 +411,16 @@ impl RegexDiag { self.new_site_verify_bytes, self.new_side_table_inserts, self.new_site_key_hit, + self.pointer_table_inserts, + self.source_table_inserts, + self.pointer_table_removals, + self.source_table_removals, + self.side_table_rekeys, + self.site_test_no_alloc, + self.site_test_declined, + self.site_test_declined_patched_prototype, + self.site_test_declined_callee_mismatch, + self.site_test_declined_non_literal, ); // Merge by content (prefix, len, flags): distinct literal sites with // the same pattern are one row. diff --git a/crates/perry-runtime/src/object/exotic_expando.rs b/crates/perry-runtime/src/object/exotic_expando.rs index ff1f71f90e..1037086688 100644 --- a/crates/perry-runtime/src/object/exotic_expando.rs +++ b/crates/perry-runtime/src/object/exotic_expando.rs @@ -639,6 +639,33 @@ pub fn scan_exotic_expando_roots_mut(visitor: &mut crate::gc::RuntimeRootVisitor } } +/// `PERRY_GC_CENSUS`: attribute the RegExp-owned share of the mixed exotic +/// expando table. Hash-table storage is divided evenly by owner; each RegExp +/// owner's vector and key buffers are then charged exactly to its row. +#[cfg(feature = "regex-engine")] +pub(crate) fn regex_expando_census() -> (usize, usize, usize) { + let map = crate::state::state().exotic_expando.entries.borrow(); + let regex = map + .iter() + .filter(|(owner, _)| exotic_expando_kind(**owner) == Some(ExoticKind::RegExp)) + .collect::>(); + let owners = regex.len(); + let properties = regex.iter().map(|(_, entries)| entries.len()).sum(); + let shared = if map.is_empty() { + 0 + } else { + crate::gc::census::map_bytes(&*map) * owners / map.len() + }; + let inner = regex + .iter() + .map(|(_, entries)| { + crate::gc::census::vec_bytes(entries) + + entries.iter().map(|(key, _)| key.capacity()).sum::() + }) + .sum::(); + (owners, properties, shared + inner) +} + #[cfg(test)] mod tests { use super::*; diff --git a/crates/perry-runtime/src/regex.rs b/crates/perry-runtime/src/regex.rs index 2a7c5461e0..14a77af6a7 100644 --- a/crates/perry-runtime/src/regex.rs +++ b/crates/perry-runtime/src/regex.rs @@ -6,13 +6,12 @@ #[cfg(feature = "regex-engine")] use regex::Regex; use std::cell::RefCell; -// Every use of `HashMap` in this file is inside a `#[cfg(feature = "regex-engine")]` -// block, so an unconditional import is an unused-import error under the -// `warnings` job's `-D warnings` when `perry`'s own binaries pull the runtime -// in without that feature. +// Every `HashMap` use is behind `regex-engine`; gate the import too so the +// feature-off `-D warnings` build does not see it as unused. #[cfg(feature = "regex-engine")] use std::collections::HashMap; use std::ptr; +#[cfg(feature = "regex-engine")] use std::sync::Arc; #[cfg(feature = "regex-engine")] @@ -23,15 +22,17 @@ use crate::value::js_nanbox_string; use crate::object::ObjectHeader; -/// The compiled standard-engine regex type. When the regex engine is gated -/// off, `RegExpHeader::regex_ptr` is typed `*mut ()` (a never-dereferenced -/// dangling field) so the identity/display layer keeps the same struct -/// layout without pulling in the `regex` crate. +/// The shared compiled-program set. When the regex engine is gated off, +/// `RegExpHeader::programs_ptr` is typed `*const ()` (a never-dereferenced +/// field) so the identity/display layer keeps the same struct layout without +/// pulling in the matcher crates. #[cfg(feature = "regex-engine")] -type CompiledRegex = regex::Regex; +type CompiledPrograms = site_cache::Programs; #[cfg(not(feature = "regex-engine"))] -type CompiledRegex = (); +type CompiledPrograms = (); +#[cfg(feature = "regex-engine")] +pub(crate) mod census_rows; #[cfg(feature = "regex-engine")] mod class_range_validate; #[cfg(feature = "regex-engine")] @@ -46,6 +47,8 @@ mod program_key; #[cfg(feature = "regex-engine")] mod replace_expand_fancy; #[cfg(feature = "regex-engine")] +pub(crate) use census_rows::{census_snapshot, RegexCensusRow}; +#[cfg(feature = "regex-engine")] pub(crate) use program_key::{ProgramKey, NEVER_MATCH_PATTERN}; #[cfg(feature = "regex-engine")] pub use replace_expand_fancy::{ @@ -62,6 +65,7 @@ mod grammar; mod lazy; #[cfg(feature = "regex-engine")] mod match_all; +mod properties; #[cfg(feature = "regex-engine")] mod repeat_matcher; #[cfg(feature = "regex-engine")] @@ -75,6 +79,8 @@ mod site_cache; #[cfg(feature = "regex-engine")] mod site_key; #[cfg(feature = "regex-engine")] +pub(crate) mod site_test; +#[cfg(feature = "regex-engine")] mod unicode17; #[cfg(feature = "regex-engine")] mod unicode17_data; @@ -83,7 +89,6 @@ mod utf16; use class_range_validate::has_out_of_order_double_dash_class_range; #[cfg(feature = "regex-engine")] pub use compile::js_regexp_compile_value; -use escape::escape_regexp_source; pub use escape::js_regexp_escape; #[cfg(feature = "regex-engine")] use exec_array::{ @@ -105,11 +110,14 @@ pub(crate) use match_all::dispatch_regexp_string_iterator_method_builtin; pub use match_all::{ dispatch_regexp_string_iterator_method, js_string_match_all, js_string_match_all_value, }; +pub use properties::{ + js_regexp_empty_source, js_regexp_get_flags, js_regexp_get_last_index, js_regexp_get_source, + js_regexp_set_last_index, js_regexp_to_string, +}; -/// Class id for `RegExp String Iterator` exotic objects. Referenced by the -/// always-linked iterator-prototype dispatch, so it stays ungated even when -/// the regex engine (which produces these iterators) is compiled out. +/// Class id shared with the always-linked RegExp string-iterator dispatch. pub const REGEXP_STRING_ITERATOR_CLASS_ID: u32 = 0xFFFF_000A; + #[cfg(feature = "regex-engine")] use replace_expand::expand_js_replacement; #[cfg(feature = "regex-engine")] @@ -145,21 +153,6 @@ crate::perry_thread_local! { /// relocate or die. Header magic remains the primary identity check. static REGEX_POINTERS: RefCell> = RefCell::new(crate::fast_hash::new_ptr_hash_set()); - /// Issue #637: Owned copies of pattern and flags strings keyed by - /// the RegExpHeader pointer. The header's `pattern_ptr` / `flags_ptr` - /// fields hold raw `*const StringHeader` pointers to the input - /// strings — when those inputs are temporaries (e.g. the result of - /// a template-literal expression `\`^${p}\``), the GC frees them - /// after the function call returns and subsequent `.source` / - /// `.flags` reads dereference dangling memory. We side-table an - /// owned `String` copy at construction time; readers prefer this - /// over `pattern_ptr` whenever an entry exists. - /// - /// The copies are `Arc` shared with `regex::site_cache`: every - /// header built from the same literal text bumps two refcounts instead - /// of copying the pattern (12 KB for emoji-class patterns, once per - /// evaluation of the literal). - static REGEX_SOURCE_TABLE: RefCell, Arc)>> = RefCell::new(crate::fast_hash::new_ptr_hash_map()); } /// Check whether `ptr` is a RegExpHeader pointer that was allocated in @@ -206,36 +199,40 @@ pub(crate) fn regex_header_moved_for_gc(old_addr: usize, new_addr: usize) { if old_addr == new_addr { return; } + if crate::hot_diag::regex_on() { + crate::hot_diag::regex_counters(|d| d.side_table_rekeys += 1); + } REGEX_POINTERS.with(|table| { let mut table = table.borrow_mut(); if table.remove(&old_addr) { table.insert(new_addr); } }); - REGEX_SOURCE_TABLE.with(|table| { - let mut table = table.borrow_mut(); - if let Some(source) = table.remove(&old_addr) { - table.insert(new_addr, source); - } - }); crate::object::exotic_expando::exotic_expando_owner_moved(old_addr, new_addr); } /// Remove address-owned RegExp metadata when the cell is proven dead. pub(crate) fn regex_header_clear_dead_for_gc(addr: usize) { + // Counted, not timed: this runs inside a collection, so a probe here must + // allocate nothing and must not dump. `regex_counters` does neither, and + // `regex_on`'s one-time env read cannot first happen here — a header can + // only die after `js_regexp_new` created it, and that path arms the + // instrument first. + if crate::hot_diag::regex_on() { + crate::hot_diag::regex_counters(|d| { + d.pointer_table_removals += 1; + }); + } REGEX_POINTERS.with(|table| { table.borrow_mut().remove(&addr); }); - REGEX_SOURCE_TABLE.with(|table| { - table.borrow_mut().remove(&addr); - }); crate::object::exotic_expando::exotic_expando_owner_clear_dead(addr); } /// Release the compiled programs owned by a dead `RegExpHeader`, then remove /// its address-owned metadata. /// -/// The program pointers are raw `Arc` references installed by +/// The program pointer is a raw `Arc` reference installed by /// `lazy::build_and_install_programs` or `RegExp.prototype.compile`. Null them /// before reconstructing the `Arc`s because arena cleanup can visit the /// metadata and finalizer paths for the same dead cell. @@ -245,23 +242,11 @@ pub(crate) unsafe fn regex_header_finalize_for_gc(re: *mut RegExpHeader) { } #[cfg(feature = "regex-engine")] { - let regex_ptr = (*re).regex_ptr; - let fancy_ptr = (*re).fancy_ptr; - let repeat_matcher_ptr = (*re).repeat_matcher_ptr; - (*re).regex_ptr = ptr::null_mut(); - (*re).fancy_ptr = ptr::null(); - (*re).repeat_matcher_ptr = ptr::null(); - - if !regex_ptr.is_null() { - drop(Arc::from_raw(regex_ptr as *const Regex)); - } - if !fancy_ptr.is_null() { - drop(Arc::from_raw(fancy_ptr as *const fancy_regex::Regex)); - } - if !repeat_matcher_ptr.is_null() { - drop(Arc::from_raw( - repeat_matcher_ptr as *const repeat_matcher::RepeatMatcherRegex, - )); + let programs_ptr = (*re).programs_ptr; + (*re).programs_ptr = ptr::null(); + + if !programs_ptr.is_null() { + drop(Arc::from_raw(programs_ptr)); } } regex_header_clear_dead_for_gc(re as usize); @@ -271,7 +256,7 @@ pub(crate) unsafe fn regex_header_finalize_for_gc(re: *mut RegExpHeader) { /// /// The copying minor's from-space flip runs no per-object finalize hooks, so /// a nursery header that was neither evacuated nor pinned would otherwise keep -/// its `Arc` programs and its `REGEX_POINTERS` / `REGEX_SOURCE_TABLE` / expando +/// its program-set `Arc` and its `REGEX_POINTERS` / expando /// entries forever. Same shape as `map::finalize_dead_copied_minor_from_space_maps`: /// walk the registry after the flip, collect the provably-dead addresses, then /// finalize each (the finalizer removes its own registry entries, which is why @@ -370,14 +355,14 @@ pub(crate) fn test_construct_regexp_and_exec_once(pattern: &str, flags: &str) -> /// Test support: strong count of the standard program a header holds (the /// observer clone taken here is released before returning). #[cfg(all(test, feature = "regex-engine"))] -pub(crate) fn test_regexp_std_program_strong_count(re: *const RegExpHeader) -> usize { +pub(crate) fn test_regexp_program_set_strong_count(re: *const RegExpHeader) -> usize { unsafe { - let raw = (*re).regex_ptr as *const Regex; - assert!(!raw.is_null(), "program must be installed"); - let arc = Arc::from_raw(raw); - let n = Arc::strong_count(&arc); + let programs = (*re).programs_ptr; + assert!(!programs.is_null(), "program must be installed"); + let arc = Arc::from_raw(programs); + let count = Arc::strong_count(&arc); std::mem::forget(arc); - n + count } } @@ -386,11 +371,6 @@ pub(crate) fn test_regex_pointer_entry_exists(addr: usize) -> bool { REGEX_POINTERS.with(|table| table.borrow().contains(&addr)) } -#[cfg(test)] -pub(crate) fn test_regex_source_entry_exists(addr: usize) -> bool { - REGEX_SOURCE_TABLE.with(|table| table.borrow().contains_key(&addr)) -} - /// Build a minimal nursery-resident RegExp payload for the copying collector's /// relocation contract test. Production construction currently chooses the /// malloc-backed arm of `ArenaOrMalloc`; this exercises the same registered GC @@ -398,6 +378,9 @@ pub(crate) fn test_regex_source_entry_exists(addr: usize) -> bool { /// strand the address-owned tables. #[cfg(all(test, feature = "regex-engine"))] pub(crate) fn test_alloc_nursery_regexp_for_move(source: &str, flags: &str) -> *mut RegExpHeader { + let scope = crate::gc::RuntimeHandleScope::new(); + let pattern = scope.root_string_ptr(js_string_from_str(source)); + let flags_string = scope.root_string_ptr(js_string_from_str(flags)); unsafe { let ptr = crate::arena::arena_alloc_gc( std::mem::size_of::(), @@ -407,9 +390,13 @@ pub(crate) fn test_alloc_nursery_regexp_for_move(source: &str, flags: &str) -> * // Neither `gc_malloc` nor the arena zeroes reused memory, so this // must be set explicitly or the GC follows a garbage pointer. (*ptr).meta = std::ptr::null_mut(); - (*ptr).regex_ptr = std::ptr::null_mut(); - (*ptr).pattern_ptr = std::ptr::null(); - (*ptr).flags_ptr = std::ptr::null(); + (*ptr).programs_ptr = std::ptr::null(); + pattern.with_const_ptr::(|pattern| { + (*ptr).pattern_ptr = pattern; + }); + flags_string.with_const_ptr::(|flags| { + (*ptr).flags_ptr = flags; + }); (*ptr).case_insensitive = flags.contains('i'); (*ptr).global = flags.contains('g'); (*ptr).multiline = flags.contains('m'); @@ -417,20 +404,14 @@ pub(crate) fn test_alloc_nursery_regexp_for_move(source: &str, flags: &str) -> * (*ptr).dot_all = flags.contains('s'); (*ptr).unicode = flags.contains('u') || flags.contains('v'); (*ptr).has_indices = flags.contains('d'); + (*ptr).matcher_kind = MatcherKind::Unbuilt; (*ptr).last_index = crate::value::JSValue::number(0.0).bits(); (*ptr).magic = REGEXP_MAGIC; - (*ptr).fancy_ptr = std::ptr::null(); - (*ptr).repeat_matcher_ptr = std::ptr::null(); REGEX_EVER_REGISTERED.arm(); REGEX_POINTERS.with(|table| { table.borrow_mut().insert(ptr as usize); }); - REGEX_SOURCE_TABLE.with(|table| { - table - .borrow_mut() - .insert(ptr as usize, (Arc::from(source), Arc::from(flags))); - }); ptr } } @@ -474,7 +455,7 @@ pub(crate) fn regex_header_has_magic(re: *const RegExpHeader) -> bool { /// * `flags_ptr` — the flags `StringHeader`, /// * `last_index` — a writable JSValue (`re.lastIndex = …`) that may be a /// NaN-boxed heap pointer. -/// The compiled matcher pointers point to OFF-heap leaked Rust allocations and the +/// The compiled-program pointer points to an OFF-heap Rust allocation and the /// bool/`magic` fields are never heap refs, so they must NOT be scanned. /// /// `pattern_ptr` and `flags_ptr` are consecutive equal-width fields, so under @@ -508,9 +489,9 @@ crate::perry_thread_local! { /// validation. Validity is a pure function of the pair, so the answer is /// worth remembering; `js_regexp_new` used to get this from a /// `REGEX_CACHE` hit, which stopped being a proxy once the compiled - /// program became lazy (see `regex::lazy`). Same cap and - /// clear-on-overflow policy as the program caches — the cost of a clear - /// is a repeated parse, never a wrong verdict. The unit value keeps + /// program became lazy (see `regex::lazy`). Same cap and one-entry + /// eviction policy as the program caches — eviction can repeat one parse, + /// never change a verdict. The unit value keeps /// `evict_regex_cache_if_full` shared with the three program caches. static VALIDATED_PATTERNS: RefCell> = RefCell::new(HashMap::new()); } @@ -520,14 +501,31 @@ mod compile_cache; #[cfg(feature = "regex-engine")] pub(crate) use compile_cache::*; +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +#[repr(u8)] +#[cfg_attr( + not(feature = "regex-engine"), + allow( + dead_code, + reason = "the feature-off runtime preserves RegExpHeader layout but constructs no matchers" + ) +)] +pub(super) enum MatcherKind { + Unbuilt, + Standard, + Fancy, + Repeat, +} + /// Header for heap-allocated RegExp objects #[repr(C)] pub struct RegExpHeader { - /// Pointer to the compiled Regex object (boxed). Typed via the - /// `CompiledRegex` alias so the struct layout is identical whether or not - /// the regex engine is linked (it's `*mut ()` when gated off and never - /// dereferenced — all dereferencing sites are themselves engine-gated). - regex_ptr: *mut CompiledRegex, + /// Header-owned `Arc` raw pointer, or null until first use. + /// The program set contains the standard engine and optional fancy/repeat + /// matchers once per pattern instead of repeating three pointers in every + /// RegExp object. Typed through `CompiledPrograms` so the layout is stable + /// when the regex engine is gated off. + programs_ptr: *const CompiledPrograms, /// Original pattern string (for debugging/serialization) pattern_ptr: *const StringHeader, /// Flags string (e.g., "gi" for global+ignoreCase) @@ -543,6 +541,9 @@ pub struct RegExpHeader { pub dot_all: bool, pub unicode: bool, pub has_indices: bool, + /// Selected engine after the first build. This occupies the byte that was + /// padding before `last_index`, so it does not grow the 56-byte header. + matcher_kind: MatcherKind, /// `lastIndex` is a writable data property holding an *arbitrary* JSValue /// (spec: `Set(R, "lastIndex", v)` with no coercion on write). Stored as the /// raw NaN-boxed bits; `exec`/`test` apply `ToLength` on read to derive the @@ -562,19 +563,10 @@ pub struct RegExpHeader { /// string pattern → never matches → get-intrinsic's `stringToPath` returns /// `[]` → `intrinsic %% does not exist!` → express adapter load `exit(1)`. /// - /// Storing the marker (and the fancy-regex Arc) ON the heap header makes - /// identity + fancy-fallback resolution independent of WHICH runtime copy's + /// Storing the marker and program-set handle ON the heap header makes + /// identity + fallback resolution independent of WHICH runtime copy's /// thread-locals are live. Set to `REGEXP_MAGIC` by `js_regexp_new`. pub magic: u64, - /// Leaked `Arc` (as a raw pointer) for patterns the - /// `regex` crate can't compile (lookahead/lookbehind/backrefs), or null. - /// Header-resident twin of the `FANCY_CACHE` thread-local so the fancy - /// fallback survives the duplicate-runtime split described above. - pub fancy_ptr: *const (), - /// Header-owned `Arc` for quantified capture groups, - /// or null for the ordinary linear/fancy paths. Like `fancy_ptr`, this - /// survives cache eviction and duplicate statically-linked runtime copies. - pub repeat_matcher_ptr: *const (), /// #6759 phase 1 (header unification): per-object metadata record, or /// null. Appended LAST so `regex_gc_slot_ptrs`' adjacency assertion on /// `pattern_ptr`/`flags_ptr` and every other offset are undisturbed. @@ -724,8 +716,8 @@ fn newborn_barrier_gate_enabled() -> bool { /// /// Validates the pattern and allocates the header; it does NOT build the /// compiled program. That happens on the first operation that needs a matcher -/// — see `regex::lazy`, and the `regex_ptr`/`fancy_ptr`/`repeat_matcher_ptr` -/// fields, which are null until then. A fresh header per call is required: +/// — see `regex::lazy`; `programs_ptr` is null until then. A fresh header per +/// call is required: /// ECMA-262 evaluates a regex literal to a NEW object every time, and the /// distinction is observable through `===`, expandos and `lastIndex`. #[cfg(feature = "regex-engine")] @@ -811,7 +803,7 @@ fn js_regexp_new_impl( // A `site_key` of 0 (every dynamic construction, and every runtime caller) // misses by construction and takes the content-keyed path below unchanged. let site_entry = site_key::lookup(site_key, raw_flags_str); - let (owned_pattern, owned_flags, programs, bits, shared_flags_root) = match site_entry { + let (programs, bits, shared_flags_root, owned_flags) = match site_entry { Some(hit) => { // The site's own flags literal, so this is the same sharing // decision the first construction at this site made (#9819). @@ -845,13 +837,7 @@ fn js_regexp_new_impl( picked } }; - ( - hit.pattern, - hit.flags, - programs, - hit.bits, - shared_flags_root, - ) + (programs, hit.bits, shared_flags_root, hit.flags) } None => { let pattern_str = if is_valid_ptr(pattern) { @@ -1032,16 +1018,16 @@ fn js_regexp_new_impl( // established that the pattern is legal, and a bundle evaluates hundreds // of module-level literals it never matches with — building each one's // NFA at construction is what put ~14% of a claude-code `--help` run - // inside `regex_syntax`/`regex_automata`. `regex_ptr` stays null (the + // inside `regex_syntax`/`regex_automata`. `programs_ptr` stays null (the // "not built yet" state) and `lazy::ensure_regex_compiled` installs the // owned `Arc`s on the first operation that needs a matcher. // ★ Last use of the borrowed pattern text before this function allocates. // `pattern_str` borrows the GC string; the two allocations below can move - // it, and everything after this point reads the pattern from `owned_pattern` - // (a shared `Arc`, which relocation cannot invalidate) or from - // `pattern_root` (a runtime handle the collector rewrites). Nothing below - // may use `pattern_str` or the incoming `pattern` argument again. + // it. The site/content cache snapshots it into `owned_pattern`, and + // the header store below re-reads it from `pattern_root` (a runtime + // handle the collector rewrites). Nothing below may use `pattern_str` + // or the incoming `pattern` argument again. let (owned_pattern, owned_flags, programs) = match site_hit { Some(hit) => (hit.pattern, hit.flags, hit.programs), None => { @@ -1069,19 +1055,13 @@ fn js_regexp_new_impl( site_key::record( site_key, raw_flags_owned, - owned_pattern.clone(), + owned_pattern, owned_flags.clone(), flags_are_canonical, bits, programs.clone(), ); - ( - owned_pattern, - owned_flags, - programs, - bits, - shared_flags_root, - ) + (programs, bits, shared_flags_root, owned_flags) } }; let site_key::FlagBits { @@ -1110,14 +1090,14 @@ fn js_regexp_new_impl( // old-generation prices to do it. // // `GC_TYPE_REGEXP` has been movable (`GcMoveHookKind::RegExpSideTables` - // rekeys `REGEX_POINTERS`, `REGEX_SOURCE_TABLE` and the expando owner + // rekeys `REGEX_POINTERS` and the expando owner // after evacuation; `GcLayoutSlotKind::RegExpFields` traces the two string // edges and `meta`) since the copying collector landed, and // `test_movable_regexp_evacuation_migrates_all_address_owned_state` has // exercised the arena arm all along. What kept production on malloc was // finalization: the copying minor's from-space flip runs no per-object // finalize hooks (`gc::copying`), so a nursery header that dies young - // would leak its three `Arc` programs and its registry entries. That is + // would leak its program-set `Arc` and its registry entries. That is // now handled the way Map/Set/Error handle theirs — // `finalize_dead_copied_minor_from_space_regexps` after a copied minor and // `collect_dead_registered_regexps_post_trace` at sweep entry for the @@ -1125,8 +1105,8 @@ fn js_regexp_new_impl( // old-generation sweep's ordinary `gc_type_finalize_unmarked_payload`. let header_size = std::mem::size_of::(); // `flags_ptr` must hold the CANONICAL form, so that `flags_ptr`-keyed - // lookups (FANCY_CACHE, lookup_fancy_regex) and the GC-survivable source - // table all agree. When the caller's string already is that text it is + // lookups (FANCY_CACHE, lookup_fancy_regex) agree. When the caller's + // string already is that text it is // shared (rooted above); only a non-canonical spelling (`/x/ig` → `"gi"`, // or a computed `new RegExp(p, f)`) still has to materialize one. The // counter makes the removal provable rather than asserted. @@ -1182,7 +1162,7 @@ fn js_regexp_new_impl( // must be set explicitly or the GC follows a garbage pointer. (*ptr).meta = std::ptr::null_mut(); // Null = not compiled yet; see `lazy::ensure_regex_compiled`. - (*ptr).regex_ptr = std::ptr::null_mut(); + (*ptr).programs_ptr = std::ptr::null(); (*ptr).pattern_ptr = pattern; (*ptr).flags_ptr = canonical_flags_ptr; // `pattern_ptr` / `flags_ptr` are GC-managed StringHeaders — the GC scans @@ -1258,30 +1238,17 @@ fn js_regexp_new_impl( (*ptr).dot_all = dot_all; (*ptr).unicode = unicode; (*ptr).has_indices = has_indices; + (*ptr).matcher_kind = MatcherKind::Unbuilt; (*ptr).last_index = crate::value::JSValue::number(0.0).bits(); // Wall 18: self-identifying marker so identity checks survive a // duplicate-runtime thread-local split. (*ptr).magic = REGEXP_MAGIC; - // The header-resident fancy-regex fallback (lookahead/lookbehind/ - // backrefs) and the ECMAScript backtracking matcher are installed - // alongside `regex_ptr` by `lazy::ensure_regex_compiled`, from the - // same caches, on the first operation that needs a matcher. Keeping - // all three on one publish point is what makes `regex_ptr.is_null()` - // a sound built/not-built flag. - (*ptr).fancy_ptr = std::ptr::null(); - (*ptr).repeat_matcher_ptr = std::ptr::null(); - // Born built: the site cache already holds the programs the first - // execution of this text compiled. Install the same three owned - // references `lazy::build_and_install_programs` would, publishing - // `regex_ptr` last for the same reason it does. + // Born built: the site cache already holds the shared program set the + // first execution of this text compiled. Install one owned reference; + // null remains the sound not-built state. if let Some(programs) = programs { - (*ptr).fancy_ptr = programs - .fancy - .map_or(std::ptr::null(), |arc| Arc::into_raw(arc) as *const ()); - (*ptr).repeat_matcher_ptr = programs - .repeat - .map_or(std::ptr::null(), |arc| Arc::into_raw(arc) as *const ()); - (*ptr).regex_ptr = Arc::into_raw(programs.std) as *mut Regex; + (*ptr).matcher_kind = programs.matcher_kind(); + (*ptr).programs_ptr = Arc::into_raw(programs); } // Record the pointer so that js_string_split can detect @@ -1292,21 +1259,16 @@ fn js_regexp_new_impl( s.borrow_mut().insert(ptr as usize); }); if crate::hot_diag::regex_on() { - // Two address-keyed inserts per construction (this one and - // `REGEX_SOURCE_TABLE` below), each a `PtrHasher` hash plus a - // hashbrown insert, mirrored by two removals at death and two - // rekeys per evacuation. Counted so the pair is a number rather - // than a reading of the profile. - crate::hot_diag::regex_counters(|d| d.new_side_table_inserts += 2); + // One address-keyed insert per construction. `REGEX_POINTERS` + // remains because the copied-minor finaliser enumerates it; the + // former source table became redundant when #9845 made the + // header's two string slots traced GC edges. + crate::hot_diag::regex_counters(|d| { + d.new_side_table_inserts += 1; + d.pointer_table_inserts += 1; + }); } - // Issue #637: side-table owned copies of pattern + flags so - // `.source` / `.flags` survive GC of the input StringHeaders. - REGEX_SOURCE_TABLE.with(|t| { - t.borrow_mut() - .insert(ptr as usize, (owned_pattern, owned_flags)); - }); - ptr } } @@ -1335,10 +1297,18 @@ pub extern "C" fn js_regexp_construct(pattern: f64, flags: f64) -> *mut RegExpHe let (source_string, inherited_flags) = if pattern_is_regex { let re = pv.as_pointer::(); - let entry = REGEX_SOURCE_TABLE.with(|t| t.borrow().get(&(re as usize)).cloned()); - match entry { - Some((pat, fl)) => (pat.to_string(), Some(fl.to_string())), - None => (String::new(), Some(String::new())), + unsafe { + let source = if is_valid_ptr((*re).pattern_ptr) { + string_as_str((*re).pattern_ptr).to_string() + } else { + String::new() + }; + let inherited = if is_valid_ptr((*re).flags_ptr) { + string_as_str((*re).flags_ptr).to_string() + } else { + String::new() + }; + (source, Some(inherited)) } } else if pv.is_undefined() { (String::new(), None) @@ -1469,16 +1439,32 @@ pub(crate) fn regexp_test_str_bounded(re: *const RegExpHeader, hay: &str) -> Opt if crate::hot_diag::regex_on() { diag_note_op(re, crate::hot_diag::RegexOp::Test); } - if let Some(repeat_matcher) = lookup_repeat_matcher(re) { - return Some(repeat_matcher.regex.find(hay).is_some()); - } - if let Some(fre) = lookup_fancy_regex(re) { - return match fre.is_match(hay) { - Ok(v) => Some(v), - Err(_) => None, - }; + lazy::ensure_regex_compiled(re); + let programs = &*(*re).programs_ptr; + match (*re).matcher_kind { + MatcherKind::Repeat => { + let repeat = programs + .repeat + .as_ref() + .expect("repeat matcher tag must name a repeat program"); + Some(repeat.regex.find(hay).is_some()) + } + MatcherKind::Fancy => { + let fancy = programs + .fancy + .as_ref() + .expect("fancy matcher tag must name a fancy program"); + match fancy.is_match(hay) { + Ok(v) => Some(v), + Err(_) => None, + } + } + MatcherKind::Standard => Some(programs.std.is_match(hay)), + MatcherKind::Unbuilt => { + debug_assert!(false, "compiled header kept the unbuilt matcher tag"); + Some(programs.std.is_match(hay)) + } } - Some(lazy::header_std_regex(re).is_match(hay)) } } @@ -1565,32 +1551,14 @@ pub(super) fn diag_note_op(re: *const RegExpHeader, op: crate::hot_diag::RegexOp /// pattern (backreferences, lookbehind, etc.). #[cfg(feature = "regex-engine")] pub(crate) fn lookup_fancy_regex(re: *const RegExpHeader) -> Option> { - // The header's programs are built on first use; `fancy_ptr` is null until - // then, and a null there is indistinguishable from "this pattern has no - // fancy fallback" — so build before reading it. + // The header's shared program set is built on first use. lazy::ensure_regex_compiled(re); unsafe { - // Wall 18: header-resident fancy Arc first (duplicate-runtime - // thread-local resilient). `fancy_ptr` is a leaked `Arc` raw pointer; to - // hand back an owned `Arc` clone WITHOUT consuming the header's - // reference, reconstruct, clone, then `mem::forget` the reconstructed - // one so the header's strong count is preserved. + // Wall 18: header-resident program set first (duplicate-runtime + // thread-local resilient). if regex_header_has_magic(re) { - if (*re).fancy_ptr.is_null() { - // Built (see `ensure_regex_compiled` above) with no fancy - // fallback: every install path (`lazy`, `compile`, the site - // cache) publishes all three program pointers together, so a - // null here is the answer, not "not looked up yet". Falling - // through to the cache probe re-hashed the whole pattern on - // EVERY exec of every ordinary regex (#keystroke profile: - // 514 samples under this function alone). - return None; - } - let raw = (*re).fancy_ptr as *const fancy_regex::Regex; - let arc = Arc::from_raw(raw); - let cloned = arc.clone(); - std::mem::forget(arc); - return Some(cloned); + let programs = &*(*re).programs_ptr; + return programs.fancy.clone(); } let pat = string_as_str((*re).pattern_ptr); let flags_str = string_as_str((*re).flags_ptr); @@ -1638,11 +1606,11 @@ pub(crate) fn lookup_fancy_regex(re: *const RegExpHeader) -> Option bool { unsafe { - let program = (*re).regex_ptr; - if program.is_null() { + let programs = (*re).programs_ptr; + if programs.is_null() { return false; } - let program: &Regex = &*program; + let program: &Regex = &(*programs).std; if program.as_str() == NEVER_MATCH_PATTERN { // The `regex` crate refused this pattern (lookaround / // backreference); it has no opinion about the subject. @@ -1676,22 +1644,11 @@ fn lookup_repeat_matcher_for( fn lookup_repeat_matcher( re: *const RegExpHeader, ) -> Option> { - // Same first-use build as `lookup_fancy_regex`: a null - // `repeat_matcher_ptr` means "not built yet" before it can mean "this - // pattern needs no backtracking matcher". lazy::ensure_regex_compiled(re); unsafe { if regex_header_has_magic(re) { - if (*re).repeat_matcher_ptr.is_null() { - // Same reasoning as `lookup_fancy_regex`: a built header with - // a null pointer has no backtracking matcher. - return None; - } - let raw = (*re).repeat_matcher_ptr as *const repeat_matcher::RepeatMatcherRegex; - let arc = Arc::from_raw(raw); - let cloned = arc.clone(); - std::mem::forget(arc); - return Some(cloned); + let programs = &*(*re).programs_ptr; + return programs.repeat.clone(); } let pat = string_as_str((*re).pattern_ptr); let flags_str = string_as_str((*re).flags_ptr); @@ -1804,93 +1761,11 @@ pub(crate) fn test_last_exec_groups() -> usize { LAST_EXEC_GROUPS.with(|g| *g.borrow() as usize) } -/// Get regex.source — returns the pattern string -#[no_mangle] -pub extern "C" fn js_regexp_get_source(re: *const RegExpHeader) -> *mut StringHeader { - if !is_valid_regex_ptr(re) { - return js_string_from_str("(?:)"); - } - // Issue #637: prefer the side-tabled owned copy so we survive GC - // of the input StringHeader (e.g. template-literal temporary). - if let Some(pat) = - REGEX_SOURCE_TABLE.with(|t| t.borrow().get(&(re as usize)).map(|(p, _)| p.clone())) - { - return js_string_from_str(&escape_regexp_source(&pat)); - } - unsafe { - if is_valid_ptr((*re).pattern_ptr) { - // Return a copy of the pattern string - let pattern_str = string_as_str((*re).pattern_ptr); - js_string_from_str(&escape_regexp_source(pattern_str)) - } else { - js_string_from_str("(?:)") - } - } -} - -/// `RegExp.prototype.source` for the prototype object itself (no -/// `[[OriginalSource]]`) returns the canonical empty source `"(?:)"`. -#[no_mangle] -pub extern "C" fn js_regexp_empty_source() -> *mut StringHeader { - js_string_from_str("(?:)") -} - -/// Get regex.flags — returns the flags string -#[no_mangle] -pub extern "C" fn js_regexp_get_flags(re: *const RegExpHeader) -> *mut StringHeader { - if !is_valid_regex_ptr(re) { - return js_string_from_str(""); - } - // Issue #637: prefer the side-tabled owned copy. - if let Some(flags) = - REGEX_SOURCE_TABLE.with(|t| t.borrow().get(&(re as usize)).map(|(_, f)| f.clone())) - { - return js_string_from_str(&flags); - } - unsafe { - if is_valid_ptr((*re).flags_ptr) { - let flags_str = string_as_str((*re).flags_ptr); - js_string_from_str(flags_str) - } else { - js_string_from_str("") - } - } -} - -/// `RegExp.prototype.toString()` — `/source/flags`. Used by both the -/// `regex.toString()` method dispatch and ToString coercion (`String(re)`, -/// template literals). Node never produces `"[object Object]"` for a RegExp. -#[no_mangle] -pub extern "C" fn js_regexp_to_string(re: *const RegExpHeader) -> *mut StringHeader { - let src = js_regexp_get_source(re); - let flg = js_regexp_get_flags(re); - let out = format!("/{}/{}", string_as_str(src), string_as_str(flg)); - js_string_from_str(&out) -} - -/// Get regex.lastIndex — returns the stored value (NaN-boxed JSValue bits as -/// f64). Usually a number, but `re.lastIndex = obj` round-trips the object. -#[no_mangle] -pub extern "C" fn js_regexp_get_last_index(re: *const RegExpHeader) -> f64 { - if !is_valid_regex_ptr(re) { - return 0.0; - } - unsafe { f64::from_bits((*re).last_index) } -} - -/// Set regex.lastIndex — stores the value verbatim (no coercion on write, per -/// spec `Set(R, "lastIndex", v)`). -#[no_mangle] -pub extern "C" fn js_regexp_set_last_index(re: *mut RegExpHeader, value: f64) { - if !is_valid_regex_ptr(re) { - return; - } - unsafe { - (*re).last_index = value.to_bits(); - } -} - #[cfg(all(test, feature = "regex-engine"))] mod tests; #[cfg(all(test, feature = "regex-engine"))] +mod tests_cache; +#[cfg(all(test, feature = "regex-engine"))] +mod tests_header; +#[cfg(all(test, feature = "regex-engine"))] mod tests_part2; diff --git a/crates/perry-runtime/src/regex/census_rows.rs b/crates/perry-runtime/src/regex/census_rows.rs new file mode 100644 index 0000000000..7418484397 --- /dev/null +++ b/crates/perry-runtime/src/regex/census_rows.rs @@ -0,0 +1,444 @@ +//! Diagnostic-only `PERRY_GC_CENSUS` rows for RegExp-owned Rust tables. +//! +//! Nothing in this module is called from construction, matching, collection, +//! or cache maintenance. The census enters it only after a request has armed a +//! synchronous full collection. Engine crates do not expose the size of the +//! heap graph behind their public `Regex` values, so program bytes are an +//! explicitly labelled opaque lower-bound: the `Arc` allocation, public value, +//! and source/capture buffers that can be observed without unsafe layout +//! assumptions. + +use std::collections::HashSet; +use std::sync::Arc; + +use super::*; + +pub(crate) struct RegexCensusRow { + pub(crate) table: &'static str, + pub(crate) entries: usize, + pub(crate) bytes: usize, + fields: serde_json::Map, +} + +impl RegexCensusRow { + fn new(table: &'static str, entries: usize, bytes: usize) -> Self { + Self { + table, + entries, + bytes, + fields: serde_json::Map::new(), + } + } + + fn usize(mut self, name: &'static str, value: usize) -> Self { + self.fields.insert(name.into(), serde_json::json!(value)); + self + } + + fn u64(mut self, name: &'static str, value: u64) -> Self { + self.fields.insert(name.into(), serde_json::json!(value)); + self + } + + fn bool(mut self, name: &'static str, value: bool) -> Self { + self.fields.insert(name.into(), serde_json::json!(value)); + self + } + + fn text(mut self, name: &'static str, value: &'static str) -> Self { + self.fields.insert(name.into(), serde_json::json!(value)); + self + } + + pub(crate) fn json(&self) -> serde_json::Value { + let mut value = serde_json::Map::new(); + value.insert("table".into(), serde_json::json!(self.table)); + value.insert("entries".into(), serde_json::json!(self.entries)); + value.insert("bytes".into(), serde_json::json!(self.bytes)); + value.extend(self.fields.clone()); + serde_json::Value::Object(value) + } +} + +pub(crate) struct RegexCensusSnapshot { + pub(crate) rows: Vec, + /// Built independently of JSON serialization. If a row is accidentally + /// omitted from the emitted array, the reconciliation test sees the gap. + pub(crate) attributed_bytes: usize, +} + +#[cfg(test)] +static TEST_CENSUS_WALKS: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); + +#[cfg(test)] +pub(crate) fn test_reset_walks() { + TEST_CENSUS_WALKS.store(0, std::sync::atomic::Ordering::Relaxed); +} + +#[cfg(test)] +pub(crate) fn test_walks() -> usize { + TEST_CENSUS_WALKS.load(std::sync::atomic::Ordering::Relaxed) +} + +#[inline] +fn arc_allocation_bytes() -> usize { + // Two strong/weak counters precede the Arc payload in today's allocator + // representation. This is an estimate, not a promise about Arc layout. + 2 * std::mem::size_of::() + std::mem::size_of::() +} + +fn standard_program_bytes(program: ®ex::Regex) -> usize { + arc_allocation_bytes::() + program.as_str().len() +} + +fn fancy_program_bytes(program: &fancy_regex::Regex) -> usize { + arc_allocation_bytes::() + program.as_str().len() +} + +fn repeat_program_bytes(program: &repeat_matcher::RepeatMatcherRegex) -> usize { + arc_allocation_bytes::() + program.census_buffer_bytes() +} + +fn program_bundle_bytes( + bundle_ptrs: impl IntoIterator, + standard_skip: &HashSet, + fancy_skip: &HashSet, + repeat_skip: &HashSet, +) -> usize { + let mut standard_seen = standard_skip.clone(); + let mut fancy_seen = fancy_skip.clone(); + let mut repeat_seen = repeat_skip.clone(); + let mut bytes = 0usize; + for ptr in bundle_ptrs { + let programs = unsafe { &*(ptr as *const site_cache::Programs) }; + bytes += arc_allocation_bytes::(); + if standard_seen.insert(Arc::as_ptr(&programs.std) as usize) { + bytes += standard_program_bytes(&programs.std); + } + if let Some(program) = &programs.fancy { + if fancy_seen.insert(Arc::as_ptr(program) as usize) { + bytes += fancy_program_bytes(program); + } + } + if let Some(program) = &programs.repeat { + if repeat_seen.insert(Arc::as_ptr(program) as usize) { + bytes += repeat_program_bytes(program); + } + } + } + bytes +} + +fn pointer_row() -> RegexCensusRow { + REGEX_POINTERS.with(|table| { + let table = table.borrow(); + let live_headers = table + .iter() + .filter(|&&addr| unsafe { + crate::value::addr_class::try_read_gc_header(addr).is_some_and(|header| { + header.obj_type == crate::gc::GC_TYPE_REGEXP + && header.gc_flags & (crate::gc::GC_FLAG_MARKED | crate::gc::GC_FLAG_PINNED) + != 0 + }) + }) + .count(); + RegexCensusRow::new( + "regex.pointers", + table.len(), + crate::gc::census::set_bytes(&*table), + ) + .usize("live_headers", live_headers) + }) +} + +fn standard_cache_row() -> RegexCensusRow { + REGEX_CACHE.with(|cache| { + let cache = cache.borrow(); + let programs = cache + .values() + .map(|program| (Arc::as_ptr(program) as usize, program)) + .collect::>(); + let opaque = programs + .values() + .map(|program| standard_program_bytes(program)) + .sum::(); + let mut cleared = 0; + let mut evictions = 0; + if crate::hot_diag::regex_on() { + crate::hot_diag::regex_counters(|diag| { + cleared = diag.cache_clears; + evictions = diag.cache_evictions; + }); + } + RegexCensusRow::new( + "regex.program_cache", + cache.len(), + crate::gc::census::map_bytes(&*cache) + opaque, + ) + .usize("compiled_programs", programs.len()) + .usize("opaque_program_bytes", opaque) + .text("program_bytes_estimate", "opaque_inline_lower_bound") + .bool("program_bytes_inside_side_table_bytes", true) + .u64("cleared", cleared) + .u64("evictions", evictions) + .text("cache_event_scope", "all_regex_caches") + }) +} + +fn fancy_cache_row() -> RegexCensusRow { + FANCY_CACHE.with(|cache| { + let cache = cache.borrow(); + let programs = cache + .values() + .map(|program| (Arc::as_ptr(program) as usize, program)) + .collect::>(); + let opaque = programs + .values() + .map(|program| fancy_program_bytes(program)) + .sum::(); + RegexCensusRow::new( + "regex.fancy_cache", + cache.len(), + crate::gc::census::map_bytes(&*cache) + opaque, + ) + .usize("compiled_programs", programs.len()) + .usize("opaque_program_bytes", opaque) + .text("program_bytes_estimate", "opaque_inline_lower_bound") + .bool("program_bytes_inside_side_table_bytes", true) + }) +} + +fn repeat_cache_row() -> RegexCensusRow { + REPEAT_MATCHER_CACHE.with(|cache| { + let cache = cache.borrow(); + let programs = cache + .values() + .map(|program| (Arc::as_ptr(program) as usize, program)) + .collect::>(); + let opaque = programs + .values() + .map(|program| repeat_program_bytes(program)) + .sum::(); + RegexCensusRow::new( + "regex.repeat_cache", + cache.len(), + crate::gc::census::map_bytes(&*cache) + opaque, + ) + .usize("compiled_programs", programs.len()) + .usize("opaque_program_bytes", opaque) + .text("program_bytes_estimate", "opaque_inline_lower_bound") + .bool("program_bytes_inside_side_table_bytes", true) + }) +} + +fn validation_cache_row() -> RegexCensusRow { + VALIDATED_PATTERNS.with(|cache| { + let cache = cache.borrow(); + let text_bytes = cache + .keys() + .map(|(pattern, flags)| pattern.capacity() + flags.capacity()) + .sum::(); + RegexCensusRow::new( + "regex.validated_patterns", + cache.len(), + crate::gc::census::map_bytes(&*cache) + text_bytes, + ) + .usize("text_bytes", text_bytes) + }) +} + +fn matcher_kind_row() -> RegexCensusRow { + let mut counts = [0usize; 4]; + REGEX_POINTERS.with(|table| { + for &addr in table.borrow().iter() { + let re = addr as *const RegExpHeader; + if !is_valid_regex_ptr(re) { + continue; + } + let index = unsafe { + match (*re).matcher_kind { + MatcherKind::Unbuilt => 0, + MatcherKind::Standard => 1, + MatcherKind::Fancy => 2, + MatcherKind::Repeat => 3, + } + }; + counts[index] += 1; + } + }); + RegexCensusRow::new("regex.matcher_kinds", counts.iter().sum(), 0) + .usize("unbuilt", counts[0]) + .usize("standard", counts[1]) + .usize("fancy", counts[2]) + .usize("repeat", counts[3]) + .bool("bytes_inside_side_table_bytes", false) + .text("storage", "RegExpHeader.matcher_kind") +} + +fn content_row( + standard_cached: &HashSet, + fancy_cached: &HashSet, + repeat_cached: &HashSet, +) -> RegexCensusRow { + let (entries, table_bytes, bundle_ptrs) = site_cache::census_parts(); + let opaque = program_bundle_bytes( + bundle_ptrs.iter().copied(), + standard_cached, + fancy_cached, + repeat_cached, + ); + RegexCensusRow::new("regex.content_cache", entries, table_bytes + opaque) + .usize("pinned_programs", bundle_ptrs.len()) + .usize("opaque_program_bytes", opaque) + .text("program_bytes_estimate", "opaque_inline_lower_bound") + .bool("program_bytes_inside_side_table_bytes", true) +} + +fn site_table_row( + content_bundles: &HashSet, + standard_cached: &HashSet, + fancy_cached: &HashSet, + repeat_cached: &HashSet, +) -> RegexCensusRow { + let (sites, table_bytes, header_ptrs, bundle_ptrs) = site_test::census_parts(); + let rooted_headers = header_ptrs.len(); + let exclusive = bundle_ptrs + .iter() + .copied() + .filter(|ptr| !content_bundles.contains(ptr)) + .collect::>(); + let attributed_program_bytes = program_bundle_bytes( + exclusive.iter().copied(), + standard_cached, + fancy_cached, + repeat_cached, + ); + let pinned_program_bytes = program_bundle_bytes( + bundle_ptrs.iter().copied(), + &HashSet::new(), + &HashSet::new(), + &HashSet::new(), + ); + RegexCensusRow::new( + "regex.site_table", + sites, + table_bytes + attributed_program_bytes, + ) + .usize("sites", sites) + .usize("rooted_headers", rooted_headers) + .usize( + "rooted_header_bytes", + rooted_headers * std::mem::size_of::(), + ) + .bool("rooted_header_bytes_inside_side_table_bytes", false) + .usize("pinned_programs", bundle_ptrs.len()) + .usize("exclusively_attributed_programs", exclusive.len()) + .usize("pinned_program_bytes", pinned_program_bytes) + .usize("attributed_program_bytes", attributed_program_bytes) + .text("program_bytes_estimate", "opaque_inline_lower_bound") + .bool("pinned_program_bytes_inside_side_table_bytes", false) + .bool("attributed_program_bytes_inside_side_table_bytes", true) +} + +fn literal_site_row() -> RegexCensusRow { + let (sites, bytes) = site_key::census_parts(); + RegexCensusRow::new("regex.literal_sites", sites, bytes).usize("sites", sites) +} + +fn active_factory_row() -> RegexCensusRow { + let (entries, bytes) = site_test::active_factory_census_parts(); + RegexCensusRow::new("regex.active_factory_sites", entries, bytes) +} + +fn expando_row() -> RegexCensusRow { + let (owners, properties, bytes) = crate::object::exotic_expando::regex_expando_census(); + RegexCensusRow::new("regex.expando_owners", owners, bytes) + .usize("owners", owners) + .usize("properties", properties) +} + +/// Snapshot every RegExp-owned table only when the census requests it. +pub(crate) fn census_snapshot() -> RegexCensusSnapshot { + #[cfg(test)] + TEST_CENSUS_WALKS.fetch_add(1, std::sync::atomic::Ordering::Relaxed); + + let standard_cached = REGEX_CACHE.with(|cache| { + cache + .borrow() + .values() + .map(|program| Arc::as_ptr(program) as usize) + .collect::>() + }); + let fancy_cached = FANCY_CACHE.with(|cache| { + cache + .borrow() + .values() + .map(|program| Arc::as_ptr(program) as usize) + .collect::>() + }); + let repeat_cached = REPEAT_MATCHER_CACHE.with(|cache| { + cache + .borrow() + .values() + .map(|program| Arc::as_ptr(program) as usize) + .collect::>() + }); + let content_bundles = site_cache::census_program_ptrs(); + + let rows = vec![ + pointer_row(), + standard_cache_row(), + fancy_cache_row(), + repeat_cache_row(), + validation_cache_row(), + content_row(&standard_cached, &fancy_cached, &repeat_cached), + literal_site_row(), + site_table_row( + &content_bundles, + &standard_cached, + &fancy_cached, + &repeat_cached, + ), + active_factory_row(), + expando_row(), + matcher_kind_row(), + ]; + + // Deliberately independent from JSON row registration below: this second + // diagnostic walk is what makes an omitted row fail reconciliation. + let attributed_bytes = pointer_row().bytes + + standard_cache_row().bytes + + fancy_cache_row().bytes + + repeat_cache_row().bytes + + validation_cache_row().bytes + + content_row(&standard_cached, &fancy_cached, &repeat_cached).bytes + + literal_site_row().bytes + + site_table_row( + &content_bundles, + &standard_cached, + &fancy_cached, + &repeat_cached, + ) + .bytes + + active_factory_row().bytes + + expando_row().bytes + + matcher_kind_row().bytes; + + RegexCensusSnapshot { + rows, + attributed_bytes, + } +} + +#[cfg(test)] +pub(crate) fn test_reset_tables() { + REGEX_POINTERS.with(|table| table.borrow_mut().clear()); + REGEX_CACHE.with(|cache| cache.borrow_mut().clear()); + FANCY_CACHE.with(|cache| cache.borrow_mut().clear()); + REPEAT_MATCHER_CACHE.with(|cache| cache.borrow_mut().clear()); + VALIDATED_PATTERNS.with(|cache| cache.borrow_mut().clear()); + site_cache::test_reset(); + site_key::test_reset(); + site_test::test_reset(); + test_reset_walks(); +} diff --git a/crates/perry-runtime/src/regex/compile.rs b/crates/perry-runtime/src/regex/compile.rs index f8472f50d6..cb0aae6e43 100644 --- a/crates/perry-runtime/src/regex/compile.rs +++ b/crates/perry-runtime/src/regex/compile.rs @@ -4,8 +4,6 @@ use std::sync::Arc; -use regex::Regex; - use super::class_range_validate::has_out_of_order_double_dash_class_range; use super::grammar::{ has_invalid_repeated_quantifier, has_unicode_forbidden_legacy_escape, @@ -145,34 +143,31 @@ pub extern "C" fn js_regexp_compile_value( )); } - // The header OWNS raw `Arc` references to its compiled program(s) + // The header OWNS one raw `Arc` reference to its compiled program set // (mirrors `js_regexp_new`), so the capped `REGEX_CACHE`/`FANCY_CACHE` // (see `REGEX_CACHE_MAX_ENTRIES`) can evict without invalidating this - // receiver. Refresh `fancy_ptr` too — it must track the NEW pattern, not - // the one the receiver was constructed with. + // receiver. Refresh the whole program set so it tracks the NEW pattern, + // not the one the receiver was constructed with. // `RegExp.prototype.compile` re-initialises an existing receiver — once per // call from user code, not per object — so materialising the shared key - // here costs nothing measurable, and the same `Arc`s go into the source - // table below. + // here costs nothing measurable. let pattern_key: std::sync::Arc = std::sync::Arc::from(pattern_str); let flags_key: std::sync::Arc = std::sync::Arc::from(flags_str); - let arc = get_or_compile_regex(&pattern_key, &flags_key); - let regex_ptr = Arc::into_raw(arc) as *mut Regex; - let fancy_ptr: *const () = super::FANCY_CACHE.with(|fc| { - match fc.borrow().get(&(pattern_key.clone(), flags_key.clone())) { - Some(arc) => Arc::into_raw(arc.clone()) as *const (), - None => std::ptr::null(), - } + let std = get_or_compile_regex(&pattern_key, &flags_key); + let fancy = super::FANCY_CACHE.with(|fc| { + fc.borrow() + .get(&(pattern_key.clone(), flags_key.clone())) + .cloned() }); - let repeat_matcher_ptr: *const () = super::REPEAT_MATCHER_CACHE.with(|cache| { - match cache + let repeat = super::REPEAT_MATCHER_CACHE.with(|cache| { + cache .borrow() .get(&(pattern_key.clone(), flags_key.clone())) - { - Some(arc) => Arc::into_raw(arc.clone()) as *const (), - None => std::ptr::null(), - } + .cloned() }); + let programs = Arc::new(super::site_cache::Programs { std, fancy, repeat }); + let matcher_kind = programs.matcher_kind(); + let programs_ptr = Arc::into_raw(programs); let (canonical_flags_ptr, _) = re_handle.across_mut::(|| js_string_from_str(flags_str)); let canonical_flags_handle = scope.root_string_ptr(canonical_flags_ptr); @@ -181,28 +176,31 @@ pub extern "C" fn js_regexp_compile_value( .across_const::(|| js_string_from_str(pattern_str)) }); unsafe { - let old_regex_ptr = (*re).regex_ptr; - let old_fancy_ptr = (*re).fancy_ptr; - let old_repeat_matcher_ptr = (*re).repeat_matcher_ptr; - (*re).regex_ptr = regex_ptr; - (*re).fancy_ptr = fancy_ptr; - (*re).repeat_matcher_ptr = repeat_matcher_ptr; + let old_programs_ptr = (*re).programs_ptr; + (*re).matcher_kind = matcher_kind; + (*re).programs_ptr = programs_ptr; // Release the receiver's PREVIOUS owned references now that the new // ones are installed (recompiling the same pattern is fine: the fresh // `into_raw` reference above keeps the shared program alive). - if !old_regex_ptr.is_null() { - drop(Arc::from_raw(old_regex_ptr as *const Regex)); - } - if !old_fancy_ptr.is_null() { - drop(Arc::from_raw(old_fancy_ptr as *const fancy_regex::Regex)); - } - if !old_repeat_matcher_ptr.is_null() { - drop(Arc::from_raw( - old_repeat_matcher_ptr as *const super::repeat_matcher::RepeatMatcherRegex, - )); + if !old_programs_ptr.is_null() { + drop(Arc::from_raw(old_programs_ptr)); } (*re).pattern_ptr = pattern_ptr; (*re).flags_ptr = canonical_flags_ptr; + // These are traced header edges. Unlike construction, `compile` can + // rewrite a tenured receiver with newly allocated nursery strings, so + // both stores need the ordinary runtime barrier. + let parent = re as usize; + crate::gc::runtime_write_barrier_gc_slot( + parent, + std::ptr::addr_of!((*re).pattern_ptr) as usize, + crate::value::js_nanbox_string(pattern_ptr as i64).to_bits(), + ); + crate::gc::runtime_write_barrier_gc_slot( + parent, + std::ptr::addr_of!((*re).flags_ptr) as usize, + crate::value::js_nanbox_string(canonical_flags_ptr as i64).to_bits(), + ); (*re).case_insensitive = flags_str.contains('i'); (*re).global = flags_str.contains('g'); (*re).multiline = flags_str.contains('m'); @@ -210,10 +208,6 @@ pub extern "C" fn js_regexp_compile_value( (*re).dot_all = flags_str.contains('s'); (*re).unicode = flags_str.contains('u') || flags_str.contains('v'); (*re).has_indices = flags_str.contains('d'); - super::REGEX_SOURCE_TABLE.with(|t| { - t.borrow_mut() - .insert(re as usize, (Arc::from(pattern_str), Arc::from(flags_str))); - }); } // Spec RegExpInitialize step 12: `Set(obj, "lastIndex", 0, true)` runs LAST, // with the *Throw* flag. A user-frozen `lastIndex` diff --git a/crates/perry-runtime/src/regex/compile_cache.rs b/crates/perry-runtime/src/regex/compile_cache.rs index bde70bc77f..4a4270aeb6 100644 --- a/crates/perry-runtime/src/regex/compile_cache.rs +++ b/crates/perry-runtime/src/regex/compile_cache.rs @@ -86,26 +86,27 @@ pub(crate) fn build_fancy_regex(pattern: &str) -> Result(cache: &mut HashMap) { +pub(crate) fn evict_regex_cache_if_full( + cache: &mut HashMap, +) { if cache.len() >= REGEX_CACHE_MAX_ENTRIES { - cache.clear(); + let victim = cache.keys().next().cloned(); + if let Some(victim) = victim { + cache.remove(&victim); + } + #[cfg(test)] + super::tests_cache::note_cache_eviction(); if crate::hot_diag::regex_on() { - crate::hot_diag::regex_with(|d| d.cache_clears += 1); + crate::hot_diag::regex_counters(|d| d.cache_evictions += 1); } } } @@ -131,7 +132,7 @@ pub(crate) fn evict_regex_cache_if_full(cache: &mut HashMap) { /// One shared never-match program per thread. /// /// Only used by the `PERRY_REGEX_ENGINE=regress` measurement path, where every -/// pattern needs a value in `regex_ptr` (the built/not-built flag) but no NFA: +/// pattern needs a value in `programs_ptr` (the built/not-built flag) but no NFA: /// building a fresh one per pattern would be exactly the compile cost the /// experiment exists to remove from the measurement. #[cfg(feature = "regex-engine")] @@ -173,7 +174,7 @@ pub(crate) fn compile_and_cache_regex_checked(pattern: &Arc, flags: &Arc f64 { #[used] static KEEP_REGEXP_ESCAPE: extern "C" fn(f64) -> f64 = js_regexp_escape; -/// ECMA-262 22.2.6.10 EscapeRegExpPattern: produce a string that, placed -/// between two `/` characters, parses as the same pattern. An empty pattern -/// becomes `"(?:)"`; an unescaped `/` outside a character class becomes `\/`; -/// the four LineTerminators become their `\n`/`\r`/`
`/`
` escapes -/// (even inside a character class). A backslash escapes the following code -/// point, which is copied verbatim. -pub(super) fn escape_regexp_source(pattern: &str) -> String { +/// ECMA-262 22.2.6.10 EscapeRegExpPattern for a valid UTF-8 pattern. +fn escape_regexp_source_utf8(pattern: &str) -> String { if pattern.is_empty() { return "(?:)".to_string(); } @@ -183,3 +178,73 @@ pub(super) fn escape_regexp_source(pattern: &str) -> String { } out } + +/// ECMA-262 22.2.6.10 EscapeRegExpPattern: produce WTF-8 bytes that, placed +/// between two `/` characters, parse as the same pattern. JavaScript strings +/// may contain lone UTF-16 surrogates, represented by Perry as WTF-8; those +/// bytes must round-trip rather than pass through Rust's `str::chars()`. +pub(super) fn escape_regexp_source(pattern: &[u8]) -> Vec { + if let Ok(pattern) = std::str::from_utf8(pattern) { + return escape_regexp_source_utf8(pattern).into_bytes(); + } + if pattern.is_empty() { + return b"(?:)".to_vec(); + } + + let mut out = Vec::with_capacity(pattern.len() + 2); + let mut in_class = false; + let mut i = 0; + while i < pattern.len() { + match pattern[i] { + b'\\' => { + out.push(b'\\'); + i += 1; + if i < pattern.len() { + let (advance, _, _) = crate::string::wtf8_step(pattern, i); + let end = i.saturating_add(advance).min(pattern.len()); + out.extend_from_slice(&pattern[i..end]); + i = end; + } + } + b'[' if !in_class => { + in_class = true; + out.push(b'['); + i += 1; + } + b']' if in_class => { + in_class = false; + out.push(b']'); + i += 1; + } + b'/' if !in_class => { + out.extend_from_slice(b"\\/"); + i += 1; + } + b'\n' => { + out.extend_from_slice(b"\\n"); + i += 1; + } + b'\r' => { + out.extend_from_slice(b"\\r"); + i += 1; + } + 0xE2 if pattern.get(i..i + 3) == Some(&[0xE2, 0x80, 0xA8]) + || pattern.get(i..i + 3) == Some(&[0xE2, 0x80, 0xA9]) => + { + out.extend_from_slice(if pattern[i + 2] == 0xA8 { + b"\\u2028" + } else { + b"\\u2029" + }); + i += 3; + } + _ => { + let (advance, _, _) = crate::string::wtf8_step(pattern, i); + let end = i.saturating_add(advance).min(pattern.len()); + out.extend_from_slice(&pattern[i..end]); + i = end; + } + } + } + out +} diff --git a/crates/perry-runtime/src/regex/lazy.rs b/crates/perry-runtime/src/regex/lazy.rs index f5492c317f..fbc583eb19 100644 --- a/crates/perry-runtime/src/regex/lazy.rs +++ b/crates/perry-runtime/src/regex/lazy.rs @@ -37,14 +37,13 @@ //! lookbehind/backreferences still decides, and still throws when both //! engines refuse); //! * `.source` / `.flags` / `.global` / `.sticky` / `lastIndex` are header -//! and side-table reads that never touched the compiled program; +//! reads that never touched the compiled program; //! * identity is untouched — `js_regexp_new` still allocates a fresh header //! per evaluation. //! //! The build itself happens on the first operation that needs a matcher, -//! through [`ensure_regex_compiled`], and installs exactly the pointers -//! `js_regexp_new` used to install eagerly (`regex_ptr`, `fancy_ptr`, -//! `repeat_matcher_ptr`), each a leaked `Arc` the header owns. +//! through [`ensure_regex_compiled`], and installs one leaked `Arc` to the +//! shared standard/fancy/repeat program set. use std::sync::Arc; @@ -53,8 +52,7 @@ use regex::Regex; use super::grammar::{collapse_redos_guard_quantifiers, js_regex_to_rust_with_flags}; use super::{ evict_regex_cache_if_full, get_or_compile_regex, is_valid_ptr, is_valid_regex_ptr, - string_as_str, RegExpHeader, FANCY_CACHE, REGEX_SOURCE_TABLE, REPEAT_MATCHER_CACHE, - VALIDATED_PATTERNS, + string_as_str, RegExpHeader, FANCY_CACHE, REPEAT_MATCHER_CACHE, VALIDATED_PATTERNS, }; /// The exact string `build_std_regex` is handed for `(pattern, flags)`: the @@ -160,15 +158,10 @@ pub(super) fn mark_pattern_validated(pattern: &str, flags: &str) { /// The `(source, flags)` a header was built from. /// -/// Prefers the GC-survivable side table (issue #637) and falls back to the -/// header's own string payloads, which — unlike the thread-local table — are -/// readable from a second statically-linked copy of the runtime (Wall 18). +/// Since #9845 the header's string slots are traced GC edges, so the payloads +/// are both collection-safe and readable from a second statically-linked copy +/// of the runtime (Wall 18). pub(super) fn source_and_flags(re: *const RegExpHeader) -> (Arc, Arc) { - if let Some(source) = - REGEX_SOURCE_TABLE.with(|table| table.borrow().get(&(re as usize)).cloned()) - { - return source; - } unsafe { let pattern: Arc = if is_valid_ptr((*re).pattern_ptr) { Arc::from(string_as_str((*re).pattern_ptr)) @@ -186,16 +179,12 @@ pub(super) fn source_and_flags(re: *const RegExpHeader) -> (Arc, Arc) /// Build this header's compiled program(s) if it has none yet. /// -/// `regex_ptr == null` is the "not built yet" state. It is published LAST so -/// a header is never observable as built while `fancy_ptr` / -/// `repeat_matcher_ptr` are still stale — every reader that consults those -/// two goes through [`lookup_fancy_regex`](super::lookup_fancy_regex) / -/// `lookup_repeat_matcher`, which call this first. +/// `programs_ptr == null` is the "not built yet" state. The one-pointer +/// publication keeps the three engines coherent. /// -/// The header OWNS a leaked `Arc` reference to each program (mirroring what -/// `js_regexp_new` used to do inline), so the capped `REGEX_CACHE` / -/// `FANCY_CACHE` / `REPEAT_MATCHER_CACHE` can evict without invalidating a -/// live receiver. +/// The header OWNS one leaked `Arc` to the complete program set, so the capped +/// `REGEX_CACHE` / `FANCY_CACHE` / `REPEAT_MATCHER_CACHE` can evict without +/// invalidating a live receiver. /// /// Contains no JS allocation and cannot re-enter the interpreter, so it is /// safe to call from inside a phase that holds a borrow of a GC string. @@ -215,7 +204,7 @@ pub(crate) fn ensure_regex_compiled(re: *const RegExpHeader) { if !is_valid_ptr(re) { return; } - if unsafe { !(*re).regex_ptr.is_null() } { + if unsafe { !(*re).programs_ptr.is_null() } { return; } build_and_install_programs(re); @@ -228,6 +217,8 @@ fn build_and_install_programs(re: *const RegExpHeader) { if !is_valid_regex_ptr(re) { return; } + #[cfg(test)] + crate::hot_diag::test_note_regex_program_build(); let (pattern, flags) = source_and_flags(re); if crate::hot_diag::regex_on() { let cache_hit = super::REGEX_CACHE.with(|cache| { @@ -254,14 +245,12 @@ fn build_and_install_programs(re: *const RegExpHeader) { }); // ── Repair before publishing ────────────────────────────────────────── // - // A built header is treated as AUTHORITATIVE — `lookup_fancy_regex` / - // `lookup_repeat_matcher` read a null slot beside a non-null `regex_ptr` - // as "this pattern has no such program" — and `install_programs` below + // A built header is treated as AUTHORITATIVE, and `install_programs` below // memoizes the triple against the pattern text, so whatever is assembled // here becomes the answer for every later construction of the same // literal. It therefore has to be complete, and the probes above cannot // guarantee that on their own: the three caches are capped independently - // and each `clear()`s wholesale, while + // and each can evict a different entry, while // `compile_and_cache_regex_checked` returns early whenever `REGEX_CACHE` // already holds the pattern — so it never re-runs the fancy or // repeat-matcher build for a pattern whose `REGEX_CACHE` entry survived a @@ -306,32 +295,32 @@ fn build_and_install_programs(re: *const RegExpHeader) { // Remember the built programs against the pattern text, so the next // construction of the same literal is born built (`js_regexp_new`). - super::site_cache::install_programs( - &pattern, - &flags, - super::site_cache::Programs { - std: std_arc.clone(), - fancy: fancy_arc.clone(), - repeat: repeat_arc.clone(), - }, - ); - let regex_ptr = Arc::into_raw(std_arc) as *mut Regex; - let fancy_ptr: *const () = - fancy_arc.map_or(std::ptr::null(), |arc| Arc::into_raw(arc) as *const ()); - let repeat_matcher_ptr: *const () = - repeat_arc.map_or(std::ptr::null(), |arc| Arc::into_raw(arc) as *const ()); + let programs = Arc::new(super::site_cache::Programs { + std: std_arc.clone(), + fancy: fancy_arc.clone(), + repeat: repeat_arc.clone(), + }); + super::site_cache::install_programs(&pattern, &flags, programs.clone()); unsafe { let re = re as *mut RegExpHeader; - (*re).fancy_ptr = fancy_ptr; - (*re).repeat_matcher_ptr = repeat_matcher_ptr; - // Publish last: `regex_ptr` is the built/not-built flag. - (*re).regex_ptr = regex_ptr; + (*re).matcher_kind = programs.matcher_kind(); + (*re).programs_ptr = Arc::into_raw(programs); } } +#[cfg(test)] +pub(super) fn test_reset_program_builds() { + crate::hot_diag::test_reset_regex_builds_and_evictions(); +} + +#[cfg(test)] +pub(super) fn test_program_builds() -> u64 { + crate::hot_diag::test_regex_builds_and_evictions().0 +} + /// The header's standard-engine program, building it on first use. /// -/// Every `&*(*re).regex_ptr` in the tree goes through here — the field is +/// Every standard-program borrow in the tree goes through here — the field is /// null until something needs a matcher. /// /// # Safety @@ -340,5 +329,5 @@ fn build_and_install_programs(re: *const RegExpHeader) { /// header owns until its GC finalizer runs. pub(crate) unsafe fn header_std_regex<'a>(re: *const RegExpHeader) -> &'a Regex { ensure_regex_compiled(re); - &*(*re).regex_ptr + &(*(*re).programs_ptr).std } diff --git a/crates/perry-runtime/src/regex/match_all.rs b/crates/perry-runtime/src/regex/match_all.rs index 998e2ad83f..f533dd4955 100644 --- a/crates/perry-runtime/src/regex/match_all.rs +++ b/crates/perry-runtime/src/regex/match_all.rs @@ -87,7 +87,7 @@ unsafe fn materialize_match_all_results( // Phase 1 (borrowing, no JS allocation): snapshot every match into owned // Rust data. The fancy-regex fallback (lookbehind/backreferences) is - // needed because the never-match placeholder in `regex_ptr` would yield + // needed because the never-match standard program would yield // an empty iterator otherwise. // The scan starts AT `search_start` inside the whole subject — never on a // `&str_data[search_start..]` slice, which would strip the context every diff --git a/crates/perry-runtime/src/regex/program_key.rs b/crates/perry-runtime/src/regex/program_key.rs index e753fe0f69..45852319cb 100644 --- a/crates/perry-runtime/src/regex/program_key.rs +++ b/crates/perry-runtime/src/regex/program_key.rs @@ -29,9 +29,8 @@ pub(crate) const NEVER_MATCH_PATTERN: &str = r"[^\s\S]"; /// 1,984 MB), which is what `.to_string()` on an `Arc` lowers to. /// /// Keying by `Arc` makes a probe two refcount increments and no -/// allocation: every caller that matters already holds those `Arc`s, because -/// `REGEX_SOURCE_TABLE` and `regex::site_cache` share one allocation of a -/// literal's text with every header built from it. Hashing still walks the +/// allocation: every caller that matters already holds those `Arc`s through +/// `regex::site_cache`. Hashing still walks the /// pattern bytes — the allocation is what the census measured, and what this /// removes. #[cfg(feature = "regex-engine")] diff --git a/crates/perry-runtime/src/regex/properties.rs b/crates/perry-runtime/src/regex/properties.rs new file mode 100644 index 0000000000..e4f0a53f3e --- /dev/null +++ b/crates/perry-runtime/src/regex/properties.rs @@ -0,0 +1,78 @@ +//! Observable RegExp data properties and stringification. + +use super::escape::escape_regexp_source; +use super::RegExpHeader; +use super::{is_valid_ptr, is_valid_regex_ptr, js_string_from_str, string_as_bytes, string_as_str}; +use crate::string::StringHeader; + +/// Get regex.source — returns the pattern string. +#[no_mangle] +pub extern "C" fn js_regexp_get_source(re: *const RegExpHeader) -> *mut StringHeader { + if !is_valid_regex_ptr(re) { + return js_string_from_str("(?:)"); + } + unsafe { + if is_valid_ptr((*re).pattern_ptr) { + let escaped = escape_regexp_source(string_as_bytes((*re).pattern_ptr)); + crate::string::js_string_from_wtf8_bytes(escaped.as_ptr(), escaped.len() as u32) + } else { + js_string_from_str("(?:)") + } + } +} + +/// `RegExp.prototype.source` for the prototype object itself (no +/// `[[OriginalSource]]`) returns the canonical empty source `"(?:)"`. +#[no_mangle] +pub extern "C" fn js_regexp_empty_source() -> *mut StringHeader { + js_string_from_str("(?:)") +} + +/// Get regex.flags — returns the flags string. +#[no_mangle] +pub extern "C" fn js_regexp_get_flags(re: *const RegExpHeader) -> *mut StringHeader { + if !is_valid_regex_ptr(re) { + return js_string_from_str(""); + } + unsafe { + if is_valid_ptr((*re).flags_ptr) { + let flags_str = string_as_str((*re).flags_ptr); + js_string_from_str(flags_str) + } else { + js_string_from_str("") + } + } +} + +/// `RegExp.prototype.toString()` — `/source/flags`. Used by both the +/// `regex.toString()` method dispatch and ToString coercion (`String(re)`, +/// template literals). Node never produces `"[object Object]"` for a RegExp. +#[no_mangle] +pub extern "C" fn js_regexp_to_string(re: *const RegExpHeader) -> *mut StringHeader { + let src = js_regexp_get_source(re); + let flg = js_regexp_get_flags(re); + let out = format!("/{}/{}", string_as_str(src), string_as_str(flg)); + js_string_from_str(&out) +} + +/// Get regex.lastIndex — returns the stored value (NaN-boxed JSValue bits as +/// f64). Usually a number, but `re.lastIndex = obj` round-trips the object. +#[no_mangle] +pub extern "C" fn js_regexp_get_last_index(re: *const RegExpHeader) -> f64 { + if !is_valid_regex_ptr(re) { + return 0.0; + } + unsafe { f64::from_bits((*re).last_index) } +} + +/// Set regex.lastIndex — stores the value verbatim (no coercion on write, per +/// spec `Set(R, "lastIndex", v)`). +#[no_mangle] +pub extern "C" fn js_regexp_set_last_index(re: *mut RegExpHeader, value: f64) { + if !is_valid_regex_ptr(re) { + return; + } + unsafe { + (*re).last_index = value.to_bits(); + } +} diff --git a/crates/perry-runtime/src/regex/repeat_matcher.rs b/crates/perry-runtime/src/regex/repeat_matcher.rs index da008816de..64884c7d2f 100644 --- a/crates/perry-runtime/src/regex/repeat_matcher.rs +++ b/crates/perry-runtime/src/regex/repeat_matcher.rs @@ -15,6 +15,18 @@ pub(super) struct RepeatMatcherRegex { } impl RepeatMatcherRegex { + /// Census-only visible buffers. `regress::Regex` does not expose its + /// compiled heap graph, so callers label this as an opaque lower bound. + pub(super) fn census_buffer_bytes(&self) -> usize { + self.capture_names.capacity() * std::mem::size_of::>() + + self + .capture_names + .iter() + .flatten() + .map(String::capacity) + .sum::() + } + fn named_group_range( &self, matched: ®ress::Match, diff --git a/crates/perry-runtime/src/regex/replace_expand.rs b/crates/perry-runtime/src/regex/replace_expand.rs index 70b1931f61..19214cce1a 100644 --- a/crates/perry-runtime/src/regex/replace_expand.rs +++ b/crates/perry-runtime/src/regex/replace_expand.rs @@ -374,7 +374,7 @@ pub extern "C" fn js_string_replace_regex_fn( // If the `regex` crate couldn't compile this pattern (lookahead, // backreferences, …), `get_or_compile_regex` stashed a never-match - // placeholder in `(*re).regex_ptr` and the real pattern in + // placeholder in the header's standard program and the real pattern in // `FANCY_CACHE`. Route the callback-replace through fancy-regex so the // callback actually fires — otherwise `captures_iter` below would // silently match nothing and return the input unchanged. (get-intrinsic's @@ -485,7 +485,7 @@ pub extern "C" fn js_string_replace_regex_named( // Fancy-regex fallback (lookbehind/backreferences): expand `$` // and friends against the fancy captures instead of the never-match - // placeholder stored in `regex_ptr`. + // placeholder stored as the standard program. if let Some(fre) = lookup_fancy_regex(re) { return replace_regex_str_fancy(str_data, &fre, (*re).global, repl_str); } diff --git a/crates/perry-runtime/src/regex/replace_expand_fancy.rs b/crates/perry-runtime/src/regex/replace_expand_fancy.rs index 22e78cee6a..9106ade261 100644 --- a/crates/perry-runtime/src/regex/replace_expand_fancy.rs +++ b/crates/perry-runtime/src/regex/replace_expand_fancy.rs @@ -197,7 +197,7 @@ pub extern "C" fn js_string_replace_regex( // Pattern the `regex` crate couldn't compile (lookbehind/backreferences) // → drive the replacement through fancy-regex. Otherwise the never-match - // placeholder in `regex_ptr` would leave the input unchanged. + // placeholder standard program would leave the input unchanged. if let Some(fre) = lookup_fancy_regex(re) { return replace_regex_str_fancy(str_data, &fre, (*re).global, repl_str); } @@ -369,7 +369,7 @@ pub extern "C" fn js_string_search_regex(s: *const StringHeader, re: *const RegE } // Fancy-regex fallback (lookbehind/backreferences): the never-match - // placeholder in `regex_ptr` would always report -1 otherwise. + // placeholder standard program would always report -1 otherwise. if let Some(fre) = lookup_fancy_regex(re) { return match fre.find(str_data) { Ok(Some(m)) => byte_index_to_utf16_index(str_data, m.start()) as i32, diff --git a/crates/perry-runtime/src/regex/site_cache.rs b/crates/perry-runtime/src/regex/site_cache.rs index bf0d3c1897..675896af48 100644 --- a/crates/perry-runtime/src/regex/site_cache.rs +++ b/crates/perry-runtime/src/regex/site_cache.rs @@ -8,31 +8,25 @@ //! `/…/g` on every call, once per text segment per layout pass, and //! `ansi-regex` builds the same `new RegExp(parts.join("|"), "g")` per call. //! Each construction used to copy the pattern three times (the -//! `VALIDATED_PATTERNS` probe key, `owned_pattern`, the `REGEX_SOURCE_TABLE` -//! entry) and SipHash all of it once; the first operation on each header then -//! did the same three more times — `build_and_install_programs` probes the -//! three `(String, String)`-keyed program caches — and, for the common -//! no-fallback pattern, `lookup_fancy_regex` / `lookup_repeat_matcher` -//! re-probed two of them on EVERY exec. On the claude-code keystroke profile -//! SipHash over pattern text was 31 % of the post-turn window (regex 38 % -//! inclusive), all of it under these five functions. +//! `VALIDATED_PATTERNS` probe key and `owned_pattern`) and SipHash all of it +//! once; the first operation on each header then did the same three more times. +//! On the claude-code keystroke profile SipHash over pattern text was 31 % of +//! the post-turn window (regex 38 % inclusive). //! //! # What //! -//! A direct-mapped, thread-local table keyed by a cheap CONTENT fingerprint -//! (length, first / middle / last 8 bytes, canonical flags) and verified by a -//! full byte compare — identity never depends on an address, so nothing is -//! rekeyed on a GC move and a dynamic `new RegExp(sameText)` hits too; a hit -//! costs one `memcmp` instead of a hash plus three copies. An entry owns the -//! pattern and canonical flags as `Arc` (shared into -//! `REGEX_SOURCE_TABLE`, so a header costs two refcount bumps instead of two -//! `String`s) and, once the first header built from it has been executed, the -//! compiled programs: a later construction installs those eagerly, so the -//! header is born built and never touches the `(pattern, flags)` caches. +//! A bounded, thread-local table keyed by a cheap CONTENT fingerprint (length, +//! first / middle / last 8 bytes, canonical flags) and verified by a full byte +//! compare. Identity never depends on an address, so nothing is rekeyed on a GC +//! move and a dynamic `new RegExp(sameText)` hits too; a hit costs one short +//! integer hash plus one `memcmp` instead of hashing all pattern bytes. //! -//! Validity is a pure function of `(pattern, flags)`, so a hit legitimately -//! skips validation: an entry is only ever written on the validated path, and -//! the programs it hands out were built for exactly this text. +//! An entry owns the pattern and canonical flags as `Arc` and, once the +//! first header built from it has executed, the compiled programs: a later +//! construction is born built. At capacity, entries referenced by a recorded +//! literal site are pinned and only dynamic or displaced-site entries are +//! evictable. Thus a live literal cannot rebuild, while programs for dead sites +//! can still leave the fixed-size table. //! //! Kill switch: `PERRY_REGEX_SITE_CACHE=0` (lookups miss, nothing is stored). @@ -41,6 +35,8 @@ use std::sync::Arc; use regex::Regex; +type ContentMap = crate::fast_hash::PtrHashMap>; + /// The compiled programs a header owns, in the form `lazy` installs them. pub(super) struct Programs { pub(super) std: Arc, @@ -48,12 +44,14 @@ pub(super) struct Programs { pub(super) repeat: Option>, } -impl Clone for Programs { - fn clone(&self) -> Self { - Self { - std: self.std.clone(), - fancy: self.fancy.clone(), - repeat: self.repeat.clone(), +impl Programs { + pub(super) fn matcher_kind(&self) -> super::MatcherKind { + if self.repeat.is_some() { + super::MatcherKind::Repeat + } else if self.fancy.is_some() { + super::MatcherKind::Fancy + } else { + super::MatcherKind::Standard } } } @@ -62,23 +60,22 @@ impl Clone for Programs { pub(super) struct Hit { pub(super) pattern: Arc, pub(super) flags: Arc, - pub(super) programs: Option, + pub(super) programs: Option>, } struct Entry { fp: u64, pattern: Arc, flags: Arc, - programs: Option, + programs: Option>, } -/// Direct-mapped slots (2-way: a fingerprint may live in `slot` or -/// `slot ^ 1`). Sized for a bundle's live literal working set; the -/// claude-code TUI cycles through a few dozen per render. -const SLOTS: usize = 1024; +/// Sized for the recorded literal-site table. The table never exceeds this +/// bound: if all entries are pinned, a dynamic miss remains uncached. +pub(super) const MAX_ENTRIES: usize = 1024; crate::perry_thread_local! { - static SITE_CACHE: RefCell>> = RefCell::new(Vec::new()); + static SITE_CACHE: RefCell = RefCell::new(crate::fast_hash::new_ptr_hash_map()); } fn enabled() -> bool { @@ -91,8 +88,8 @@ fn enabled() -> bool { } /// Cheap content fingerprint: length, three 8-byte windows of the pattern, -/// the (≤ 8 byte) canonical flags. Collisions are harmless — every hit is -/// verified by a full compare — they only cost the verify and a re-insert. +/// the (≤ 8 byte) canonical flags. Collisions are harmless because every hit +/// is verified by a full compare and colliding entries share one small bucket. fn fingerprint(pattern: &[u8], flags: &[u8]) -> u64 { #[inline] fn window(bytes: &[u8], at: usize) -> u64 { @@ -116,46 +113,105 @@ fn fingerprint(pattern: &[u8], flags: &[u8]) -> u64 { h } -#[inline] -fn slot_of(fp: u64) -> usize { - (fp as usize) & (SLOTS - 1) -} - fn entry_matches(entry: &Entry, fp: u64, pattern: &str, flags: &str) -> bool { entry.fp == fp && &*entry.flags == flags && &*entry.pattern == pattern } +fn entry_count(cache: &ContentMap) -> usize { + cache.values().map(Vec::len).sum() +} + +pub(super) fn census() -> crate::gc::census::SideTableRow { + let (entries, bytes, _) = census_parts(); + ("regex.content_cache", entries, bytes) +} + +pub(super) fn census_parts() -> (usize, usize, Vec) { + SITE_CACHE.with(|cache| { + let cache = cache.borrow(); + let entries = entry_count(&cache); + let inner = cache + .values() + .map(crate::gc::census::vec_bytes) + .sum::(); + let text = cache + .values() + .flatten() + .map(|entry| entry.pattern.len() + entry.flags.len()) + .sum::(); + let programs = cache + .values() + .flatten() + .filter_map(|entry| entry.programs.as_ref()) + .map(|programs| Arc::as_ptr(programs) as usize) + .collect(); + ( + entries, + crate::gc::census::map_bytes(&*cache) + inner + text, + programs, + ) + }) +} + +pub(super) fn census_program_ptrs() -> std::collections::HashSet { + census_parts().2.into_iter().collect() +} + +/// Remove one entry that has no recorded literal site. The scan happens only +/// on a distinct-content miss at capacity; literal-site hits never reach it. +fn evict_one_dynamic(cache: &mut ContentMap) -> bool { + let victim = cache.iter().find_map(|(&fp, bucket)| { + bucket + .iter() + .position(|entry| !super::site_key::references_content(&entry.pattern, &entry.flags)) + .map(|index| (fp, index)) + }); + let Some((fp, index)) = victim else { + return false; + }; + let bucket = cache.get_mut(&fp).expect("the selected bucket exists"); + bucket.swap_remove(index); + if bucket.is_empty() { + cache.remove(&fp); + } + true +} + +fn make_room(cache: &mut ContentMap) -> bool { + if entry_count(cache) < MAX_ENTRIES { + return true; + } + if evict_one_dynamic(cache) { + if crate::hot_diag::regex_on() { + crate::hot_diag::regex_counters(|d| d.cache_evictions += 1); + } + return true; + } + false +} + /// Find the verified entry for `(pattern, canonical flags)`. pub(super) fn lookup(pattern: &str, flags: &str) -> Option { if !enabled() { return None; } let fp = fingerprint(pattern.as_bytes(), flags.as_bytes()); - let slot = slot_of(fp); SITE_CACHE.with(|cache| { let cache = cache.borrow(); - if cache.is_empty() { - return None; - } - for s in [slot, slot ^ 1] { - if let Some(entry) = &cache[s] { - if entry_matches(entry, fp, pattern, flags) { - // The verify is a FULL byte compare, so its cost is - // linear in the pattern and this counter — not - // `pattern_bytes`, which counts every construction - // whether it probed or not — is the `memcmp` volume. - // Counted at the construction probe only; `insert` and - // `install_programs` verify too and are not counted here. - if crate::hot_diag::regex_on() { - let n = pattern.len() as u64; - crate::hot_diag::regex_counters(|d| d.new_site_verify_bytes += n); - } - return Some(Hit { - pattern: entry.pattern.clone(), - flags: entry.flags.clone(), - programs: entry.programs.clone(), + for entry in cache.get(&fp)? { + if entry_matches(entry, fp, pattern, flags) { + // Count only the construction probe's full byte compare, not + // the cold insert/install verification. + if crate::hot_diag::regex_on() { + crate::hot_diag::regex_counters(|d| { + d.new_site_verify_bytes += pattern.len() as u64 }); } + return Some(Hit { + pattern: entry.pattern.clone(), + flags: entry.flags.clone(), + programs: entry.programs.clone(), + }); } } None @@ -163,83 +219,68 @@ pub(super) fn lookup(pattern: &str, flags: &str) -> Option { } /// Record a validated `(pattern, canonical flags)`, returning the shared -/// owned copies a header should keep. An existing verified entry is reused -/// (its programs are kept); otherwise the fresh entry has none yet. +/// owned copies a header should keep. An existing verified entry is reused. pub(super) fn insert(pattern: &str, flags: &str) -> (Arc, Arc) { if !enabled() { return (Arc::from(pattern), Arc::from(flags)); } let fp = fingerprint(pattern.as_bytes(), flags.as_bytes()); - let slot = slot_of(fp); SITE_CACHE.with(|cache| { let mut cache = cache.borrow_mut(); - if cache.is_empty() { - cache.resize_with(SLOTS, || None); - } - for s in [slot, slot ^ 1] { - if let Some(entry) = &cache[s] { + if let Some(bucket) = cache.get(&fp) { + for entry in bucket { if entry_matches(entry, fp, pattern, flags) { return (entry.pattern.clone(), entry.flags.clone()); } } } - let victim = if cache[slot].is_none() { - slot - } else if cache[slot ^ 1].is_none() { - slot ^ 1 - } else { - slot ^ ((fp >> 11) as usize & 1) - }; let pattern: Arc = Arc::from(pattern); let flags: Arc = Arc::from(flags); - cache[victim] = Some(Entry { - fp, - pattern: pattern.clone(), - flags: flags.clone(), - programs: None, - }); + if make_room(&mut cache) { + cache.entry(fp).or_default().push(Entry { + fp, + pattern: pattern.clone(), + flags: flags.clone(), + programs: None, + }); + } (pattern, flags) }) } -/// Attach the programs the first execution built to the entry for -/// `(pattern, canonical flags)`, so every later construction of the same -/// text is born built. Inserts the entry if it was evicted meanwhile. -pub(super) fn install_programs(pattern: &str, flags: &str, programs: Programs) { +/// Attach the programs the first execution built to the content entry and +/// publish a weak view to every recorded literal site for this exact content. +pub(super) fn install_programs(pattern: &str, flags: &str, programs: Arc) { if !enabled() { return; } let fp = fingerprint(pattern.as_bytes(), flags.as_bytes()); - let slot = slot_of(fp); - SITE_CACHE.with(|cache| { + let content_owned = SITE_CACHE.with(|cache| { let mut cache = cache.borrow_mut(); - if cache.is_empty() { - cache.resize_with(SLOTS, || None); - } - for s in [slot, slot ^ 1] { - if let Some(entry) = &mut cache[s] { + if let Some(bucket) = cache.get_mut(&fp) { + for entry in bucket { if entry_matches(entry, fp, pattern, flags) { if entry.programs.is_none() { - entry.programs = Some(programs); + entry.programs = Some(programs.clone()); } - return; + return true; } } } - let victim = if cache[slot].is_none() { - slot - } else if cache[slot ^ 1].is_none() { - slot ^ 1 - } else { - slot ^ ((fp >> 11) as usize & 1) - }; - cache[victim] = Some(Entry { + if !make_room(&mut cache) { + return false; + } + cache.entry(fp).or_default().push(Entry { fp, pattern: Arc::from(pattern), flags: Arc::from(flags), - programs: Some(programs), + programs: Some(programs.clone()), }); + true }); + if content_owned { + super::site_key::install_programs_for_content(pattern, flags, &programs); + } } #[cfg(test)] @@ -250,19 +291,23 @@ pub(super) fn test_reset() { #[cfg(test)] pub(super) fn test_has_programs(pattern: &str, flags: &str) -> Option { let fp = fingerprint(pattern.as_bytes(), flags.as_bytes()); - let slot = slot_of(fp); SITE_CACHE.with(|cache| { let cache = cache.borrow(); - if cache.is_empty() { - return None; - } - for s in [slot, slot ^ 1] { - if let Some(entry) = &cache[s] { - if entry_matches(entry, fp, pattern, flags) { - return Some(entry.programs.is_some()); - } + for entry in cache.get(&fp)? { + if entry_matches(entry, fp, pattern, flags) { + return Some(entry.programs.is_some()); } } None }) } + +#[cfg(test)] +pub(super) fn test_len() -> usize { + SITE_CACHE.with(|cache| entry_count(&cache.borrow())) +} + +#[cfg(test)] +pub(super) fn test_try_evict_one_dynamic() -> bool { + SITE_CACHE.with(|cache| evict_one_dynamic(&mut cache.borrow_mut())) +} diff --git a/crates/perry-runtime/src/regex/site_key.rs b/crates/perry-runtime/src/regex/site_key.rs index aff7f4f028..70b313da63 100644 --- a/crates/perry-runtime/src/regex/site_key.rs +++ b/crates/perry-runtime/src/regex/site_key.rs @@ -50,8 +50,8 @@ use std::sync::{Arc, Weak}; use super::site_cache::Programs; -/// The site entry's view of a pattern's compiled programs: **weak**, so the -/// table can hand them out but can never be the reason they stay alive. +/// The site entry's view of a pattern's compiled programs: **weak**, because +/// the pinned content-cache entry owns the bundle. /// /// Measured cost of holding them strongly (cc, one 3300-char reply): settled /// footprint 478/474 MB → 500/527 MB and idle CPU 2.37 → 2.68 s. The site @@ -60,42 +60,23 @@ use super::site_cache::Programs; /// wants. The campaign's directive is both metrics together, and a CPU win /// bought with resident memory does not land. /// -/// Strong references remain where they belong: the `(pattern, flags)` program -/// caches, and every live header that installed them via `Arc::into_raw`. A -/// site entry whose programs have been dropped simply reports "not built -/// yet", and the next construction re-picks them up from the content cache — -/// the same path the site's very first construction takes. -struct WeakPrograms { - std: Weak<::regex::Regex>, - fancy: Option>, - repeat: Option>, -} +/// Strong references remain where they belong: the content cache and every +/// live header that installed them via `Arc::into_raw`. When this bounded site +/// entry is displaced, the content entry becomes eligible for eviction. +struct WeakPrograms(Weak); impl WeakPrograms { - fn downgrade(programs: &Programs) -> Self { - Self { - std: Arc::downgrade(&programs.std), - fancy: programs.fancy.as_ref().map(Arc::downgrade), - repeat: programs.repeat.as_ref().map(Arc::downgrade), - } + fn downgrade(programs: &Arc) -> Self { + Self(Arc::downgrade(programs)) } /// ALL-OR-NOTHING. A header must carry **every** program its pattern needs /// — that is #9801's coherence rule, and a partial upgrade is exactly the /// incoherent triple it fixed: a standard program installed beside a /// missing fancy fallback silently never-matches instead of falling back. - /// So a single dead reference makes the whole entry report unbuilt. - fn upgrade(&self) -> Option { - let std = self.std.upgrade()?; - let fancy = match &self.fancy { - None => None, - Some(weak) => Some(weak.upgrade()?), - }; - let repeat = match &self.repeat { - None => None, - Some(weak) => Some(weak.upgrade()?), - }; - Some(Programs { std, fancy, repeat }) + /// One weak pointer to the bundle makes partial upgrade unrepresentable. + fn upgrade(&self) -> Option> { + self.0.upgrade() } } @@ -137,7 +118,7 @@ pub(super) struct SiteHit { pub(super) flags: Arc, pub(super) flags_are_canonical: bool, pub(super) bits: FlagBits, - pub(super) programs: Option, + pub(super) programs: Option>, } /// Direct-mapped, 2-way (a key may live in `slot` or `slot ^ 1`). A bundle's @@ -166,6 +147,22 @@ fn slot_of(key: usize) -> usize { (key >> 3) & (SLOTS - 1) } +pub(super) fn census() -> crate::gc::census::SideTableRow { + let (entries, bytes) = census_parts(); + ("regex.literal_sites", entries, bytes) +} + +pub(super) fn census_parts() -> (usize, usize) { + SITE_KEY_TABLE.with(|table| { + let table = table.borrow(); + let entries = table.iter().filter(|entry| entry.is_some()).count(); + ( + entries, + table.capacity() * std::mem::size_of::>(), + ) + }) +} + /// The entry recorded for `key`, or `None`. pub(super) fn lookup(key: usize, raw_flags: &str) -> Option { if !enabled() || key == 0 { @@ -204,7 +201,7 @@ pub(super) fn record( flags: Arc, flags_are_canonical: bool, bits: FlagBits, - programs: Option, + programs: Option>, ) { if !enabled() || key == 0 { return; @@ -260,7 +257,7 @@ pub(super) fn record( /// Attach the programs the first execution built, so later constructions at /// this site are born built. A no-op when the site was evicted meanwhile. -pub(super) fn install_programs(key: usize, programs: Programs) { +pub(super) fn install_programs(key: usize, programs: Arc) { if !enabled() || key == 0 { return; } @@ -287,6 +284,31 @@ pub(super) fn install_programs(key: usize, programs: Programs) { }); } +/// Whether the bounded literal-site table still records this content. The +/// content cache consults this only on a collision miss, never on a site hit. +pub(super) fn references_content(pattern: &str, flags: &str) -> bool { + SITE_KEY_TABLE.with(|table| { + table + .borrow() + .iter() + .flatten() + .any(|entry| &*entry.pattern == pattern && &*entry.flags == flags) + }) +} + +/// Publish a freshly built bundle to every literal site for this content. The +/// content cache owns it; sites observe the complete bundle through one weak +/// reference, preserving the all-or-nothing matcher rule. +pub(super) fn install_programs_for_content(pattern: &str, flags: &str, programs: &Arc) { + SITE_KEY_TABLE.with(|table| { + for entry in table.borrow_mut().iter_mut().flatten() { + if &*entry.pattern == pattern && &*entry.flags == flags { + entry.programs = Some(WeakPrograms::downgrade(programs)); + } + } + }); +} + #[cfg(test)] pub(super) fn test_reset() { SITE_KEY_TABLE.with(|table| table.borrow_mut().clear()); @@ -323,42 +345,38 @@ mod tests { /// #9801 fixed an incoherent triple — a standard program memoized beside a /// missing fancy fallback — which does not error: it silently never /// matches. Holding the site entry's programs weakly reintroduces exactly - /// that shape unless a dead reference invalidates the WHOLE entry, because - /// the three `Arc`s have independent lifetimes and the fancy fallback is - /// the one a pattern the linear engine refused depends on. - /// - /// A sabotage that upgrades each field independently — the natural way to - /// write it — returns `Some(Programs { std, fancy: None, .. })` here and - /// fails on the second assertion. + /// that shape if it weakens each matcher separately. A single weak pointer + /// to the program bundle makes an incoherent partial upgrade impossible. #[test] - fn one_dead_reference_invalidates_the_whole_entry() { + fn the_program_set_weak_reference_expires_atomically() { let std_program = Arc::new(::regex::Regex::new("a(b)c").expect("linear program")); let fancy_program = Arc::new(::fancy_regex::Regex::new("a(?=b)c").expect("fancy program")); - let programs = Programs { + let programs = Arc::new(Programs { std: std_program.clone(), fancy: Some(fancy_program.clone()), repeat: None, - }; + }); let weak = WeakPrograms::downgrade(&programs); - drop(programs); let upgraded = weak .upgrade() - .expect("both strong references are still held here"); + .expect("the shared program set is still held here"); assert!( upgraded.fancy.is_some(), "the fancy fallback must survive the round trip while its Arc is alive" ); drop(upgraded); - // Only the FANCY program dies. The standard one is still strongly held. - drop(fancy_program); + // Individual matcher Arcs do not keep the SET alive. Once the bundle + // is gone the weak entry expires all three lanes together, which is + // the coherence property a triple of independent Weak pointers had + // to implement manually. + drop(programs); assert!( weak.upgrade().is_none(), - "one dead reference must invalidate the whole entry — handing back a header with a \ - standard program and no fancy fallback is #9801's incoherent triple, which never \ - matches instead of failing" + "the site entry must never upgrade only part of a program set" ); + drop(fancy_program); drop(std_program); assert!(weak.upgrade().is_none()); } @@ -371,11 +389,11 @@ mod tests { test_reset(); let key = 0x5171_E000_usize; let std_program = Arc::new(::regex::Regex::new("keepalive").expect("linear program")); - let programs = Programs { + let programs = Arc::new(Programs { std: std_program.clone(), fancy: None, repeat: None, - }; + }); record( key, Arc::from("g"), @@ -391,7 +409,7 @@ mod tests { unicode: false, has_indices: false, }, - Some(programs), + Some(programs.clone()), ); assert!( lookup(key, "g") @@ -401,6 +419,7 @@ mod tests { "precondition: the entry answers with its programs while they are alive" ); + drop(programs); drop(std_program); let hit = lookup(key, "g").expect("the entry itself survives"); assert!( diff --git a/crates/perry-runtime/src/regex/site_test.rs b/crates/perry-runtime/src/regex/site_test.rs new file mode 100644 index 0000000000..bbf18e1b35 --- /dev/null +++ b/crates/perry-runtime/src/regex/site_test.rs @@ -0,0 +1,922 @@ +//! Allocation-free headers for regex literals whose only observable use at a +//! source site is the receiver of one `.test` call. +//! +//! A cached header is a strong mutable GC root. It is never returned from a +//! transformed expression: direct literals feed it only to the paired test +//! dispatch, and a factory participates only while its structurally proven +//! body (`return /literal/`) is executing under a recorded caller site. The +//! caller still invokes the factory on every evaluation, so reassignment and +//! member lookup retain their ordinary effects. + +use std::cell::RefCell; +use std::collections::HashMap; + +use super::{is_valid_regex_ptr, js_regexp_new_impl, RegExpHeader}; + +struct Entry { + header: *mut RegExpHeader, + /// Zero for a direct literal; otherwise the compiler-emitted public + /// function entry for the exact-return factory body. + factory_identity: usize, + /// Set by the construction/factory entry after this evaluation's + /// canonicality check, consumed by the immediately following property Get. + approved: bool, +} + +#[derive(Clone, Copy)] +struct ActiveFactorySite { + site_key: usize, + /// Native entry resolved from the actual callee value before invocation. + /// A literal in a nested helper must not consume its caller's site. + expected_identity: usize, + handled_by_literal: bool, +} + +crate::perry_thread_local! { + static SITE_TEST_HEADERS: RefCell> = RefCell::new(HashMap::new()); + static ACTIVE_FACTORY_SITES: RefCell> = RefCell::new(Vec::new()); +} + +const BUILTIN_TEST_MARKER: u64 = crate::value::TAG_MARKER; + +#[inline] +fn note_no_alloc() { + #[cfg(test)] + TEST_NO_ALLOC.with(|counter| counter.set(counter.get() + 1)); + if crate::hot_diag::regex_on() { + crate::hot_diag::regex_counters(|d| d.site_test_no_alloc += 1); + } +} + +#[inline] +fn note_declined(reason: DeclineReason) { + #[cfg(test)] + { + TEST_DECLINED.with(|counter| counter.set(counter.get() + 1)); + let bucket = match reason { + DeclineReason::PatchedPrototype => &TEST_DECLINED_PATCHED, + DeclineReason::CalleeMismatch => &TEST_DECLINED_CALLEE, + DeclineReason::NonLiteral => &TEST_DECLINED_NON_LITERAL, + }; + bucket.with(|counter| counter.set(counter.get() + 1)); + } + if crate::hot_diag::regex_on() { + crate::hot_diag::regex_counters(|d| { + d.site_test_declined += 1; + match reason { + DeclineReason::PatchedPrototype => d.site_test_declined_patched_prototype += 1, + DeclineReason::CalleeMismatch => d.site_test_declined_callee_mismatch += 1, + DeclineReason::NonLiteral => d.site_test_declined_non_literal += 1, + } + }); + } +} + +#[cfg(test)] +thread_local! { + static TEST_NO_ALLOC: std::cell::Cell = const { std::cell::Cell::new(0) }; + static TEST_DECLINED: std::cell::Cell = const { std::cell::Cell::new(0) }; + static TEST_DECLINED_PATCHED: std::cell::Cell = const { std::cell::Cell::new(0) }; + static TEST_DECLINED_CALLEE: std::cell::Cell = const { std::cell::Cell::new(0) }; + static TEST_DECLINED_NON_LITERAL: std::cell::Cell = const { std::cell::Cell::new(0) }; + static TEST_ALLOCATIONS: std::cell::Cell = const { std::cell::Cell::new(0) }; +} + +#[derive(Clone, Copy)] +enum DeclineReason { + PatchedPrototype, + CalleeMismatch, + NonLiteral, +} + +fn lookup(site_key: usize) -> Option<(*mut RegExpHeader, usize)> { + SITE_TEST_HEADERS.with(|table| { + table + .borrow() + .get(&site_key) + .map(|entry| (entry.header, entry.factory_identity)) + }) +} + +fn install(site_key: usize, header: *mut RegExpHeader, factory_identity: usize) { + if site_key == 0 || header.is_null() { + return; + } + let mut entry = Entry { + header: std::ptr::null_mut(), + factory_identity, + approved: true, + }; + // GC_STORE_AUDIT(ROOT): `entry.header` becomes a mutable raw root when the + // entry is inserted into SITE_TEST_HEADERS; `scan_roots_mut` visits it. + // SAFETY: both pointers are either null or allocator-returned RegExp + // addresses; the destination is the root slot that will own `header`. + unsafe { + crate::gc::runtime_store_root_raw_mut_ptr_slot(&mut entry.header, header); + } + SITE_TEST_HEADERS.with(|table| { + table.borrow_mut().insert(site_key, entry); + }); +} + +#[inline] +fn allocate( + pattern: *const crate::StringHeader, + flags: *const crate::StringHeader, + site_key: usize, +) -> *mut RegExpHeader { + #[cfg(test)] + TEST_ALLOCATIONS.with(|counter| counter.set(counter.get() + 1)); + js_regexp_new_impl(pattern, flags, site_key) +} + +fn approve(site_key: usize, header: *mut RegExpHeader) { + SITE_TEST_HEADERS.with(|table| { + if let Some(entry) = table.borrow_mut().get_mut(&site_key) { + if entry.header == header { + entry.approved = true; + } + } + }); +} + +fn take_approval(site_key: usize, header: *mut RegExpHeader) -> bool { + SITE_TEST_HEADERS.with(|table| { + let mut table = table.borrow_mut(); + let Some(entry) = table.get_mut(&site_key) else { + return false; + }; + if entry.header != header || !entry.approved { + return false; + } + entry.approved = false; + true + }) +} + +fn canonical_rooted_header(header: *mut RegExpHeader) -> Option<*mut RegExpHeader> { + if !is_valid_regex_ptr(header) { + return None; + } + let scope = crate::gc::RuntimeHandleScope::new(); + let rooted = scope.root_raw_mut_ptr(header); + let value = f64::from_bits(crate::value::JSValue::pointer(header.cast::()).bits()); + let (canonical, header) = rooted.across_mut::(|| { + crate::object::regex_proto_thunks::regexp_prototype_test_is_canonical(value) + }); + if !canonical { + return None; + } + Some(header) +} + +/// Direct literal receiver for `/literal/flags.test(arg)`. +#[no_mangle] +pub extern "C" fn js_regexp_site_test_new( + pattern: *const crate::StringHeader, + flags: *const crate::StringHeader, + site_key: i64, +) -> *mut RegExpHeader { + let site_key = site_key as usize; + if let Some((header, factory_identity)) = lookup(site_key) { + if factory_identity == 0 { + if let Some(header) = canonical_rooted_header(header) { + approve(site_key, header); + // Price the construction this function avoided. The cold + // install is deliberately excluded. + note_no_alloc(); + return header; + } + note_declined(DeclineReason::PatchedPrototype); + return allocate(pattern, flags, site_key); + } + } + + let header = allocate(pattern, flags, site_key); + if let Some(header) = canonical_rooted_header(header) { + install(site_key, header, 0); + header + } else { + note_declined(DeclineReason::PatchedPrototype); + header + } +} + +fn mark_active_literal(factory_identity: usize) -> Option { + ACTIVE_FACTORY_SITES.with(|stack| { + let mut stack = stack.borrow_mut(); + let frame = stack.last_mut()?; + if frame.expected_identity != factory_identity { + return None; + } + frame.handled_by_literal = true; + Some(frame.site_key) + }) +} + +/// Literal construction inside a HIR-proven exact regex factory. Outside a +/// transformed caller it is exactly `js_regexp_new_site`; inside one it uses +/// the caller site's `(site, function-entry)` record. +#[no_mangle] +pub extern "C" fn js_regexp_new_factory_site( + pattern: *const crate::StringHeader, + flags: *const crate::StringHeader, + literal_site_key: i64, + factory_identity: i64, +) -> *mut RegExpHeader { + let factory_identity = factory_identity as usize; + let Some(call_site_key) = mark_active_literal(factory_identity) else { + return allocate(pattern, flags, literal_site_key as usize); + }; + if let Some((header, recorded_identity)) = lookup(call_site_key) { + if recorded_identity != factory_identity { + note_declined(DeclineReason::CalleeMismatch); + return allocate(pattern, flags, literal_site_key as usize); + } + if let Some(header) = canonical_rooted_header(header) { + approve(call_site_key, header); + // Price the construction this function avoided. The cold + // install is deliberately excluded. + note_no_alloc(); + return header; + } + note_declined(DeclineReason::PatchedPrototype); + return allocate(pattern, flags, literal_site_key as usize); + } + + let header = allocate(pattern, flags, literal_site_key as usize); + if let Some(header) = canonical_rooted_header(header) { + install(call_site_key, header, factory_identity); + header + } else { + note_declined(DeclineReason::PatchedPrototype); + header + } +} + +struct ActiveFactoryGuard { + depth_before: usize, +} + +impl ActiveFactoryGuard { + fn push(site_key: usize, expected_identity: usize) -> Self { + let depth_before = ACTIVE_FACTORY_SITES.with(|stack| { + let depth_before = stack.borrow().len(); + stack.borrow_mut().push(ActiveFactorySite { + site_key, + expected_identity, + handled_by_literal: false, + }); + depth_before + }); + Self { depth_before } + } + + fn handled(&self) -> bool { + ACTIVE_FACTORY_SITES.with(|stack| { + stack + .borrow() + .get(self.depth_before) + .is_some_and(|frame| frame.handled_by_literal) + }) + } +} + +impl Drop for ActiveFactoryGuard { + fn drop(&mut self) { + ACTIVE_FACTORY_SITES.with(|stack| { + let mut stack = stack.borrow_mut(); + // `js_throw` may already have restored the stack before a system + // unwind runs this Drop. Never pop a still-live outer frame. + if stack.len() > self.depth_before { + stack.truncate(self.depth_before); + } + }); + } +} + +pub(crate) fn active_factory_stack_savepoint() -> usize { + ACTIVE_FACTORY_SITES.with(|stack| stack.borrow().len()) +} + +pub(crate) fn active_factory_stack_restore(depth: usize) { + ACTIVE_FACTORY_SITES.with(|stack| stack.borrow_mut().truncate(depth)); +} + +struct ImplicitThisGuard<'scope> { + previous: crate::gc::RuntimeHandle<'scope>, +} + +impl<'scope> ImplicitThisGuard<'scope> { + fn bind(scope: &'scope crate::gc::RuntimeHandleScope, receiver: f64) -> Self { + Self { + previous: scope.root_nanbox_f64(crate::object::js_implicit_this_set(receiver)), + } + } +} + +impl Drop for ImplicitThisGuard<'_> { + fn drop(&mut self) { + crate::object::js_implicit_this_set(self.previous.get_nanbox_f64()); + } +} + +fn call_value_at_site(site_key: usize, callee: f64, this_value: Option) -> f64 { + let scope = crate::gc::RuntimeHandleScope::new(); + let callee = scope.root_nanbox_f64(callee); + let callee_value = crate::value::JSValue::from_bits(callee.get_nanbox_f64().to_bits()); + let expected_identity = if callee_value.is_pointer() { + let closure = callee_value.as_pointer::(); + crate::closure::get_valid_func_ptr(closure) as usize + } else { + 0 + }; + let this_value = this_value.map(|value| scope.root_nanbox_f64(value)); + let this_guard = this_value + .as_ref() + .map(|value| ImplicitThisGuard::bind(&scope, value.get_nanbox_f64())); + let active = ActiveFactoryGuard::push(site_key, expected_identity); + let result = unsafe { + crate::closure::js_native_call_value(callee.get_nanbox_f64(), std::ptr::null(), 0) + }; + if !active.handled() { + let reason = if lookup(site_key).is_some() { + DeclineReason::CalleeMismatch + } else { + DeclineReason::NonLiteral + }; + note_declined(reason); + } + drop(active); + drop(this_guard); + result +} + +#[cfg(panic = "abort")] +#[no_mangle] +pub extern "C" fn js_regexp_site_factory_call_value(site_key: i64, callee: f64) -> f64 { + call_value_at_site(site_key as usize, callee, None) +} + +#[cfg(not(panic = "abort"))] +#[no_mangle] +pub extern "C-unwind" fn js_regexp_site_factory_call_value(site_key: i64, callee: f64) -> f64 { + call_value_at_site(site_key as usize, callee, None) +} + +#[cfg(panic = "abort")] +#[no_mangle] +pub unsafe extern "C" fn js_regexp_site_factory_call_method( + site_key: i64, + receiver: f64, + method_key: f64, +) -> f64 { + unsafe { site_factory_call_method_impl(site_key, receiver, method_key) } +} + +#[cfg(not(panic = "abort"))] +#[no_mangle] +pub unsafe extern "C-unwind" fn js_regexp_site_factory_call_method( + site_key: i64, + receiver: f64, + method_key: f64, +) -> f64 { + unsafe { site_factory_call_method_impl(site_key, receiver, method_key) } +} + +#[inline(always)] +unsafe fn site_factory_call_method_impl(site_key: i64, receiver: f64, method_key: f64) -> f64 { + // Resolve the member before activating the factory site. An accessor is + // arbitrary user code and must not let an incidental regex construction + // masquerade as the function the call actually selected. + let scope = crate::gc::RuntimeHandleScope::new(); + let receiver = scope.root_nanbox_f64(receiver); + let method_key = scope.root_nanbox_f64(method_key); + let method = + crate::value::js_dyn_index_get(receiver.get_nanbox_f64(), method_key.get_nanbox_f64()); + call_value_at_site(site_key as usize, method, Some(receiver.get_nanbox_f64())) +} + +/// Resolve `.test` at the spec-mandated point (before argument evaluation). +/// The internal marker means the builtin was validated for the cached header; +/// every generic decline returns the actual property value instead. +#[cfg(panic = "abort")] +#[no_mangle] +pub extern "C" fn js_regexp_site_test_get_method(site_key: i64, receiver: f64) -> f64 { + site_test_get_method_impl(site_key, receiver) +} + +#[cfg(not(panic = "abort"))] +#[no_mangle] +pub extern "C-unwind" fn js_regexp_site_test_get_method(site_key: i64, receiver: f64) -> f64 { + site_test_get_method_impl(site_key, receiver) +} + +#[inline(always)] +fn site_test_get_method_impl(site_key: i64, receiver: f64) -> f64 { + let site_key = site_key as usize; + let receiver_value = crate::value::JSValue::from_bits(receiver.to_bits()); + let receiver_ptr = receiver_value + .is_pointer() + .then(|| receiver_value.as_pointer::() as *mut RegExpHeader); + if let (Some(receiver_ptr), Some((header, _))) = (receiver_ptr, lookup(site_key)) { + if receiver_ptr == header { + if take_approval(site_key, header) { + return f64::from_bits(BUILTIN_TEST_MARKER); + } + note_declined(DeclineReason::PatchedPrototype); + } + } + + let scope = crate::gc::RuntimeHandleScope::new(); + let receiver = scope.root_nanbox_f64(receiver); + let key = crate::string::intern_ascii_literal(b"test"); + crate::value::js_dyn_index_get( + receiver.get_nanbox_f64(), + crate::js_nanbox_string(key as i64), + ) +} + +/// Consume the method captured above and the now-evaluated argument. +#[cfg(panic = "abort")] +#[no_mangle] +pub extern "C" fn js_regexp_site_test_dispatch( + _site_key: i64, + receiver: f64, + method: f64, + argument: f64, +) -> f64 { + site_test_dispatch_impl(receiver, method, argument) +} + +#[cfg(not(panic = "abort"))] +#[no_mangle] +pub extern "C-unwind" fn js_regexp_site_test_dispatch( + _site_key: i64, + receiver: f64, + method: f64, + argument: f64, +) -> f64 { + site_test_dispatch_impl(receiver, method, argument) +} + +#[inline(always)] +fn site_test_dispatch_impl(receiver: f64, method: f64, argument: f64) -> f64 { + let scope = crate::gc::RuntimeHandleScope::new(); + let receiver = scope.root_nanbox_f64(receiver); + let argument = scope.root_nanbox_f64(argument); + if method.to_bits() == BUILTIN_TEST_MARKER { + let string = crate::value::js_jsvalue_to_string_coerce(argument.get_nanbox_f64()); + let receiver = receiver.get_nanbox_f64(); + let value = crate::value::JSValue::from_bits(receiver.to_bits()); + if !value.is_pointer() { + return f64::from_bits(crate::value::TAG_FALSE); + } + let header = value.as_pointer::() as *mut RegExpHeader; + if !is_valid_regex_ptr(header) { + return f64::from_bits(crate::value::TAG_FALSE); + } + // A fresh literal begins every evaluation at zero. `test` may write + // the cached header's lastIndex, but no reference to this header leaves + // the transformed expression and the next call resets it again. + unsafe { + (*header).last_index = crate::value::JSValue::number(0.0).bits(); + } + return f64::from_bits( + crate::value::JSValue::bool(super::js_regexp_test(header, string) != 0).bits(), + ); + } + + let method = scope.root_nanbox_f64(method); + let _this_guard = ImplicitThisGuard::bind(&scope, receiver.get_nanbox_f64()); + let args = [argument.get_nanbox_f64()]; + unsafe { crate::closure::js_native_call_value(method.get_nanbox_f64(), args.as_ptr(), 1) } +} + +/// Strong root for every cached header. The visitor rewrites entries in +/// place during evacuation, so later probes never retain a from-space address. +pub(crate) fn scan_roots_mut(visitor: &mut crate::gc::RuntimeRootVisitor<'_>) { + SITE_TEST_HEADERS.with(|table| { + for entry in table.borrow_mut().values_mut() { + visitor.visit_raw_mut_ptr_slot(&mut entry.header); + } + }); +} + +pub(super) fn census() -> crate::gc::census::SideTableRow { + SITE_TEST_HEADERS.with(|table| { + let table = table.borrow(); + ( + "regex.site_test_headers", + table.len(), + crate::gc::census::map_bytes(&*table), + ) + }) +} + +pub(super) fn census_parts() -> (usize, usize, Vec, Vec) { + SITE_TEST_HEADERS.with(|table| { + let table = table.borrow(); + let headers = table + .values() + .map(|entry| entry.header as usize) + .collect::>(); + let programs = table + .values() + .filter_map(|entry| { + let header = entry.header; + if header.is_null() || !is_valid_regex_ptr(header) { + return None; + } + let programs = unsafe { (*header).programs_ptr }; + (!programs.is_null()).then_some(programs as usize) + }) + .collect::>() + .into_iter() + .collect(); + ( + table.len(), + crate::gc::census::map_bytes(&*table), + headers, + programs, + ) + }) +} + +pub(super) fn active_factory_census_parts() -> (usize, usize) { + ACTIVE_FACTORY_SITES.with(|stack| { + let stack = stack.borrow(); + ( + stack.len(), + stack.capacity() * std::mem::size_of::(), + ) + }) +} + +pub(crate) fn side_table_census() -> Vec { + vec![ + super::site_cache::census(), + super::site_key::census(), + census(), + ] +} + +#[cfg(test)] +pub(super) fn test_reset() { + SITE_TEST_HEADERS.with(|table| table.borrow_mut().clear()); + ACTIVE_FACTORY_SITES.with(|stack| stack.borrow_mut().clear()); + TEST_NO_ALLOC.with(|counter| counter.set(0)); + TEST_DECLINED.with(|counter| counter.set(0)); + TEST_DECLINED_PATCHED.with(|counter| counter.set(0)); + TEST_DECLINED_CALLEE.with(|counter| counter.set(0)); + TEST_DECLINED_NON_LITERAL.with(|counter| counter.set(0)); + TEST_ALLOCATIONS.with(|counter| counter.set(0)); +} + +#[cfg(test)] +pub(super) fn test_header(site_key: usize) -> Option { + lookup(site_key).map(|(header, _)| header as usize) +} + +#[cfg(test)] +mod tests { + use super::*; + + static DIRECT_G: u64 = 1; + static DIRECT_Y: u64 = 2; + static FACTORY_CALL: u64 = 3; + static MEMBER_CALL: u64 = 4; + static NESTED_FACTORY_CALL: u64 = 5; + static NESTED_WRAPPER_CALLS: std::sync::atomic::AtomicUsize = + std::sync::atomic::AtomicUsize::new(0); + + fn string(text: &str) -> *mut crate::StringHeader { + crate::string::js_string_from_bytes(text.as_ptr(), text.len() as u32) + } + + fn key(slot: &'static u64) -> i64 { + slot as *const u64 as i64 + } + + fn run_direct(site: i64, flags: &str, input: &str) -> (usize, bool) { + let receiver = js_regexp_site_test_new(string("x"), string(flags), site); + let receiver_value = crate::value::js_nanbox_pointer(receiver as i64); + let method = js_regexp_site_test_get_method(site, receiver_value); + let result = js_regexp_site_test_dispatch( + site, + receiver_value, + method, + crate::js_nanbox_string(string(input) as i64), + ); + (receiver as usize, crate::value::js_is_truthy(result) != 0) + } + + fn run_fresh_generic(flags: &str, input: &str) -> bool { + let receiver = super::super::js_regexp_new(string("x"), string(flags)); + super::super::js_regexp_test(receiver, string(input)) != 0 + } + + fn ensure_regexp_builtins() { + let value = crate::object::builtin_prototype_value("RegExp"); + assert!( + crate::value::JSValue::from_bits(value.to_bits()).is_pointer(), + "the RegExp intrinsic must be installed before testing its recorded test site" + ); + } + + #[test] + fn direct_global_site_allocates_one_header_and_resets_last_index() { + let _lock = crate::gc::global_side_table_test_lock(); + test_reset(); + ensure_regexp_builtins(); + let site = key(&DIRECT_G); + let inputs = ["x", "x", "a", "xx"]; + let mut header = None; + for input in inputs { + let want = run_fresh_generic("g", input); + let (current, got) = run_direct(site, "g", input); + assert_eq!(got, want, "site result must equal a fresh /x/g.test(input)"); + assert_eq!(*header.get_or_insert(current), current); + } + assert_eq!(test_header(site as usize), header); + assert_eq!( + TEST_NO_ALLOC.with(std::cell::Cell::get), + inputs.len() as u64 - 1 + ); + assert_eq!( + TEST_ALLOCATIONS.with(std::cell::Cell::get), + 1, + "the site must allocate exactly its one rooted header" + ); + } + + #[test] + fn direct_sticky_site_starts_each_evaluation_at_zero() { + let _lock = crate::gc::global_side_table_test_lock(); + test_reset(); + ensure_regexp_builtins(); + let site = key(&DIRECT_Y); + let first = run_direct(site, "y", "x"); + let second = run_direct(site, "y", "x"); + let offset_only = run_direct(site, "y", "ax"); + assert_eq!(first.0, second.0, "one rooted header serves the site"); + assert!( + first.1 && second.1, + "lastIndex must reset before the second test" + ); + assert!(!offset_only.1, "fresh /x/y is anchored at index zero"); + assert_eq!(offset_only.1, run_fresh_generic("y", "ax")); + assert_eq!(TEST_ALLOCATIONS.with(std::cell::Cell::get), 1); + } + + #[test] + fn escaping_generic_global_header_carries_last_index_between_tests() { + let _lock = crate::gc::global_side_table_test_lock(); + let re = super::super::js_regexp_new(string("x"), string("g")); + let subject = string("xx"); + assert_ne!(super::super::js_regexp_test(re, subject), 0); + assert_eq!( + unsafe { (*re).last_index }, + crate::value::JSValue::number(1.0).bits() + ); + assert_ne!( + super::super::js_regexp_test(re, subject), + 0, + "the second test must start at the first test's lastIndex, not at zero" + ); + assert_eq!( + unsafe { (*re).last_index }, + crate::value::JSValue::number(2.0).bits() + ); + } + + extern "C" fn exact_factory(_closure: *const crate::closure::ClosureHeader) -> f64 { + let re = js_regexp_new_factory_site( + string("x"), + string("g"), + key(&DIRECT_G), + exact_factory as *const u8 as i64, + ); + crate::value::js_nanbox_pointer(re as i64) + } + + extern "C" fn replacement_factory(_closure: *const crate::closure::ClosureHeader) -> f64 { + let object = crate::object::js_object_alloc(0, 0); + crate::object::js_object_set_field_by_name( + object, + string("test"), + closure_with_arity(patched_test as *const u8, 1), + ); + crate::value::js_nanbox_pointer(object as i64) + } + + #[inline(never)] + extern "C" fn nested_factory_wrapper(_closure: *const crate::closure::ClosureHeader) -> f64 { + // Keep this observably distinct from `exact_factory` under release + // function merging while modeling a non-literal wrapper with effects. + NESTED_WRAPPER_CALLS.fetch_add(1, std::sync::atomic::Ordering::Relaxed); + exact_factory(std::ptr::null()) + } + + fn closure(function: *const u8) -> f64 { + closure_with_arity(function, 0) + } + + fn closure_with_arity(function: *const u8, arity: usize) -> f64 { + crate::closure::js_register_closure_arity(function, arity as u32); + crate::value::js_nanbox_pointer(crate::closure::js_closure_alloc(function, 0) as i64) + } + + fn dispatch_test(site: i64, receiver: f64, input: &str) -> f64 { + let method = js_regexp_site_test_get_method(site, receiver); + js_regexp_site_test_dispatch( + site, + receiver, + method, + crate::js_nanbox_string(string(input) as i64), + ) + } + + #[test] + fn direct_factory_site_reuses_only_the_recorded_callee() { + let _lock = crate::gc::global_side_table_test_lock(); + test_reset(); + ensure_regexp_builtins(); + let site = key(&FACTORY_CALL); + let callee = closure(exact_factory as *const u8); + let first = js_regexp_site_factory_call_value(site, callee); + let second = js_regexp_site_factory_call_value(site, callee); + assert_eq!( + first.to_bits(), + second.to_bits(), + "the exact factory reuses its header" + ); + assert_eq!(TEST_NO_ALLOC.with(std::cell::Cell::get), 1); + + let replacement = closure(replacement_factory as *const u8); + let receiver = js_regexp_site_factory_call_value(site, replacement); + let result = dispatch_test(site, receiver, "does not contain the pattern"); + assert_eq!(result.to_bits(), crate::value::TAG_TRUE); + assert_eq!( + TEST_DECLINED.with(std::cell::Cell::get), + 1, + "the very next call must record the callee mismatch" + ); + assert_eq!(TEST_DECLINED_CALLEE.with(std::cell::Cell::get), 1); + } + + #[test] + fn nested_exact_factory_cannot_claim_a_different_callees_site() { + let _lock = crate::gc::global_side_table_test_lock(); + test_reset(); + ensure_regexp_builtins(); + NESTED_WRAPPER_CALLS.store(0, std::sync::atomic::Ordering::Relaxed); + let site = key(&NESTED_FACTORY_CALL); + let wrapper = closure(nested_factory_wrapper as *const u8); + let scope = crate::gc::RuntimeHandleScope::new(); + let first = scope.root_nanbox_f64(js_regexp_site_factory_call_value(site, wrapper)); + let second = js_regexp_site_factory_call_value(site, wrapper); + assert_ne!( + first.get_nanbox_f64().to_bits(), + second.to_bits(), + "a nested literal must keep fresh-object semantics" + ); + assert_eq!(test_header(site as usize), None); + assert_eq!(TEST_ALLOCATIONS.with(std::cell::Cell::get), 2); + assert_eq!(TEST_DECLINED.with(std::cell::Cell::get), 2); + assert_eq!(TEST_DECLINED_NON_LITERAL.with(std::cell::Cell::get), 2); + assert_eq!( + NESTED_WRAPPER_CALLS.load(std::sync::atomic::Ordering::Relaxed), + 2 + ); + } + + #[test] + fn caught_throw_restores_an_orphaned_factory_site_frame() { + let _lock = crate::gc::global_side_table_test_lock(); + test_reset(); + let base = active_factory_stack_savepoint(); + let _jump_buffer = crate::exception::js_try_push(); + let active = ActiveFactoryGuard::push(key(&FACTORY_CALL) as usize, usize::MAX); + assert_eq!(active_factory_stack_savepoint(), base + 1); + + // A raw throw skips this Drop. Model that transport, then replay the + // exception path's recorded savepoint restoration. + std::mem::forget(active); + crate::exception::test_unwind_innermost_shadow_restore(); + crate::exception::js_try_end(); + assert_eq!(active_factory_stack_savepoint(), base); + } + + #[test] + fn namespace_member_factory_site_is_covered() { + let _lock = crate::gc::global_side_table_test_lock(); + test_reset(); + ensure_regexp_builtins(); + let site = key(&MEMBER_CALL); + let namespace = crate::object::js_object_alloc(0, 0); + let name = string("default"); + crate::object::js_object_set_field_by_name( + namespace, + name, + closure(exact_factory as *const u8), + ); + let namespace_object = namespace; + let namespace = crate::value::js_nanbox_pointer(namespace_object as i64); + let member = crate::js_nanbox_string(name as i64); + let first = unsafe { js_regexp_site_factory_call_method(site, namespace, member) }; + let second = unsafe { js_regexp_site_factory_call_method(site, namespace, member) }; + assert_eq!(first.to_bits(), second.to_bits()); + assert_eq!(TEST_NO_ALLOC.with(std::cell::Cell::get), 1); + assert_eq!( + test_header(site as usize), + Some(crate::value::js_nanbox_get_pointer(first) as usize) + ); + + crate::object::js_object_set_field_by_name( + namespace_object, + name, + closure(replacement_factory as *const u8), + ); + let replacement = unsafe { js_regexp_site_factory_call_method(site, namespace, member) }; + assert_eq!( + dispatch_test(site, replacement, "no match").to_bits(), + crate::value::TAG_TRUE + ); + assert_eq!( + TEST_DECLINED.with(std::cell::Cell::get), + 1, + "rebinding the namespace member must decline on the next call" + ); + assert_eq!(TEST_DECLINED_CALLEE.with(std::cell::Cell::get), 1); + } + + extern "C" fn patched_test(_closure: *const crate::closure::ClosureHeader, _arg: f64) -> f64 { + f64::from_bits(crate::value::TAG_TRUE) + } + + #[test] + fn patched_regexp_prototype_test_declines_on_the_next_call() { + let _lock = crate::gc::global_side_table_test_lock(); + test_reset(); + ensure_regexp_builtins(); + let site = key(&DIRECT_G); + let (warm_header, _) = run_direct(site, "g", "x"); + + let proto_value = crate::object::builtin_prototype_value("RegExp"); + let proto = crate::value::JSValue::from_bits(proto_value.to_bits()) + .as_pointer::() + as *mut crate::object::ObjectHeader; + let test_key = string("test"); + let original = crate::object::js_object_get_field_by_name(proto, test_key); + crate::object::js_object_set_field_by_name( + proto, + test_key, + closure_with_arity(patched_test as *const u8, 1), + ); + + let receiver = js_regexp_site_test_new(string("x"), string("g"), site); + assert_ne!( + receiver as usize, warm_header, + "patched prototype forces a fresh header" + ); + let receiver_value = crate::value::js_nanbox_pointer(receiver as i64); + let method = js_regexp_site_test_get_method(site, receiver_value); + let result = js_regexp_site_test_dispatch( + site, + receiver_value, + method, + crate::js_nanbox_string(string("no match") as i64), + ); + assert_eq!(result.to_bits(), crate::value::TAG_TRUE); + assert_eq!(TEST_DECLINED.with(std::cell::Cell::get), 1); + assert_eq!(TEST_DECLINED_PATCHED.with(std::cell::Cell::get), 1); + + crate::object::js_object_set_field_by_name( + proto, + test_key, + f64::from_bits(original.bits()), + ); + } + + #[test] + fn site_header_root_is_rewritten_by_a_copying_minor() { + let _guard = crate::gc::CopyingNurseryTestGuard::new(0); + test_reset(); + crate::gc::gc_register_mutable_root_scanner(scan_roots_mut); + + let site = key(&DIRECT_G) as usize; + let header = super::super::test_alloc_nursery_regexp_for_move("site-root", "g"); + let old = header as usize; + assert!(crate::arena::pointer_in_nursery(old)); + install(site, header, 0); + + let _ = crate::gc::gc_collect_minor(); + let moved = test_header(site).expect("the site header remains rooted"); + assert_ne!(moved, old, "the scanner must rewrite the cached address"); + assert!(super::super::regex_header_has_magic( + moved as *const RegExpHeader + )); + test_reset(); + } +} diff --git a/crates/perry-runtime/src/regex/tests.rs b/crates/perry-runtime/src/regex/tests.rs index 11a6347e77..af67a400ea 100644 --- a/crates/perry-runtime/src/regex/tests.rs +++ b/crates/perry-runtime/src/regex/tests.rs @@ -15,6 +15,18 @@ pub(super) fn string_payload(s: *const StringHeader) -> Vec { } } +pub(super) fn regex_is_built(re: *const RegExpHeader) -> bool { + !unsafe { (*re).programs_ptr.is_null() } +} + +pub(super) fn regex_has_fancy_program(re: *const RegExpHeader) -> bool { + regex_is_built(re) && unsafe { (*(*re).programs_ptr).fancy.is_some() } +} + +pub(super) fn regex_has_repeat_program(re: *const RegExpHeader) -> bool { + regex_is_built(re) && unsafe { (*(*re).programs_ptr).repeat.is_some() } +} + #[test] fn regexp_has_dedicated_gc_kind_and_is_not_a_shaped_object() { let _lock = crate::gc::global_side_table_test_lock(); @@ -32,7 +44,17 @@ fn regexp_has_dedicated_gc_kind_and_is_not_a_shaped_object() { } #[test] -fn malloc_finalize_clears_regexp_address_owned_tables() { +#[cfg(target_pointer_width = "64")] +fn regexp_header_is_one_56_byte_per_object_record() { + assert_eq!( + std::mem::size_of::(), + 56, + "the three per-program matcher pointers must stay collapsed into one handle" + ); +} + +#[test] +fn malloc_finalize_clears_regexp_address_owned_state() { let _lock = crate::gc::global_side_table_test_lock(); let scope = crate::gc::RuntimeHandleScope::new(); let pattern = scope.root_string_ptr(make_string("finalize")); @@ -42,7 +64,6 @@ fn malloc_finalize_clears_regexp_address_owned_tables() { }); let addr = re as usize; assert!(test_regex_pointer_entry_exists(addr)); - assert!(test_regex_source_entry_exists(addr)); crate::object::exotic_expando::test_seed_exotic_expando_entry( addr, "owned", @@ -55,7 +76,6 @@ fn malloc_finalize_clears_regexp_address_owned_tables() { } assert!(!test_regex_pointer_entry_exists(addr)); - assert!(!test_regex_source_entry_exists(addr)); assert!(!crate::object::exotic_expando::test_exotic_expando_entry_exists(addr)); } @@ -78,10 +98,10 @@ fn regexp_finalize_releases_all_header_owned_programs() { re } - // Every compiled header owns the standard-engine program, including the - // never-match placeholder used by fancy-regex patterns. + // Every compiled header owns one shared program bundle, including the + // never-match placeholder and any fallback matcher. let standard = compile(r"needle\d+", "needle42"); - let standard_raw = unsafe { (*standard).regex_ptr as *const Regex }; + let standard_raw = unsafe { (*standard).programs_ptr }; assert!(!standard_raw.is_null()); let standard_observer = clone_raw_arc(standard_raw); let standard_before = std::sync::Arc::strong_count(&standard_observer); @@ -95,11 +115,11 @@ fn regexp_finalize_releases_all_header_owned_programs() { std::sync::Arc::strong_count(&standard_observer) + 1, standard_before ); - assert!(unsafe { (*standard).regex_ptr.is_null() }); + assert!(!regex_is_built(standard)); let fancy = compile(r"(?<=pre)\d+", "pre77"); - let fancy_raw = unsafe { (*fancy).fancy_ptr as *const fancy_regex::Regex }; - assert!(!fancy_raw.is_null()); + let fancy_raw = unsafe { (*fancy).programs_ptr }; + assert!(regex_has_fancy_program(fancy)); let fancy_observer = clone_raw_arc(fancy_raw); let fancy_before = std::sync::Arc::strong_count(&fancy_observer); unsafe { @@ -109,12 +129,11 @@ fn regexp_finalize_releases_all_header_owned_programs() { std::sync::Arc::strong_count(&fancy_observer) + 1, fancy_before ); - assert!(unsafe { (*fancy).fancy_ptr.is_null() }); + assert!(!regex_is_built(fancy)); let repeat = compile(r"(a?b??)*", "ab"); - let repeat_raw = - unsafe { (*repeat).repeat_matcher_ptr as *const repeat_matcher::RepeatMatcherRegex }; - assert!(!repeat_raw.is_null()); + let repeat_raw = unsafe { (*repeat).programs_ptr }; + assert!(regex_has_repeat_program(repeat)); let repeat_observer = clone_raw_arc(repeat_raw); let repeat_before = std::sync::Arc::strong_count(&repeat_observer); unsafe { @@ -125,7 +144,7 @@ fn regexp_finalize_releases_all_header_owned_programs() { } let repeat_after = std::sync::Arc::strong_count(&repeat_observer); assert_eq!(repeat_after + 1, repeat_before); - assert!(unsafe { (*repeat).repeat_matcher_ptr.is_null() }); + assert!(!regex_is_built(repeat)); // Arena overflow cleanup and finalization can overlap. A second finalizer // must observe null pointers rather than release an owned reference twice. @@ -885,7 +904,7 @@ fn unicode17_scripts_expand_to_codepoint_ranges() { /// 2026-07-09 GC audit (wave 2 batch A): the compiled-regex caches were /// unbounded — one entry per distinct `(pattern, flags)` ever compiled, up to /// 64 MiB each — so `new RegExp(userInput)` was an attacker-driven OOM. The -/// caches are now capped (clear-on-overflow) and every `RegExpHeader` OWNS a +/// caches are now capped (one-entry eviction) and every `RegExpHeader` OWNS a /// leaked Arc reference to its compiled program(s), so a header created /// before an eviction keeps matching afterwards. #[test] @@ -953,7 +972,7 @@ fn regex_cache_capped_and_prior_headers_survive_eviction() { assert!( js_regexp_test(fancy, make_string("pre77")) != 0, "fancy-fallback header must keep matching after cache eviction \ - (header-resident fancy_ptr, not the cleared FANCY_CACHE)" + (header-resident program set, not the cleared FANCY_CACHE)" ); assert!( js_regexp_test(fancy, make_string("nope77")) == 0, diff --git a/crates/perry-runtime/src/regex/tests_cache.rs b/crates/perry-runtime/src/regex/tests_cache.rs new file mode 100644 index 0000000000..297198c31d --- /dev/null +++ b/crates/perry-runtime/src/regex/tests_cache.rs @@ -0,0 +1,139 @@ +use super::*; +use std::collections::HashSet; + +pub(super) fn note_cache_eviction() { + crate::hot_diag::test_note_regex_cache_eviction(); +} + +fn make_string(text: &str) -> *mut StringHeader { + js_string_from_str(text) +} + +fn cache_test_key(slot: usize) -> i64 { + (0x6000_0000usize + (slot << 3)) as i64 +} + +/// Sabotage: remove the literal pin in `site_cache::evict_one_dynamic`, or put +/// back the whole-map clear in `evict_regex_cache_if_full`. +/// +/// The first sabotage lets a content-capacity eviction discard the target's +/// sole `Arc` owner. The second removes 512 answers instead of one. +/// After an explicit young collection finalizes both target headers, either +/// regression makes the next evaluation run the lazy builder again. +#[test] +fn literal_site_program_is_not_rebuilt_after_cache_overflow_and_young_collection() { + let _lock = crate::gc::global_side_table_test_lock(); + site_key::test_reset(); + site_cache::test_reset(); + REGEX_CACHE.with(|cache| cache.borrow_mut().clear()); + FANCY_CACHE.with(|cache| cache.borrow_mut().clear()); + REPEAT_MATCHER_CACHE.with(|cache| cache.borrow_mut().clear()); + VALIDATED_PATTERNS.with(|cache| cache.borrow_mut().clear()); + lazy::test_reset_program_builds(); + + let target = "literal-site-overflow-target-[0-9]+"; + let target_key = cache_test_key(0); + let first = js_regexp_new_site(make_string(target), make_string(""), target_key); + assert_eq!( + js_regexp_test(first, make_string("literal-site-overflow-target-42")), + 1 + ); + assert_eq!( + lazy::test_program_builds(), + 1, + "the target builds exactly once" + ); + + // A second construction installs the content cache's program bundle into + // the site's weak lane. Both headers must then die so that only the cache + // policy, not a surviving receiver, can keep that weak reference live. + let second = js_regexp_new_site(make_string(target), make_string(""), target_key); + assert!(!unsafe { (*second).programs_ptr.is_null() }); + let first_addr = first as usize; + let second_addr = second as usize; + assert!( + !site_cache::test_try_evict_one_dynamic(), + "the only content entry is site-referenced, so no dynamic victim exists" + ); + + // Overflow the bounded content table with dynamic patterns. The target is + // the only recorded literal among them, so every capacity eviction must + // choose one of the dynamic entries and leave its program bundle intact. + for i in 0..site_cache::MAX_ENTRIES { + let _ = js_regexp_new( + make_string(&format!("content-overflow-{i}[a-z]")), + make_string(""), + ); + } + assert_eq!(site_cache::test_len(), site_cache::MAX_ENTRIES); + assert_eq!( + site_cache::test_has_programs(target, ""), + Some(true), + "a recorded literal's content entry must survive capacity eviction" + ); + + // 513 compiled literals (the target plus this 512-pattern flood) cross the + // former 512-entry wholesale-clear boundary. Snapshot at capacity so the + // assertion proves the overflow path preserved 511 old answers. + for i in 0..(REGEX_CACHE_MAX_ENTRIES - 1) { + let pattern = format!("overflow-literal-{i}$"); + let re = js_regexp_new_site( + make_string(&pattern), + make_string(""), + cache_test_key(100 + i), + ); + assert_eq!( + js_regexp_test(re, make_string(&format!("overflow-literal-{i}"))), + 1 + ); + } + let before: HashSet<_> = REGEX_CACHE.with(|cache| cache.borrow().keys().cloned().collect()); + assert_eq!(before.len(), REGEX_CACHE_MAX_ENTRIES); + + let last_i = REGEX_CACHE_MAX_ENTRIES - 1; + let last_pattern = format!("overflow-literal-{last_i}$"); + let last = js_regexp_new_site( + make_string(&last_pattern), + make_string(""), + cache_test_key(100 + last_i), + ); + assert_eq!( + js_regexp_test(last, make_string(&format!("overflow-literal-{last_i}"))), + 1 + ); + let survivors = REGEX_CACHE.with(|cache| { + cache + .borrow() + .keys() + .filter(|key| before.contains(*key)) + .count() + }); + assert_eq!(survivors, REGEX_CACHE_MAX_ENTRIES - 1); + assert!( + crate::hot_diag::test_regex_builds_and_evictions().1 > 0, + "the capacity eviction path must have executed" + ); + + let builds_before_gc = lazy::test_program_builds(); + let _ = crate::gc::gc_collect_minor(); + assert!( + !test_regex_pointer_entry_exists(first_addr) + && !test_regex_pointer_entry_exists(second_addr), + "the explicit young collection must finalize both unrooted target headers" + ); + + let rebuilt = js_regexp_new_site(make_string(target), make_string(""), target_key); + assert!( + !unsafe { (*rebuilt).programs_ptr.is_null() }, + "the still-recorded site must be born built after cache overflow and young GC" + ); + assert_eq!( + js_regexp_test(rebuilt, make_string("literal-site-overflow-target-7")), + 1 + ); + assert_eq!( + lazy::test_program_builds(), + builds_before_gc, + "the target site's compiled program must not be rebuilt" + ); +} diff --git a/crates/perry-runtime/src/regex/tests_header.rs b/crates/perry-runtime/src/regex/tests_header.rs new file mode 100644 index 0000000000..dcf8eacc17 --- /dev/null +++ b/crates/perry-runtime/src/regex/tests_header.rs @@ -0,0 +1,281 @@ +use super::*; + +fn make_string(s: &str) -> *mut StringHeader { + crate::string::js_string_from_bytes(s.as_ptr(), s.len() as u32) +} + +fn make_wtf8(bytes: &[u8]) -> *mut StringHeader { + crate::string::js_string_from_wtf8_bytes(bytes.as_ptr(), bytes.len() as u32) +} + +fn string_payload(s: *const StringHeader) -> Vec { + unsafe { + std::slice::from_raw_parts(crate::string::string_data(s), (*s).byte_len as usize).to_vec() + } +} + +fn regex_is_built(re: *const RegExpHeader) -> bool { + !unsafe { (*re).programs_ptr.is_null() } +} + +fn regex_has_fancy_program(re: *const RegExpHeader) -> bool { + regex_is_built(re) && unsafe { (*(*re).programs_ptr).fancy.is_some() } +} + +fn regex_has_repeat_program(re: *const RegExpHeader) -> bool { + regex_is_built(re) && unsafe { (*(*re).programs_ptr).repeat.is_some() } +} + +/// Construction must NOT build the automaton; the first operation that needs a +/// matcher must. +/// +/// This is the structural half of the perf fix — the wall-clock half is a +/// fixture whose 200 literals cost 73 ms to construct before and ~0 after. A +/// regression here (something re-introducing an eager build) would not fail any +/// behavioural test, only make every program slower, so assert the state +/// directly: `programs_ptr` is the built/not-built flag. +#[test] +fn construction_defers_the_program_build_until_first_use() { + let re = js_regexp_new( + make_string("[A-Za-z]+(?:foo|bar)[0-9]{1,4}"), + make_string("i"), + ); + assert!( + !regex_is_built(re), + "constructing a RegExp must not build its program" + ); + // Everything observable without matching stays available. + assert_eq!( + string_payload(js_regexp_get_source(re)), + b"[A-Za-z]+(?:foo|bar)[0-9]{1,4}".to_vec() + ); + assert_eq!(string_payload(js_regexp_get_flags(re)), b"i".to_vec()); + assert!(unsafe { (*re).case_insensitive }); + assert!( + !regex_is_built(re), + "reading .source/.flags must not build the program either" + ); + + assert!(js_regexp_test(re, make_string("XFOO12")) != 0); + assert!( + regex_is_built(re), + "the first match must build and install the program" + ); +} + +/// Removing the address-keyed source table must not turn the RegExp-pattern +/// constructor arm into an empty-pattern fallback. This calls the exported +/// constructor entry point, so deleting its direct header read fails both the +/// source and inherited-flags assertions. +#[test] +fn regexp_construct_reads_source_and_flags_from_the_pattern_header() { + let original = js_regexp_new(make_string("left/right"), make_string("ig")); + let pattern = crate::value::js_nanbox_pointer(original as i64); + let undefined = f64::from_bits(crate::value::TAG_UNDEFINED); + + let copy = js_regexp_construct(pattern, undefined); + assert_ne!( + copy, original, + "construction must still allocate a fresh object" + ); + assert_eq!(string_payload(js_regexp_get_source(copy)), b"left\\/right"); + assert_eq!(string_payload(js_regexp_get_flags(copy)), b"gi"); + + let override_flags = crate::value::js_nanbox_string(make_string("m") as i64); + let overridden = js_regexp_construct(pattern, override_flags); + assert_eq!( + string_payload(js_regexp_get_source(overridden)), + b"left\\/right" + ); + assert_eq!(string_payload(js_regexp_get_flags(overridden)), b"m"); +} + +/// `RegExp.prototype.compile` rewrites the header in place. The source table +/// used to mask a stale header slot here; after its removal both observable +/// strings must come from the newly stored, traced edges. +#[test] +fn regexp_compile_replaces_the_header_source_and_flags() { + let receiver = js_regexp_new(make_string("old"), make_string("m")); + js_regexp_set_last_index(receiver, 9.0); + let pattern = crate::value::js_nanbox_string(make_string("new/source") as i64); + let flags = crate::value::js_nanbox_string(make_string("ig") as i64); + let result = js_regexp_compile_value(receiver, pattern, flags); + let receiver = crate::value::JSValue::from_bits(result.to_bits()).as_pointer::(); + + assert_eq!( + string_payload(js_regexp_get_source(receiver)), + b"new\\/source" + ); + assert_eq!(string_payload(js_regexp_get_flags(receiver)), b"gi"); + assert_eq!(js_regexp_get_last_index(receiver), 0.0); + assert_eq!(js_regexp_test(receiver, make_string("NEW/source")), 1); +} + +/// Perry stores lone JavaScript surrogates as WTF-8. `.source` must copy those +/// exact bytes from the traced pattern slot; routing them through Rust's UTF-8 +/// scalar iterator either replaces the surrogate or invokes undefined +/// behaviour. +#[test] +fn regexp_source_round_trips_wtf8_lone_surrogates_from_the_header() { + let lone_high = [b'a', 0xED, 0xA0, 0x80, b'/', b'b']; + let re = js_regexp_new(make_string("placeholder"), make_string("")); + let pattern = make_wtf8(&lone_high); + unsafe { + (*re).pattern_ptr = pattern; + } + let expected = [b'a', 0xED, 0xA0, 0x80, b'\\', b'/', b'b']; + assert_eq!(string_payload(js_regexp_get_source(re)), expected); +} + +/// The deferred build installs the fancy-regex and RepeatMatcher programs too, +/// not just the linear one — they live on the same publish point, so a header +/// whose pattern needs one must still get it on first use. +#[test] +fn deferred_build_installs_the_fancy_and_repeat_matcher_fallbacks() { + let fancy = js_regexp_new(make_string(r"(?<=pre)\d+"), make_string("")); + assert!(!regex_is_built(fancy)); + assert!(js_regexp_test(fancy, make_string("pre77")) != 0); + assert!( + regex_has_fancy_program(fancy), + "first use must install the fancy-regex fallback" + ); + assert!(js_regexp_test(fancy, make_string("nope77")) == 0); + + let repeat = js_regexp_new(make_string(r"(a?b??)*"), make_string("")); + assert!(!regex_is_built(repeat)); + assert!(js_regexp_test(repeat, make_string("ab")) != 0); + assert!( + regex_has_repeat_program(repeat), + "first use must install the ECMAScript RepeatMatcher" + ); +} + +/// Sabotage for the bounded Segmenter lane: this pattern's standard program +/// is the never-match placeholder, so a wrong `Standard` tag returns false. +/// Exercise all three installation routes that must publish the tag beside the +/// program handle: lazy build, born-built cache hit, and `compile`. +#[test] +fn bounded_test_matcher_tag_routes_fancy_patterns_to_fancy_regex() { + let _lock = crate::gc::global_side_table_test_lock(); + site_cache::test_reset(); + let pattern = r"(?<=left)right"; + + let cold = js_regexp_new(make_string(pattern), make_string("")); + assert_eq!(unsafe { (*cold).matcher_kind }, MatcherKind::Unbuilt); + assert_eq!(regexp_test_str_bounded(cold, "leftright"), Some(true)); + assert_eq!(unsafe { (*cold).matcher_kind }, MatcherKind::Fancy); + + let born_built = js_regexp_new(make_string(pattern), make_string("")); + assert!(regex_is_built(born_built)); + assert_eq!(unsafe { (*born_built).matcher_kind }, MatcherKind::Fancy); + assert_eq!( + regexp_test_str_bounded(born_built, "leftwrong"), + Some(false) + ); + + let compiled = js_regexp_new(make_string("plain"), make_string("")); + let pattern_value = crate::value::js_nanbox_string(make_string(pattern) as i64); + let flags_value = crate::value::js_nanbox_string(make_string("") as i64); + let result = js_regexp_compile_value(compiled, pattern_value, flags_value); + let compiled = crate::value::JSValue::from_bits(result.to_bits()).as_pointer::(); + assert_eq!(unsafe { (*compiled).matcher_kind }, MatcherKind::Fancy); + assert_eq!(regexp_test_str_bounded(compiled, "leftright"), Some(true)); +} + +/// Two evaluations of the same pattern are still distinct objects with +/// independent `lastIndex`, and deferring the build does not let them share a +/// header (ECMA-262 requires a fresh object per evaluation — the same +/// invariant the closure-literal singleton fix restored for functions). +#[test] +fn deferred_build_keeps_per_object_identity_and_last_index() { + let a = js_regexp_new(make_string("x"), make_string("g")); + let b = js_regexp_new(make_string("x"), make_string("g")); + assert_ne!( + a as usize, b as usize, + "each evaluation is a distinct object" + ); + assert!(!js_regexp_exec(a, make_string("xx")).is_null()); + assert_eq!(regex_last_index_offset(a), 1); + assert_eq!( + regex_last_index_offset(b), + 0, + "a sibling regex must not inherit lastIndex through the shared program" + ); +} + +/// The validated-pattern set is capped like the program caches: it holds owned +/// pattern text (`emoji-regex` is ~12,807 chars) and is fed by `new +/// RegExp(userInput)`, so an uncapped one would be the same attacker-driven +/// growth the compiled-program caches were capped for. +#[test] +fn validated_pattern_set_is_capped() { + for i in 0..(REGEX_CACHE_MAX_ENTRIES * 2 + 10) { + lazy::mark_pattern_validated(&format!("validfill{i}[a-z]+"), ""); + } + let len = VALIDATED_PATTERNS.with(|c| c.borrow().len()); + assert!( + len <= REGEX_CACHE_MAX_ENTRIES, + "VALIDATED_PATTERNS must stay capped at {REGEX_CACHE_MAX_ENTRIES} entries, got {len}" + ); +} + +/// The `[\s\S]` → `(?s:.)` rewrite must not move a single match result. +/// +/// The rewrite exists purely to dodge a 1.1-million-iteration case fold in +/// `regex_syntax` (see `grammar::push_any_char`), so the only thing that may +/// change is how long construction takes. Everything a program can observe — +/// what matches, what a capture group holds, which group number it is, and +/// that the NEGATED forms still match nothing — is pinned here, because a +/// silently widened character class produces no error anywhere: only a wrong +/// answer, on inputs a syntax test never looks at. +#[test] +fn any_char_rewrite_preserves_match_behaviour() { + // Matches every code point, newlines included, with and without `i`. + for pattern in ["[\\s\\S]", "[^]", "[\\d\\D]", "[\\w\\W]", "[\\S\\s]"] { + for flags in ["", "i", "u", "iu", "m"] { + let re = js_regexp_new(make_string(pattern), make_string(flags)); + for subject in ["a", "\n", " ", "\u{1F600}", "Ω", "\r"] { + assert!( + js_regexp_test(re, make_string(subject)) != 0, + "/{pattern}/{flags} must match {subject:?}" + ); + } + } + } + + // The negated forms are the exact opposite and must still match NOTHING. + for pattern in ["[^\\s\\S]", "[^\\w\\W]", "[]"] { + let re = js_regexp_new(make_string(pattern), make_string("i")); + for subject in ["a", "\n", "Ω"] { + assert!( + js_regexp_test(re, make_string(subject)) == 0, + "/{pattern}/i must not match {subject:?}" + ); + } + } + + // A class that is NOT a complementary pair keeps its narrow meaning. + let narrow = js_regexp_new(make_string("[\\d\\s]"), make_string("i")); + assert!(js_regexp_test(narrow, make_string("7")) != 0); + assert!(js_regexp_test(narrow, make_string("a")) == 0); + + // The rewrite emits a NON-capturing group, so group numbering is + // unchanged: `$1` is still `b`, not the any-char. + let re = js_regexp_new(make_string("a[\\s\\S](b)"), make_string("")); + let m = js_regexp_exec(re, make_string("a\nb")); + assert!(!m.is_null(), "a[\\s\\S](b) must match \"a\\nb\""); + + // Quantifiers still bind to the any-char, lazily and greedily. + let lazy = js_regexp_new(make_string("([\\s\\S]*?)"), make_string("i")); + assert!(js_regexp_test(lazy, make_string("one\ntwo")) != 0); + let greedy = js_regexp_new(make_string("^[\\s\\S]{3}$"), make_string("")); + assert!(js_regexp_test(greedy, make_string("a\nb")) != 0); + assert!(js_regexp_test(greedy, make_string("a\nbc")) == 0); + + // `.source` still reports what the author wrote, not the translation. + let re = js_regexp_new(make_string("[\\s\\S]+"), make_string("gi")); + assert_eq!( + string_payload(js_regexp_get_source(re)), + b"[\\s\\S]+".to_vec() + ); +} diff --git a/crates/perry-runtime/src/regex/tests_part2.rs b/crates/perry-runtime/src/regex/tests_part2.rs index 3ea07b9671..5896ff2c94 100644 --- a/crates/perry-runtime/src/regex/tests_part2.rs +++ b/crates/perry-runtime/src/regex/tests_part2.rs @@ -2,7 +2,9 @@ //! A sibling child of `regex`, so `use super::*` resolves exactly as it does //! in `tests.rs`; the shared fixtures come from there. -use super::tests::{make_string, match_capture_text, string_payload}; +use super::tests::{ + make_string, match_capture_text, regex_has_fancy_program, regex_is_built, string_payload, +}; use super::*; #[test] @@ -114,164 +116,6 @@ fn syntax_check_agrees_with_full_build() { } } -/// Construction must NOT build the automaton; the first operation that needs a -/// matcher must. -/// -/// This is the structural half of the perf fix — the wall-clock half is a -/// fixture whose 200 literals cost 73 ms to construct before and ~0 after. A -/// regression here (something re-introducing an eager build) would not fail any -/// behavioural test, only make every program slower, so assert the state -/// directly: `regex_ptr` is the built/not-built flag. -#[test] -fn construction_defers_the_program_build_until_first_use() { - let re = js_regexp_new( - make_string("[A-Za-z]+(?:foo|bar)[0-9]{1,4}"), - make_string("i"), - ); - assert!( - unsafe { (*re).regex_ptr.is_null() }, - "constructing a RegExp must not build its program" - ); - // Everything observable without matching stays available. - assert_eq!( - string_payload(js_regexp_get_source(re)), - b"[A-Za-z]+(?:foo|bar)[0-9]{1,4}".to_vec() - ); - assert_eq!(string_payload(js_regexp_get_flags(re)), b"i".to_vec()); - assert!(unsafe { (*re).case_insensitive }); - assert!( - unsafe { (*re).regex_ptr.is_null() }, - "reading .source/.flags must not build the program either" - ); - - assert!(js_regexp_test(re, make_string("XFOO12")) != 0); - assert!( - !unsafe { (*re).regex_ptr.is_null() }, - "the first match must build and install the program" - ); -} - -/// The deferred build installs the fancy-regex and RepeatMatcher programs too, -/// not just the linear one — they live on the same publish point, so a header -/// whose pattern needs one must still get it on first use. -#[test] -fn deferred_build_installs_the_fancy_and_repeat_matcher_fallbacks() { - let fancy = js_regexp_new(make_string(r"(?<=pre)\d+"), make_string("")); - assert!(unsafe { (*fancy).fancy_ptr.is_null() }); - assert!(js_regexp_test(fancy, make_string("pre77")) != 0); - assert!( - !unsafe { (*fancy).fancy_ptr.is_null() }, - "first use must install the fancy-regex fallback" - ); - assert!(js_regexp_test(fancy, make_string("nope77")) == 0); - - let repeat = js_regexp_new(make_string(r"(a?b??)*"), make_string("")); - assert!(unsafe { (*repeat).repeat_matcher_ptr.is_null() }); - assert!(js_regexp_test(repeat, make_string("ab")) != 0); - assert!( - !unsafe { (*repeat).repeat_matcher_ptr.is_null() }, - "first use must install the ECMAScript RepeatMatcher" - ); -} - -/// Two evaluations of the same pattern are still distinct objects with -/// independent `lastIndex`, and deferring the build does not let them share a -/// header (ECMA-262 requires a fresh object per evaluation — the same -/// invariant the closure-literal singleton fix restored for functions). -#[test] -fn deferred_build_keeps_per_object_identity_and_last_index() { - let a = js_regexp_new(make_string("x"), make_string("g")); - let b = js_regexp_new(make_string("x"), make_string("g")); - assert_ne!( - a as usize, b as usize, - "each evaluation is a distinct object" - ); - assert!(!js_regexp_exec(a, make_string("xx")).is_null()); - assert_eq!(regex_last_index_offset(a), 1); - assert_eq!( - regex_last_index_offset(b), - 0, - "a sibling regex must not inherit lastIndex through the shared program" - ); -} - -/// The validated-pattern set is capped like the program caches: it holds owned -/// pattern text (`emoji-regex` is ~12,807 chars) and is fed by `new -/// RegExp(userInput)`, so an uncapped one would be the same attacker-driven -/// growth the compiled-program caches were capped for. -#[test] -fn validated_pattern_set_is_capped() { - for i in 0..(REGEX_CACHE_MAX_ENTRIES * 2 + 10) { - lazy::mark_pattern_validated(&format!("validfill{i}[a-z]+"), ""); - } - let len = VALIDATED_PATTERNS.with(|c| c.borrow().len()); - assert!( - len <= REGEX_CACHE_MAX_ENTRIES, - "VALIDATED_PATTERNS must stay capped at {REGEX_CACHE_MAX_ENTRIES} entries, got {len}" - ); -} - -/// The `[\s\S]` → `(?s:.)` rewrite must not move a single match result. -/// -/// The rewrite exists purely to dodge a 1.1-million-iteration case fold in -/// `regex_syntax` (see `grammar::push_any_char`), so the only thing that may -/// change is how long construction takes. Everything a program can observe — -/// what matches, what a capture group holds, which group number it is, and -/// that the NEGATED forms still match nothing — is pinned here, because a -/// silently widened character class produces no error anywhere: only a wrong -/// answer, on inputs a syntax test never looks at. -#[test] -fn any_char_rewrite_preserves_match_behaviour() { - // Matches every code point, newlines included, with and without `i`. - for pattern in ["[\\s\\S]", "[^]", "[\\d\\D]", "[\\w\\W]", "[\\S\\s]"] { - for flags in ["", "i", "u", "iu", "m"] { - let re = js_regexp_new(make_string(pattern), make_string(flags)); - for subject in ["a", "\n", " ", "\u{1F600}", "Ω", "\r"] { - assert!( - js_regexp_test(re, make_string(subject)) != 0, - "/{pattern}/{flags} must match {subject:?}" - ); - } - } - } - - // The negated forms are the exact opposite and must still match NOTHING. - for pattern in ["[^\\s\\S]", "[^\\w\\W]", "[]"] { - let re = js_regexp_new(make_string(pattern), make_string("i")); - for subject in ["a", "\n", "Ω"] { - assert!( - js_regexp_test(re, make_string(subject)) == 0, - "/{pattern}/i must not match {subject:?}" - ); - } - } - - // A class that is NOT a complementary pair keeps its narrow meaning. - let narrow = js_regexp_new(make_string("[\\d\\s]"), make_string("i")); - assert!(js_regexp_test(narrow, make_string("7")) != 0); - assert!(js_regexp_test(narrow, make_string("a")) == 0); - - // The rewrite emits a NON-capturing group, so group numbering is - // unchanged: `$1` is still `b`, not the any-char. - let re = js_regexp_new(make_string("a[\\s\\S](b)"), make_string("")); - let m = js_regexp_exec(re, make_string("a\nb")); - assert!(!m.is_null(), "a[\\s\\S](b) must match \"a\\nb\""); - - // Quantifiers still bind to the any-char, lazily and greedily. - let lazy = js_regexp_new(make_string("([\\s\\S]*?)"), make_string("i")); - assert!(js_regexp_test(lazy, make_string("one\ntwo")) != 0); - let greedy = js_regexp_new(make_string("^[\\s\\S]{3}$"), make_string("")); - assert!(js_regexp_test(greedy, make_string("a\nb")) != 0); - assert!(js_regexp_test(greedy, make_string("a\nbc")) == 0); - - // `.source` still reports what the author wrote, not the translation. - let re = js_regexp_new(make_string("[\\s\\S]+"), make_string("gi")); - assert_eq!( - string_payload(js_regexp_get_source(re)), - b"[\\s\\S]+".to_vec() - ); -} - /// #9305 fallout: the translator spells ECMAScript's ASCII `\b`/`\B` as /// `(?-iu:\b)`, which fancy-regex's parser rejects (`NonUnicodeUnsupported`). /// Any lookaround/backreference pattern containing a word boundary therefore @@ -609,10 +453,7 @@ fn site_cache_reconstruction_is_born_built() { let _lock = crate::gc::global_side_table_test_lock(); site_cache::test_reset(); let re1 = js_regexp_new(make_string("born[0-9]+built"), make_string("g")); - assert!( - unsafe { (*re1).regex_ptr.is_null() }, - "construction stays lazy" - ); + assert!(!regex_is_built(re1), "construction stays lazy"); assert_eq!( site_cache::test_has_programs("born[0-9]+built", "g"), Some(false), @@ -626,28 +467,20 @@ fn site_cache_reconstruction_is_born_built() { ); let re2 = js_regexp_new(make_string("born[0-9]+built"), make_string("g")); assert!( - !unsafe { (*re2).regex_ptr.is_null() }, + regex_is_built(re2), "the second construction installs the programs eagerly" ); assert!( - std::ptr::eq(unsafe { (*re1).regex_ptr }, unsafe { (*re2).regex_ptr }), + std::ptr::eq(unsafe { (*re1).programs_ptr }, unsafe { + (*re2).programs_ptr + }), "both headers share one compiled program" ); - // The owned source copies are shared too (two refcount bumps per header, - // not two `String`s). - let (p1, p2) = REGEX_SOURCE_TABLE.with(|t| { - let t = t.borrow(); - ( - t.get(&(re1 as usize)).map(|(p, _)| p.clone()).unwrap(), - t.get(&(re2 as usize)).map(|(p, _)| p.clone()).unwrap(), - ) - }); - assert!(Arc::ptr_eq(&p1, &p2), "source text is shared, not copied"); assert_eq!(js_regexp_test(re2, make_string("born7built")), 1); assert_eq!(js_regexp_test(re2, make_string("nothing")), 0); // Different flags are a different entry. let re3 = js_regexp_new(make_string("born[0-9]+built"), make_string("i")); - assert!(unsafe { (*re3).regex_ptr.is_null() }); + assert!(!regex_is_built(re3)); } /// `test` on a global/sticky receiver advances `lastIndex` exactly like @@ -765,15 +598,13 @@ fn a_single_program_cache_clear_cannot_disarm_a_lookbehind_literal() { site_cache::test_reset(); let cold = build(); - unsafe { - lazy::ensure_regex_compiled(cold); - assert!( - !(*cold).fancy_ptr.is_null(), - "a built header must carry every program its pattern needs — a null \ - fancy_ptr here is memoized by site_cache::install_programs and makes \ - the breakage permanent for this literal" - ); - } + lazy::ensure_regex_compiled(cold); + assert!( + regex_has_fancy_program(cold), + "a built header must carry every program its pattern needs — a null \ + the fancy program here is memoized by site_cache::install_programs and makes \ + the breakage permanent for this literal" + ); assert_eq!( subject.with_const_ptr::(|s| js_regexp_test(cold, s)), 1, @@ -1014,7 +845,7 @@ fn a_dynamic_construction_records_nothing_in_the_site_table() { /// A site hit must be born built: the second construction at a site whose /// first header has already executed installs the compiled programs eagerly, -/// so `regex_ptr` is non-null before any match runs. +/// so `programs_ptr` is non-null before any match runs. /// /// This is what makes the fast path complete — a hit that skipped the content /// cache but arrived unbuilt would push the pattern's hash back onto the first @@ -1027,19 +858,19 @@ fn a_site_hit_after_the_first_execution_is_born_built() { let first = js_regexp_new_site(make_string("bo+rn"), make_string(""), key); assert!( - unsafe { (*first).regex_ptr }.is_null(), + !regex_is_built(first), "construction must not build the program (that is #5777's deferred build)" ); assert!(js_regexp_test(first, make_string("boorn")) != 0); assert!( - !unsafe { (*first).regex_ptr }.is_null(), + regex_is_built(first), "the first execution installs the programs" ); // Second construction at the SAME site. let second = js_regexp_new_site(make_string("bo+rn"), make_string(""), key); assert!( - !unsafe { (*second).regex_ptr }.is_null(), + regex_is_built(second), "a site hit must install the programs the site already compiled, so the header is born \ built and the first match pays no lookup" ); diff --git a/crates/perry-runtime/src/string/split.rs b/crates/perry-runtime/src/string/split.rs index addeb94e55..286fccf888 100644 --- a/crates/perry-runtime/src/string/split.rs +++ b/crates/perry-runtime/src/string/split.rs @@ -460,7 +460,7 @@ pub extern "C" fn js_string_split_n( // both. Detect regex delimiters by checking whether the pointer was // recorded by `js_regexp_new` and delegate to `js_string_split_regex` // on a match. Otherwise the regex header would be read as a - // StringHeader and segfault on the first byte of its `regex_ptr`. + // StringHeader and segfault on the first byte of its program-set pointer. #[cfg(feature = "regex-engine")] if crate::regex::is_regex_pointer(delimiter as *const u8) { return crate::regex::js_string_split_regex_n( diff --git a/scripts/gc_runtime_root_holders.json b/scripts/gc_runtime_root_holders.json index f1582b60c2..bf515b6a8c 100644 --- a/scripts/gc_runtime_root_holders.json +++ b/scripts/gc_runtime_root_holders.json @@ -276,7 +276,7 @@ "file": "crates/perry-runtime/src/gc/census.rs", "name": "PASS1_MARKED", "verdict": "non_moving_snapshot", - "why": "Real GC header addresses, deliberately untraced so the diagnostic does not keep its observed objects alive. Populated only at the end of mark propagation of a synchronous full cycle; consumed at sweep entry in the same run_to_completion invocation. The intervening full-cycle phases do not relocate or run JS callbacks. The Vec is used for membership comparisons and dropped with the census before sweep. Budgeted and minor cycles skip both boundaries. Pin re-audited 2026-09-05 after #9760 touched `gc/mod.rs`: that change is `mod heap_stats;` plus a `pub(crate) use` re-export and alters no mark/sweep control flow. `heap_stats()` is reached only from `js_bun_jsc_heap_stats` (the JS-facing `bun:jsc.heapStats()`), i.e. from mutator code, never inside a cycle, and its own module contract forbids allocation or collection during its walk. The mark-complete \u2192 sweep-entry window is unchanged. Re-audited 2026-09-05 (train125) after #9769 and #9771 touched pinned files. #9769 adds one `reg_scanner!` registration to `gc/mod.rs`; #9771 adds a feature-gated `alloc_census_init()` there and a feature-gated Rust-heap dump inside `take_census`. `alloc-census` is not in the default feature set, and decisively: `census_take_if_armed_at_full_sweep_start` does `PASS1_MARKED.with(|p| p.borrow_mut().take())` BEFORE calling `take_census`, so the snapshot has already left the thread-local by the time #9771's code runs \u2014 it cannot affect the window. Neither change alters mark/sweep control flow. Re-audited 2026-09-06 after #9831 touched `gc/policy.rs`. Its hunks are (a) the tiny-parse pressure guard's pricing (`tiny_parse_pressure_headroom_bytes`, `tiny_parse_pressure_due*`, a `Cell` byte-count base) consulted from JSON.parse's mutator-side boundaries (`gc_bump_malloc_trigger`, `gc_collect_pending_suppressed_parse`, `gc_schedule_parse_boundary_collection_if_pressure`), none of which is reachable from inside a cycle, and (b) one extra `Cell` store in `note_collection_finished_arena_occupancy`, which runs from `publish_reclaim_outcome` in the Publish subphase \u2014 after `step_sweep` has already consumed the snapshot. Mark/sweep control flow between `census_pass1_if_armed` and `census_take_if_armed_at_full_sweep_start` is untouched. Re-audited 2026-09-05 (train126) after #9755 restructured `gc/cycle.rs`. Its hunks are all root-scan machinery (`RootScanSubphase`, `RootScanCycleState`, the mutable-scanner iteration state), which runs BEFORE mark propagation completes; `gc/mod.rs` gains only a `mod young_log;` declaration. The bracketing is unchanged \u2014 `census_pass1_if_armed` is still inside `step_mark_propagation` and `census_take_if_armed_at_full_sweep_start` inside `step_sweep` \u2014 and a synchronous full mark-sweep still moves nothing between them. Re-pinned 2026-09-05 for the #9740 hot-TLS conversion of this file: the sole change is `thread_local!` \u2192 `crate::perry_thread_local!`, a macro-name swap with identical declaration syntax and `.with()` call sites. No control flow, no phase boundary, and no storage semantics change. Re-audited 2026-09-06 (train128) after #9794's GC diagnostics touched `gc/mod.rs` and `gc/policy.rs`: both gain diagnostic module declarations and counters only \u2014 no mark/sweep control flow, and the census bracketing in `step_mark_propagation` / `step_sweep` is unchanged. Re-audited for #9794's GC diagnostics: `gc/mod.rs` gains `mod diag_sites;` / `mod survival_diag;`, a re-export, a `diag_sites::full_started(...)` call at TRIGGER time (before mark propagation begins), and exit-time reporting. Nothing executes between mark-complete and sweep-entry, so the window is unchanged. Re-audited 2026-09-06 for the retained array-growth verifier fix: the cycle.rs change passes the existing non-copying evacuation verifier an explicit all-forwarded policy. That call remains in minor finalization, outside the synchronous full-cycle census window; its root and heap reads do not allocate GC objects, move objects, or invoke JS callbacks. The mark-complete and sweep-entry boundaries are unchanged. Re-audited 2026-09-05 after #9830 touched `gc/policy.rs`. That change is (a) six `thread_local! {` blocks rewritten as `crate::perry_thread_local! {` and (b) one `#[cfg(test)]` accessor listing the trigger path's hot-slot indices. The macro keeps the same storage, the same `.with()` at every read and write, and the same destructor registration (the teardown guard exists exactly when `needs_drop` holds, which is what `std::thread_local!` already decided); no value, predicate or branch in the file changes, so no mark or sweep control flow does. The one new behaviour is on a declaration's FIRST read: `HotKey::resolve_and_cache` takes a mutex and allocates a key through the GLOBAL allocator. Even if a first read landed inside this window it would be sound \u2014 the window's contract is that nothing relocates and no JS callback runs, and a mimalloc allocation does neither. `census_pass1_if_armed` is still inside `step_mark_propagation` and `census_take_if_armed_at_full_sweep_start` inside `step_sweep`; the bracketing is untouched. Re-audited 2026-09-06 (train132) after #9860 and #9845 touched `gc/mod.rs`. Both hunks are re-export lists and nothing else: #9860 adds `idle_reclaim_elapsed_starts` / `IDLE_RECLAIM_REARM_MS`, and #9845 adds `owner_is_dead_copied_minor_from_space_of_type`. No mark or sweep control flow changes. #9845's substantive work sits in `gc/oldgen.rs` and `gc/copying.rs`, neither pinned: the copying-minor arm (`finalize_dead_copied_minor_from_space_regexps`) runs on a MINOR, which skips both census boundaries; the full-cycle arm (`collect_dead_registered_regexps_post_trace`, from `with_dead_collection_finalize`) walks the RegExp registry building a Vec of addresses \u2014 no GC allocation, no JS callback, so it cannot relocate the snapshot's subjects \u2014 and it is reached from the sweep body, i.e. AFTER `census_take_if_armed_at_full_sweep_start` has already `take()`n the snapshot out of the thread-local. The mark-complete -> sweep-entry window is unchanged.", + "why": "Real GC header addresses, deliberately untraced so the diagnostic does not keep its observed objects alive. Populated only at the end of mark propagation of a synchronous full cycle; consumed at sweep entry in the same run_to_completion invocation. The intervening full-cycle phases do not relocate or run JS callbacks. The Vec is used for membership comparisons and dropped with the census before sweep. Budgeted and minor cycles skip both boundaries. Pin re-audited 2026-09-05 after #9760 touched `gc/mod.rs`: that change is `mod heap_stats;` plus a `pub(crate) use` re-export and alters no mark/sweep control flow. `heap_stats()` is reached only from `js_bun_jsc_heap_stats` (the JS-facing `bun:jsc.heapStats()`), i.e. from mutator code, never inside a cycle, and its own module contract forbids allocation or collection during its walk. The mark-complete \u2192 sweep-entry window is unchanged. Re-audited 2026-09-05 (train125) after #9769 and #9771 touched pinned files. #9769 adds one `reg_scanner!` registration to `gc/mod.rs`; #9771 adds a feature-gated `alloc_census_init()` there and a feature-gated Rust-heap dump inside `take_census`. `alloc-census` is not in the default feature set, and decisively: `census_take_if_armed_at_full_sweep_start` does `PASS1_MARKED.with(|p| p.borrow_mut().take())` BEFORE calling `take_census`, so the snapshot has already left the thread-local by the time #9771's code runs \u2014 it cannot affect the window. Neither change alters mark/sweep control flow. Re-audited 2026-09-06 after #9831 touched `gc/policy.rs`. Its hunks are (a) the tiny-parse pressure guard's pricing (`tiny_parse_pressure_headroom_bytes`, `tiny_parse_pressure_due*`, a `Cell` byte-count base) consulted from JSON.parse's mutator-side boundaries (`gc_bump_malloc_trigger`, `gc_collect_pending_suppressed_parse`, `gc_schedule_parse_boundary_collection_if_pressure`), none of which is reachable from inside a cycle, and (b) one extra `Cell` store in `note_collection_finished_arena_occupancy`, which runs from `publish_reclaim_outcome` in the Publish subphase \u2014 after `step_sweep` has already consumed the snapshot. Mark/sweep control flow between `census_pass1_if_armed` and `census_take_if_armed_at_full_sweep_start` is untouched. Re-audited 2026-09-05 (train126) after #9755 restructured `gc/cycle.rs`. Its hunks are all root-scan machinery (`RootScanSubphase`, `RootScanCycleState`, the mutable-scanner iteration state), which runs BEFORE mark propagation completes; `gc/mod.rs` gains only a `mod young_log;` declaration. The bracketing is unchanged \u2014 `census_pass1_if_armed` is still inside `step_mark_propagation` and `census_take_if_armed_at_full_sweep_start` inside `step_sweep` \u2014 and a synchronous full mark-sweep still moves nothing between them. Re-pinned 2026-09-05 for the #9740 hot-TLS conversion of this file: the sole change is `thread_local!` \u2192 `crate::perry_thread_local!`, a macro-name swap with identical declaration syntax and `.with()` call sites. No control flow, no phase boundary, and no storage semantics change. Re-audited 2026-09-06 (train128) after #9794's GC diagnostics touched `gc/mod.rs` and `gc/policy.rs`: both gain diagnostic module declarations and counters only \u2014 no mark/sweep control flow, and the census bracketing in `step_mark_propagation` / `step_sweep` is unchanged. Re-audited for #9794's GC diagnostics: `gc/mod.rs` gains `mod diag_sites;` / `mod survival_diag;`, a re-export, a `diag_sites::full_started(...)` call at TRIGGER time (before mark propagation begins), and exit-time reporting. Nothing executes between mark-complete and sweep-entry, so the window is unchanged. Re-audited 2026-09-06 for the retained array-growth verifier fix: the cycle.rs change passes the existing non-copying evacuation verifier an explicit all-forwarded policy. That call remains in minor finalization, outside the synchronous full-cycle census window; its root and heap reads do not allocate GC objects, move objects, or invoke JS callbacks. The mark-complete and sweep-entry boundaries are unchanged. Re-audited 2026-09-05 after #9830 touched `gc/policy.rs`. That change is (a) six `thread_local! {` blocks rewritten as `crate::perry_thread_local! {` and (b) one `#[cfg(test)]` accessor listing the trigger path's hot-slot indices. The macro keeps the same storage, the same `.with()` at every read and write, and the same destructor registration (the teardown guard exists exactly when `needs_drop` holds, which is what `std::thread_local!` already decided); no value, predicate or branch in the file changes, so no mark or sweep control flow does. The one new behaviour is on a declaration's FIRST read: `HotKey::resolve_and_cache` takes a mutex and allocates a key through the GLOBAL allocator. Even if a first read landed inside this window it would be sound \u2014 the window's contract is that nothing relocates and no JS callback runs, and a mimalloc allocation does neither. `census_pass1_if_armed` is still inside `step_mark_propagation` and `census_take_if_armed_at_full_sweep_start` inside `step_sweep`; the bracketing is untouched. Re-audited 2026-09-06 (train132) after #9860 and #9845 touched `gc/mod.rs`. Both hunks are re-export lists and nothing else: #9860 adds `idle_reclaim_elapsed_starts` / `IDLE_RECLAIM_REARM_MS`, and #9845 adds `owner_is_dead_copied_minor_from_space_of_type`. No mark or sweep control flow changes. #9845's substantive work sits in `gc/oldgen.rs` and `gc/copying.rs`, neither pinned: the copying-minor arm (`finalize_dead_copied_minor_from_space_regexps`) runs on a MINOR, which skips both census boundaries; the full-cycle arm (`collect_dead_registered_regexps_post_trace`, from `with_dead_collection_finalize`) walks the RegExp registry building a Vec of addresses \u2014 no GC allocation, no JS callback, so it cannot relocate the snapshot's subjects \u2014 and it is reached from the sweep body, i.e. AFTER `census_take_if_armed_at_full_sweep_start` has already `take()`n the snapshot out of the thread-local. The mark-complete -> sweep-entry window is unchanged. Re-audited 2026-09-07 for the regex census rows: all new work is in `take_census` after `census_take_if_armed_at_full_sweep_start` has taken PASS1_MARKED out of TLS; neither boundary nor the intervening cycle control flow changed.", "window": { "start": { "file": "crates/perry-runtime/src/gc/census.rs", @@ -291,9 +291,9 @@ "function": "run_to_completion" }, "sources": { - "crates/perry-runtime/src/gc/census.rs": "388414f9629f196e84673e91bebd04bdcdcabdaa180252d2dfe4b82d1b49ca5a", + "crates/perry-runtime/src/gc/census.rs": "0fd14b011bdbe0ae561e105d6d1acf0e9cc03e8a86ddef12685e307d1adee55a", "crates/perry-runtime/src/gc/cycle.rs": "2e2f5adca2229f74409e01a1cb571e2147cd8a33f58d0976711fce98d4777309", - "crates/perry-runtime/src/gc/mod.rs": "43523b66595c61516ef6fcd4139d3ec5b4768a13c46ae1470c1d45481eacfdd9", + "crates/perry-runtime/src/gc/mod.rs": "de687fa8e02b32e72c340e5b18ebb1ea660dd9da0179659f8d4c3d7f79a93edc", "crates/perry-runtime/src/gc/policy.rs": "dc9242ed40c0aa9c411d1ec0235c0219c6716dd82d56eb4d46578f7e889825d2", "crates/perry-runtime/src/gc/progress.rs": "a5ad3971bbe4047229ca57325234780daa85921dbc778e1c08dff4ad07ccfb96" } @@ -389,6 +389,54 @@ "verdict": "not_a_gc_pointer", "why": "RegExp construction/exec diagnostics (`PERRY_REGEX_DIAG`), off unless armed. Every field is a counter or an `Instant` except `per_pattern: HashMap`, whose VALUE is Rust-owned (an owned prefix `String`, a flags `String`, counters). The KEY is a pattern `StringHeader` address, so it is a heap address \u2014 but it is used ONLY as an opaque grouping id and is never dereferenced: `PatStat::prefix` and `byte_len` are filled from the `&[u8]` argument at first insert, never by reading the key. Nothing here is traced, rooted or rewritten. The one consequence of the address being reused after a pattern dies is that two patterns' diagnostic counters merge into one row \u2014 an inaccuracy in an off-by-default diagnostic, with no collector implication. Distinct from `PASS1_MARKED`, whose addresses ARE walked and which therefore carries `non_moving_snapshot` with a pinned window." }, + { + "file": "crates/perry-runtime/src/regex/site_test.rs", + "name": "TEST_NO_ALLOC", + "verdict": "test_only", + "why": "A cfg(test) counter for allocation-free site services; it contains only a u64 tally." + }, + { + "file": "crates/perry-runtime/src/regex/site_test.rs", + "name": "TEST_DECLINED", + "verdict": "test_only", + "why": "A cfg(test) counter for site-validation declines; it contains only a u64 tally." + }, + { + "file": "crates/perry-runtime/src/regex/site_test.rs", + "name": "TEST_DECLINED_PATCHED", + "verdict": "test_only", + "why": "A cfg(test) counter for patched-prototype declines; it contains only a u64 tally." + }, + { + "file": "crates/perry-runtime/src/regex/site_test.rs", + "name": "TEST_DECLINED_CALLEE", + "verdict": "test_only", + "why": "A cfg(test) counter for callee-identity declines; it contains only a u64 tally." + }, + { + "file": "crates/perry-runtime/src/regex/site_test.rs", + "name": "TEST_DECLINED_NON_LITERAL", + "verdict": "test_only", + "why": "A cfg(test) counter for non-literal factory declines; it contains only a u64 tally." + }, + { + "file": "crates/perry-runtime/src/regex/site_test.rs", + "name": "TEST_ALLOCATIONS", + "verdict": "test_only", + "why": "A cfg(test) counter for site-entry header allocations; it contains only a u64 tally." + }, + { + "file": "crates/perry-runtime/src/regex/site_test.rs", + "name": "DIRECT_G", + "verdict": "test_only", + "why": "A cfg(test) static whose immortal address supplies a synthetic direct global-regex site key." + }, + { + "file": "crates/perry-runtime/src/regex/site_test.rs", + "name": "MEMBER_CALL", + "verdict": "test_only", + "why": "A cfg(test) static whose immortal address supplies a synthetic namespace-member call site key." + }, { "file": "crates/perry-runtime/src/intl/segments_view.rs", "name": "DECLINE_EMPTY", @@ -704,13 +752,6 @@ "scanner": "regex::regex_header_moved_for_gc / regex_header_clear_dead_for_gc (regex.rs), the RegExp move/death hooks the copying minor and sweep invoke", "why": "Address-KEYED owner set, not a root: the key is rekeyed when the RegExpHeader moves and removed when it dies; it never keeps the header alive. Reached from GC hooks, not from a registered scanner, so the walk misses it." }, - { - "file": "crates/perry-runtime/src/regex.rs", - "name": "REGEX_SOURCE_TABLE", - "verdict": "covered_elsewhere", - "scanner": "regex::regex_header_moved_for_gc / regex_header_clear_dead_for_gc (regex.rs)", - "why": "Address-KEYED owner table (owned String copies of pattern/flags); rekeyed on move, cleared on death, same hooks as REGEX_POINTERS." - }, { "file": "crates/perry-runtime/src/regex.rs", "name": "VALIDATED_PATTERNS", @@ -3409,10 +3450,6 @@ "file": "crates/perry-runtime/src/native_arena.rs", "name": "VIEW_REGISTRY" }, - { - "file": "crates/perry-runtime/src/node_http2_constants.rs", - "name": "SENSITIVE_HEADERS_SYMBOL" - }, { "file": "crates/perry-runtime/src/node_repl.rs", "name": "RECOVERABLE_ERRORS"