perf(regex): close the backtracking cliff, allocation-free cache probes, engine prototype switch

changelog.d/9796-regex-backtracking-cliff.md

@@ -0,0 +1,43 @@
+### Performance
+
+- **A capture group no longer turns a pattern into a ReDoS.**
+  `repeat_matcher::capture_layout` takes a pattern off the linear `regex`
+  engine when ECMA-262's RepeatMatcher capture semantics are observable — a
+  capture group directly under a quantifier, or a capture inside a negative
+  lookaround. That routing is a correctness requirement (the linear engine
+  keeps the last value of a capture nested in a quantified group; the spec
+  clears it on every iteration), but the engine it routes to, `regress`, is a
+  classical backtracker with no step budget. So adding parentheses was enough
+  to fall off a linear-time path onto an exponential one:
+
+  | pattern | node | perry (before) | perry (after) |
+  |---|---|---|---|
+  | `/^(a+)+$/.test("a"×28 + "!")` | 4,798 ms | **16,522 ms** | **0 ms** |
+  | `/^(?:a+)+$/.test(…)` (same language, no capture) | 4,288 ms | 0 ms | 0 ms |
+
+  **6.3 %** of the 4,463 distinct regex literals across seven real bundles
+  take that route — claude-code 7.1 %, dayjs 25 %, luxon 29 % — including
+  shapes like `^[a-z][a-z0-9]*(-[a-z0-9]+)*$`.
+
+  The two engines accept exactly the same LANGUAGE for a pattern they both
+  compile; they disagree only about which capture assignment to report. So the
+  linear program is asked first (`linear_rules_out_match`), and when it proves
+  there is no match at or after the search offset — which is what every ReDoS
+  input is, a subject that ALMOST matches and then fails — the backtracker is
+  never entered. Every `&str`-subject entry point goes through
+  `lookup_repeat_matcher_for`: `test`, `exec`, `match`, `matchAll`, `search`,
+  `split` and `replace` with a string replacement. The gate disables itself
+  where the linear engine has no opinion (a pattern it could not compile holds
+  the never-match placeholder), which is exactly the lookaround shapes.
+
+  **This removes the reachable exponential case; it does not BOUND the worst
+  case.** A real step budget has to be counted by the backtracker, and
+  `regress` has none today (`fancy-regex`, by contrast, ships
+  `backtrack_limit: 1_000_000`). A 101-line patch adding one has been measured
+  — worst hostile search 51 s → 124 ms at a budget of 1,000,000, zero answers
+  changed across 13,389 real searches, upstream's own 544 tests unchanged — and
+  is open upstream as
+  [ridiculousfish/regress#177](https://github.com/ridiculousfish/regress/pull/177).
+  Until it lands and perry picks it up, do not read "cliff fixed" as "worst
+  case bounded".
+  (`quantified_capture_pattern_does_not_backtrack_on_a_non_matching_subject`)

changelog.d/9796-regex-borrowed-cache-keys.md

@@ -0,0 +1,26 @@
+### Performance
+
+- **Probing the compiled-program caches no longer materialises the key.** The
+  three thread-local caches were `HashMap<(String, String), _>`, and
+  `HashMap::get` needs a `&(String, String)` — so **every probe allocated two
+  Strings and copied the pattern text into them**, on a path that runs once per
+  RegExp OBJECT, and a JS regex literal evaluates to a fresh object every time
+  it is reached. A native-churn census of the claude-code binary (2026-09-05)
+  put `js_regexp_test` → `lookup_repeat_matcher` → `build_and_install_programs`
+  at **6,044 MB of 8,334 MB of estimated allocation with zero live bytes** —
+  73 % of all remaining native churn — split across the three probe sites: the
+  `get_or_compile_regex` probe (2,071 MB) and two `core::fmt::Formatter::pad`
+  frames (1,989 MB and 1,984 MB), which is what `.to_string()` on an `Arc<str>`
+  lowers to.
+
+  The caches are now keyed by `ProgramKey = (Arc<str>, Arc<str>)`. Every caller
+  that matters already holds those `Arc`s — `REGEX_SOURCE_TABLE` and
+  `regex::site_cache` share one allocation of a literal's text with every
+  header built from it — so a probe is two refcount increments and no
+  allocation at all. The two remaining `Arc::from` materialisations are on cold
+  paths: the syntax-error fallback in `js_regexp_new` (a pattern the linear
+  engine's parser refused, 7.7 % of real literals, once each) and
+  `RegExp.prototype.compile` (once per call from user code).
+
+  Hashing still walks the pattern bytes; the allocation is what the census
+  measured and what this removes.

changelog.d/9796-regex-engine-prototype-switch.md

@@ -0,0 +1,31 @@
+### Internal
+
+- **`PERRY_REGEX_ENGINE=regress` — a measurable tier-0 engine prototype.**
+  Routes every pattern through `regress` (the ECMAScript backtracker perry
+  already links for RepeatMatcher capture semantics) instead of only the ones
+  whose capture semantics require it, and installs a shared never-match
+  placeholder as the standard program so no NFA is built. Every exec-family
+  entry point already consults the repeat matcher first, so this exercises the
+  whole engine surface — `exec`, `test`, `match`, `matchAll`, `search`,
+  `split`, `replace` — without a second implementation.
+
+  It exists so the engine question is settled on measurements from a real
+  binary rather than on a corpus harness. Measured over 4,463 distinct regex
+  literals extracted from seven real bundles (two claude-code builds, ethers,
+  moment, dayjs, luxon, mongodb) with a tracking allocator and the programs
+  held live:
+
+  | engine | accepted | compile µs (med) | bytes/program (med) | corpus total |
+  |---|---|---|---|---|
+  | `regex` crate (tier 1 today) | 92.3 % | 48.5 | 12,492 | 136.7 MB |
+  | `regress` | **100 %** | **2.2** | **512** | **4.9 MB** |
+  | `fancy-regex` (tier 2 today) | 97.8 % | 59.2 | 12,623 | 146.6 MB |
+
+  node/V8, measured the same session, is ~2,600 bytes per program. A
+  differential over 4,119 patterns × 13 subjects (53,547 comparisons of match
+  presence, span and every capture span) found **0 disagreements** between the
+  linear engine and `regress`.
+
+  **Not a supported configuration**: the backtracker has no step budget, so a
+  pathological pattern can run unbounded. Off by default, one relaxed atomic
+  load when unset.