fix(gc): re-baseline the whole-arena trigger after a malloc-pressure minor (#9840)

changelog.d/9838-tiny-parse-pressure-pricing.md

@@ -0,0 +1,37 @@
+**The tiny-parse pressure guard now prices the collections it forces by the
+adaptive step's productivity backoff (#9831).** On the compiled claude-code
+TUI a 3300-character streamed reply spent 30–41 s of CPU in the base arm and
+27.8–29.2 s with the fix (mean −19 %, every interleaved pair a win), with
+post-turn and post-idle RSS flat within the base's own spread and peak RSS
+unchanged.
+
+#9831 measured the `ArenaBytes` arm firing 51 times in one 66-delta reply,
+each collection freeing a median 131 KB, while the adaptive step sat
+saturated at 1 GiB — and located the discarded backoff in the arm's own
+ceiling clamp. That clamp was not what re-fired the arm: between two firings
+the arena grew a few hundred KB against a trigger armed 16–128 MB above the
+post-collection total. What pulled the trigger down was the tiny-parse
+pressure guard, which after every `JSON.parse` growing the arena by ≤ 1 MB
+tested the absolute `arena_in_use_bytes() >= 48 MB` and, if it held, set the
+trigger to "now". That is a quantity no collection can lower below the live
+set, so on a heap that sits above it permanently every small parse (one per
+SSE delta) forced a minor whose backoff nothing read — #9589's shape one
+trigger over.
+
+The guard now also requires the arena to have grown, since the last
+collection of any kind ended, by a headroom priced from the step: the step
+rescaled so its power-on value buys the 16 MB headroom floor and each
+doubling the arm's clamp discards buys one more doubling, bounded by the
+trigger ceiling. A productive collection keeps today's cadence; an
+unproductive one earns room. The parse-boundary collector re-prices a
+pending request so a collection that already satisfied it is not followed by
+a second. `PERRY_GC_DIAG=1` gains a `[gc-tiny-parse] forced collection …`
+witness line. The arm's own arithmetic is unchanged and now documents why
+(pricing it directly was measured at −10.8 % CPU for +22 % footprint, the
+issue's refuted branch).
+
+Validation: `test_memory_json_churn.ts` (the guard's motivating shape) is
+byte-identical in output and RSS in all four GC modes; 48/48 `test_gap_gc_*`
+and 8/8 `test_gap_json_*` pass; nine new `gc::tests::tiny_parse_pressure`
+tests pin the pricing and the predicate, sabotage-proved against both the
+old absolute guard and a raw-step pricing.

changelog.d/9840-arena-trigger-rebaseline-symmetry.md

@@ -0,0 +1,76 @@
+### Fixed
+
+- **A nursery collection triggered by malloc pressure now re-baselines the
+  whole-arena GC trigger, as `GC_NEXT_TRIGGER_BYTES`'s own contract already
+  said it did.** The cell is documented as "bumped after each
+  `gc_collect_inner` based on collection effectiveness". It was not:
+  `gc_finish_arena_trigger_collection` re-baselined it, and
+  `gc_finish_malloc_trigger_collection` — the finisher for the *same nursery
+  collection* with the malloc sweep added — did not. So the whole-arena
+  threshold was measured from the last **arena-kind** collection rather than
+  from the last collection, and a run of `MallocCount` minors could walk the
+  arena total across a threshold nothing had refreshed.
+
+  The asymmetry predates the budgeted split (`9d3bd2e3b`'s pre-split
+  `gc_check_trigger` had the same two branches) and is correct in the *other*
+  direction, which is unchanged and still pinned by
+  `test_gc_check_trigger_copied_minor_without_malloc_sweep_preserves_malloc_trigger`:
+  an arena minor that skips the malloc sweep must not move the malloc trigger.
+  A `MallocCount` minor has no such exemption — it swept the arena.
+
+  Measured on the perry-compiled claude-code TUI (`PERRY_GC_DIAG=1`, per
+  firing, four 3300-character captures across two independently built
+  binaries): the streaming turn ran a strict 6:1 pattern in which six
+  `MallocCount` minors promoting ~3.2 MB each crossed the stale arena
+  threshold *inside the sixth minor*, and at the very next safepoint the arena
+  arm fired on a nursery of **856 bytes** (`promoted_bytes=216
+  freed_bytes=640`) — one collection in seven, paying the whole
+  per-collection fixed cost (root scan, side-table prune, dirty-page restore)
+  to free 640 bytes.
+
+  **The length matters and every figure here states it.** Shape (b) needs a
+  run of promoting `MallocCount` minors to walk the total across the stale
+  threshold, so it exists on the long reply only: the 3300-character captures
+  run 48–62 `MallocCount` firings each, and the 400-character capture runs
+  **zero** (its collections are all `ArenaBytes` plus a handful of
+  `OldGenBytes`). At 400 characters this change is therefore expected to be
+  flat on every counter, and that is a prediction rather than a hope — there is
+  no producer for the shape at that length, with or without the in-flight
+  change that moves `RegExpHeader`s (the arm's only measured input on this
+  program) to the nursery.
+
+  Full collections are deliberately excluded: after a full that released
+  blocks, the un-moved trigger sits *further* above the new total, which is the
+  conservative direction, and a full's cadence belongs to the old-generation
+  band rather than to this arm.
+
+  This is the same symmetry #9831 gave the tiny-parse pressure guard's base
+  cell, applied to the one pacing quantity still keyed to a single collection
+  kind. The two cells are in different units on purpose and stay that way: the
+  guard's base is `arena_in_use_bytes()` (bump offsets, what it reads at each
+  parse boundary); this trigger's base is `arena_total_bytes()` (committed),
+  which is what `next_arena_trigger_base()` is compared against.
+
+  One coupling beyond the trigger, stated because it touches a fix that landed
+  hours earlier: `GC_STEP_BYTES` had exactly one production writer — the arena
+  finisher — and #9831 made it an *input* to the tiny-parse pressure guard's
+  headroom. Scoring a `MallocCount` minor's productivity therefore moves that
+  guard too. That is the same symmetry rather than a side effect (the step is
+  documented as "collection effectiveness", not "arena-kind collection
+  effectiveness"), and on the compiled claude-code TUI it moves the guard in
+  the *conservative* direction. Estimated over 209 `MallocCount` firings in the
+  four 3300-character captures, `pct_freed` has a median of 4–5 % and lands
+  `<10 %` in 194 cases, `10–24 %` in 10, `25–84 %` in 5 and `>84 %` in none —
+  so 93 % of these collections take the "< 10 % → double" band and push the
+  step up, which raises the guard's headroom. `[gc-arena-rebaseline]` carries
+  `pct=` and `step=` for both arms so this is read off a capture rather than
+  estimated from a neighbouring diagnostic, which is all the pre-fix diag
+  allowed.
+
+  The arm's dueness predicate is byte-identical, so when it is due it still
+  fires the same collection. `PERRY_GC_ARENA_REBASELINE_ALL=0` restores the old
+  asymmetry — its OFF state is asserted in CI as the knob kill-policy requires,
+  by a test that is simultaneously the sabotage proof for the two ON-state
+  tests (the OFF branch *is* the deleted call) — and `PERRY_GC_DIAG=1` gains a
+  `[gc-arena-rebaseline] arm=…` line attributing each re-baseline to the
+  finisher that performed it.