Skip to content

fix(gc-ratchet): apply documented counter exclusions after measurement #9813

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
proggeramlug wants to merge 1 commit into from
Closed

fix(gc-ratchet): apply documented counter exclusions after measurement #9813

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 64 additions & 10 deletions benchmarks/gc_ratchet/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,13 +71,14 @@ sessions x 7 repeats (21 runs per probe), plus 5 traced runs per probe:
The GC accounting family is parsed from `PERRY_GC_DIAG=1` output in a separate,
untimed pass; enabling the trace was verified not to change `heap_used_bytes`,
so the traced pass observes the same collector the untimed pass measures. The
harness takes two traced runs on every invocation and fails if they disagree —
that is the harness proving, each time it runs, that the counters it is about to
gate on really are deterministic.
harness records two traced runs on every invocation. `check` fails if a gated
counter disagrees, even within its tolerance band. Documented probe overrides
apply here too; the measurement keeps both samples and runs every later probe.

Retention and GC accounting are semantic: they are a function of the allocation
sequence and collector policy, not of CPU speed, core count, or machine load.
That is why they can be gated on a shared CI runner and memory and time cannot.
Retention and GC accounting usually transfer across machine classes because
they describe allocation and collector work rather than CPU speed. They must
still demonstrate repeatability: block placement can change the collection
point and therefore the live cohort, as the #9790 investigation below shows.

## A probe may declare the collector it is a probe *of* (the large-Eden arm)

Expand Down Expand Up @@ -332,10 +333,63 @@ carrying a non-deterministic gating cell cannot be *pinned*. Before #7554 the
rule existed only in `tests/test_gc_ratchet.py`, which is why a bad pin could
be committed and only wedge CI afterwards.

The section is currently **empty**, which is the goal state and not an
oversight. Its one entry — `12_large_live_set.heap_used_bytes` — was deleted by
#7558, which removed the *cause* rather than the cell. That is rule 4 working
as designed.
The former `12_large_live_set.heap_used_bytes` entry was deleted by #7558,
which removed its cause. The two current entries exclude only
`07_array_grow_evacuate.copied_bytes` and `.freed_bytes`, with the following
evidence. Removing the placement dependency would allow deleting these entries.

### Array growth: placement changes the collection point (#9790)

Twenty-one executions of one unchanged binary produced two counter tuples.
Every stdout matched Node 26.5.1. Retained heap, arena capacity, minor/step
counts, copied/promoted object counts, and promoted bytes were identical.
The [receipt](evidence/9790-array-growth-pacing.json) records raw byte-counter
samples, stable metrics, and compiler/runtime/probe hashes.

Temporary logging in `move_young` cross-checked the counters against the actual
headers moved. Only the fifth minor's live cohort differed: an 8,208-byte array
with length 640, capacity 1,024, and seed 5,207 was reached through the ring's
remembered edge in one run. The other run instead copied a 144-byte array with
length 1, capacity 16, and seed 5,240 from the native stack. The difference is
exactly **8,064 bytes**, while both runs copy one object. Summing the other
copied headers gives the same result in both runs.

The freed-byte difference also balances against the actual from-space usage:

| Fifth minor | Higher copied bytes | Lower copied bytes |
|---|---:|---:|
| Eden bytes | 16,775,936 | 16,776,080 |
| Active survivor bytes | 517,248 | 517,248 |
| Copied bytes | 525,312 | 517,248 |
| Promoted bytes | 131,328 | 131,328 |
| Freed bytes | 16,636,544 | 16,644,752 |

In each column, freed = Eden + active survivor - copied - promoted; malloc
reclamation is zero. There is no unexplained accounting remainder.

Block-boundary logging located the pacing cause. `arena_cell_alloc` checks GC
pressure when its current block cannot satisfy an allocation. Promotion walks
address-keyed root tables, so equal total promoted bytes can fill individual
old blocks differently. In the higher-copy run, an old block overflowed on a
4,112-byte growth request after young occupancy had crossed the cap. In the
lower-copy run, that old-block rollover occurred earlier, below the cap; the
next nursery block overflow armed collection while starting the next array.
`js_array_grow` can fall back to old allocation when a growth cannot fit the
current nursery block, connecting this workload to that old-block geometry.
The earlier dirty-page statistics also vary with placement; they alone would
not have established the cause.

The two byte counters therefore describe real, placement-dependent work on
this workload. They remain measured and displayed, with their existing bands;
only their ability to fail the gate is excluded. The probe's correctness,
retention, cycle counts, copied/promoted object counts, and promoted bytes
remain gated, as do these byte counters on every other probe. Runtime pacing
and the probe's allocation sequence are unchanged.

The determinism check now runs in `check`, where the baseline's reviewed
overrides are available. An unlisted disagreement still fails even if its
median equals the baseline, and the full measurement artifact survives for
inspection. `assemble` still refuses to pin any nondeterministic gated cell.

### What that probe's non-determinism was, and where it went (#7558)

Expand Down
37 changes: 30 additions & 7 deletions benchmarks/gc_ratchet/baseline/gc-ratchet-v1.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,12 +103,12 @@
"job red. Every entry carries evidence that is checked, not merely stored --",
"at least 21 runs (the same number every band above is justified by) and a",
"spread that is actually non-zero, so a cell cannot be excluded on a hunch.",
"The section is EMPTY, and that is the goal state. Its one entry --",
"12_large_live_set.heap_used_bytes, added by #7554 -- was deleted by #7558,",
"which removed the cause rather than the cell: explicit gc() no longer forces",
"the conservative native-stack scan, so that reading is bit-identical again",
"and gates again. An empty section is not a disarmed rule; the evidence",
"checks and the never-gate-nothing rule still fail any entry added back.",
"The former 12_large_live_set.heap_used_bytes exclusion was removed by #7558.",
"#9790 excludes only 07_array_grow_evacuate.copied_bytes and freed_bytes:",
"promotion order changes block packing and when nursery pressure is checked.",
"The receipt records 21 runs and an allocation-level accounting cross-check.",
"Both cells remain measured and reported. All other cells retain their bands.",
"Deleting the placement dependency means deleting these exclusions.",
"",
"#7559 -- A heap_used_bytes band used to NOT be a statement about how much",
"the collector retained. The reading is taken after the probe's own gc(),",
Expand Down Expand Up @@ -295,7 +295,30 @@
"rationale": "GATED HERE ONLY. Worst cross-session spread of medians-of-7 was 0.751% on an idle box (load 1.7-2.0); worst raw within-session spread was 5.3%, which the median-of-7 damps out. 10% is ~13x the cross-session figure and ~2x the worst raw spread, so it will not fire on scheduler jitter but will catch the tens-of-percent slowdown a whole-stack conservative scan would introduce. The 15 ms floor covers the fastest probe (126 ms)."
}
},
"probe_overrides": {}
"probe_overrides": {
"07_array_grow_evacuate": {
"copied_bytes": {
"gating": false,
"rationale": "NOT GATED ON THIS PROBE (#9790). Address-dependent promotion order changes old-block packing and the allocation boundary where nursery pressure is checked. The fifth minor observes a different live cohort: a completed 8,208-byte array or a new 144-byte array. Header-size sums and from-space reclamation account for the byte deltas; this is placement-dependent pacing, not an accounting discrepancy. Keep this cell visible; correctness, retention, cycle counts, object counts and promoted bytes remain gated. See evidence/9790-array-growth-pacing.json and the README investigation.",
"evidence": {
"observed_runs": 21,
"observed_spread": 8064,
"measured_on": "2026-09-05, macOS arm64, Perry 0.5.1520; 21 executions of one unchanged binary. Compiler/runtime hashes and raw counters: benchmarks/gc_ratchet/evidence/9790-array-growth-pacing.json.",
"issue": "https://github.com/PerryTS/perry/issues/9790"
}
},
"freed_bytes": {
"gating": false,
"rationale": "NOT GATED ON THIS PROBE (#9790). Address-dependent promotion order changes old-block packing and the allocation boundary where nursery pressure is checked. The fifth minor observes a different live cohort: a completed 8,208-byte array or a new 144-byte array. Header-size sums and from-space reclamation account for the byte deltas; this is placement-dependent pacing, not an accounting discrepancy. Keep this cell visible; correctness, retention, cycle counts, object counts and promoted bytes remain gated. See evidence/9790-array-growth-pacing.json and the README investigation.",
"evidence": {
"observed_runs": 21,
"observed_spread": 8208,
"measured_on": "2026-09-05, macOS arm64, Perry 0.5.1520; 21 executions of one unchanged binary. Compiler/runtime hashes and raw counters: benchmarks/gc_ratchet/evidence/9790-array-growth-pacing.json.",
"issue": "https://github.com/PerryTS/perry/issues/9790"
}
}
}
}
},
"notes": "Re-pinned for #8122-recover: the allocation census before minor #0 object-denominates the first nursery cap (first cycle fires earlier on small-object workloads), one descriptor lookup per traced object, untraced-promotion threshold 990 -> 980. Every GC-accounting fingerprint shifts; retention improves on 12_large_live_set (-75%) and 13_large_eden_survivors moves +85 KB because its cycle 0 now holds up an in-place promotion at 581 permille (main at cap 49 retains 651 KB the same way).",
"probes": {
Expand Down
177 changes: 177 additions & 0 deletions benchmarks/gc_ratchet/evidence/9790-array-growth-pacing.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
{
"issue": 9790,
"measured_at": "2026-09-05",
"platform": "darwin-arm64",
"runtime_version": "0.5.1520",
"source": "benchmarks/gc_ratchet/probes/07_array_grow_evacuate.ts (unchanged workload)",
"run_env": {
"PERRY_GC_DIAG": "1",
"PERRY_GC_TRACE": "1"
},
"oracle": {
"version": "v26.5.1",
"matching_runs": 21,
"stdout": "probe:07_array_grow_evacuate\nchecksum:21891160\nsurvivors:94\n"
},
"binaries": {
"perry": {
"bytes": 152440456,
"sha256": "0e482d4099c396832d57acd1ed9a974107735901cafeeac609ae04ec7a3beb69"
},
"libperry_runtime.a": {
"bytes": 87787184,
"sha256": "f979e631e6e1d468db1b40bc85882660afe7c79d75c9fc11c18a71fe58c5d591"
},
"probe": {
"bytes": 16811144,
"sha256": "1f5e3b44052edf6c63fdb1cb51e3f62ec819db50435f264c6ef218da23f230d5"
}
},
"constant_counters": {
"minor_cycles": 5,
"step_cycles": 5,
"copied_objects": 6745,
"promoted_objects": 6197,
"promoted_bytes": 844600
},
"constant_retention": {
"heap_used_bytes": 1214784,
"heap_total_bytes": 25165824
},
"samples": [
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2639944,
"freed_bytes": 83567048
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2639944,
"freed_bytes": 83567048
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2639944,
"freed_bytes": 83567048
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2639944,
"freed_bytes": 83567048
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2639944,
"freed_bytes": 83567048
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2639944,
"freed_bytes": 83567048
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
},
{
"copied_bytes": 2648008,
"freed_bytes": 83558840
}
],
"diagnostic": {
"method": "Temporary move/header and block-boundary logging in runtime source at c7361c87c73738fe97cf352c75c9f00ce0a9b346; manually relinked the same generated object. Both original counter tuples and Node stdout reproduced. Diagnostic edits are not part of the runtime fix.",
"high": {
"copied_bytes": 2648008,
"freed_bytes": 83558840,
"fifth_minor_copied_bytes": 525312,
"fifth_minor_promoted_bytes": 131328,
"fifth_minor_freed_bytes": 16636544,
"fifth_minor_eden_bytes": 16775936,
"fifth_minor_survivor_bytes": 517248,
"extra_array": {
"seed": 5207,
"length": 640,
"capacity": 1024,
"header_size": 8208,
"source": "remembered_set"
},
"last_block_trigger": {
"generation": "Old",
"request": 4112,
"offset": 1045352,
"block_size": 1048576,
"young_bytes": 17293184
}
},
"low": {
"copied_bytes": 2639944,
"freed_bytes": 83567048,
"fifth_minor_copied_bytes": 517248,
"fifth_minor_promoted_bytes": 131328,
"fifth_minor_freed_bytes": 16644752,
"fifth_minor_eden_bytes": 16776080,
"fifth_minor_survivor_bytes": 517248,
"extra_array": {
"seed": 5240,
"length": 1,
"capacity": 16,
"header_size": 144,
"source": "mutable_root_slots/native_stack"
},
"last_block_trigger": {
"generation": "Nursery",
"request": 144,
"offset": 1048496,
"block_size": 1048576,
"young_bytes": 17293184
}
},
"conservation": "fifth_minor_freed_bytes = fifth_minor_eden_bytes + fifth_minor_survivor_bytes - fifth_minor_copied_bytes - fifth_minor_promoted_bytes in both arms; malloc freed bytes are zero. Total copied-byte delta = 8208 - 144 = 8064."
}
}
31 changes: 21 additions & 10 deletions benchmarks/gc_ratchet/gc_ratchet.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -721,9 +721,11 @@ def measure(

# Separate traced pass. PERRY_GC_DIAG writes one line per collection
# phase, which perturbs wall time, so it must not share a pass with
# the timing samples. Two traced runs are taken and required to
# agree: that is the harness proving, every time it runs, that the
# counters it is about to gate on are actually deterministic.
# the timing samples. Keep both traced runs, including disagreement.
# `check` owns the gating policy (and documented probe overrides),
# so it rejects disagreement in every counter that remains gated.
# Aborting here would bypass those overrides and discard all later
# probes before the gate could report their results (#9790).
traced = [
parse_gc_diag(
run_once(
Expand All @@ -733,12 +735,6 @@ def measure(
)
for _ in range(2)
]
if traced[0] != traced[1]:
differing = sorted(k for k in traced[0] if traced[0][k] != traced[1][k])
raise RatchetError(
f"{name}: GC counters are not deterministic across traced runs "
f"({', '.join(differing)}); they cannot be gated"
)
# Deliberately NOT rejecting minor_cycles == 0 here. A collector that
# has stopped running copying minors at all is the single largest
# regression this ratchet exists to catch, and it must surface as a
Expand Down Expand Up @@ -1862,6 +1858,19 @@ def evaluate(

for metric in ALL_METRICS:
tolerance = resolve_tolerance(tolerances, overrides, name, metric)
# The two traced samples are an independent premise of the band:
# even a one-byte disagreement inside the allowance invalidates a
# gated counter. Read the samples themselves, not a cached spread.
# Only an explicit probe override (or the profile) can exclude it.
current_samples = cur_entry["metrics"][metric].get("samples", [])
unstable_current = metric in GC_METRICS and tolerance.gating and (
len(current_samples) < 2 or len(set(current_samples)) != 1
)
if unstable_current:
failures.append(
f"{name}: {metric} is not deterministic across traced runs; "
"it cannot be gated"
)
# A cell the pinned artifact cannot support is demoted rather than
# trusted: comparing against a number whose own premise failed would
# dress a defect up as a verdict. The defect is already in
Expand All @@ -1882,7 +1891,9 @@ def evaluate(
else:
breach = False

if breach and quarantined:
if unstable_current:
status = "UNFIT (traced samples disagree)"
elif breach and quarantined:
status = "UNFIT (pinned cell unusable)"
elif breach:
status = "REGRESSION" if tolerance.gating else "drift (informational)"
Expand Down
10 changes: 6 additions & 4 deletions benchmarks/gc_ratchet/probes/07_array_grow_evacuate.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
// GC ratchet probe: array element storage growth and reallocation.
//
// A growing array repeatedly abandons its previous element storage, which is a
// separate allocation from the array header. Evacuation has to rewrite the
// header's pointer to the moved storage; per-object pinning would leave the old
// storage in place and fragment the region. The probe grows many arrays past
// A growing array replaces its inline header-plus-elements allocation and
// leaves a forwarding stub at the old address. Evacuation must repair holders
// of the current allocation. The probe grows many arrays past
// several reallocation boundaries, keeps a small live sample, cycles the rest
// through a ring so they are genuinely heap-allocated, then drops them.
// #9790: copied_bytes/freed_bytes remain informational on this probe because
// block placement changes when nursery pressure is checked. See the ratchet
// README and evidence/9790-array-growth-pacing.json; all other gates remain.

declare function gc(): void;

Expand Down
Loading
Loading