Skip to content

gc-ratchet: assemble/validate can reject a valid artifact — distribution() is not idempotent under its own rounding (blocks re-pinning) #9834

Description

@proggeramlug

Hit while assembling the #9829 re-pin. assemble refused a clean 7-repeat
measurement:

gc-ratchet error: 03_cross_gen_writes: wall_ms summary is inconsistent with its samples;
                  05_closure_capture: wall_ms summary is inconsistent with its samples;
                  11_collect_at_depth: wall_ms summary is inconsistent with its samples

The measurement is not corrupt. The check is comparing two values that were
computed from differently-rounded inputs.

The mechanism

validate_artifact does if recorded != distribution(samples). But samples
as stored have already been through _clean (round to 6 dp), while the
recorded summary was produced by distribution from the raw samples. Most
fields survive that, because rounding a sample rarely moves a median or a
min/max. spread_pct does not:

spread = ordered[-1] - ordered[0]                       # raw
"spread_pct": _clean(100.0 * spread / median)           # raw / raw, then rounded

Recomputing from the stored (already-rounded) samples gives a different 6th
decimal:

probe recorded spread_pct recomputed from stored samples
03_cross_gen_writes 3.580602 3.580601
05_closure_capture 4.039540 4.039541
11_collect_at_depth 3.127911 3.127910

One unit in the last place, three times in one 14-probe run.

So distribution(x) != distribution(clean(x)), and the artifact check
assumes it is. Every other field is stable under the rounding; spread_pct is
the one that divides two rounded quantities and then rounds again.

Why it matters more than a cosmetic mismatch

  • It is fatal, not deferrable. The code comment above the check is explicit:
    "a summary that disagrees with its own samples is tampering or corruption, not
    unfitness: it stays fatal". So this does not demote a cell — it refuses the
    whole artifact.
  • It is nondeterministic: it fires only when the rounding happens to move
    the last digit, which depends on the timing samples. Three of fourteen probes
    this run; a different run will hit a different subset, or none.
  • It blocks re-pinning, which is exactly what gc-ratchet has been RED on main since 2026-08-18 (19 days); every labelled PR inherits it, and the failure has grown from 19 to 43 cells #9829 needs. Anyone trying to
    re-pin can lose an afternoon to an error message that says their measurement
    is corrupt when it is fine.
  • It affects float metrics only (wall_ms, and any other non-integer),
    which is why the integer GC counters never showed it.

Fix

Make the summary a function of the stored samples, so the check is comparing
like with like — either

samples = [_clean(v) for v in values]        # clean FIRST
ordered = sorted(samples)                     # then derive everything from these

or compute spread_pct from the already-cleaned spread and median rather
than the raw ones. Either makes distribution idempotent, which is what
recorded != distribution(samples) needs in order to mean what it says.

A regression test writes an artifact, reloads it, and asserts
distribution(loaded["samples"]) == loaded for every metric — the property the
validator depends on and nothing currently checks.

Provenance

Measurement: main @ d36a1af0c plus the probe repair in #9833, 7 repeats,
macOS arm64. The GC counters in that same measurement match CI byte-for-byte
(established repeatedly in #9829), so this is not a host artefact — it is the
rounding path.

Blocks: #9829 (re-pin). Related: #9832 / #9833 (the inert probe that had to be
fixed before pinning).

https://claude.ai/code/session_014UZWia6L37DpA93VLtNK9m

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions