[BUG] Preserve partial benchmark failures instead of publishing survivor-only stats

[lukekarrys]

Problem

scripts/clean-benchmarks.js treats total and partial failure differently:

• If every run fails, it emits the DNF sentinel.
• If only some runs fail, it drops the failed runs and recomputes mean, median, stddev, min, and max from the survivors.

That makes a 1-of-3 success look like a normal result. Because the 300-second timeout preferentially removes slow runs, the published value is biased downward, and a single survivor is reported with stddev: 0. The raw exit codes disappear after processing and are not retained in gh-pages history.

Issue #11 introduced failure filtering, but did not preserve partial-failure confidence. PR #145 alerts on raw failures for the vlt registry path only; it does not correct the processed data shown by the site or cover every variation and command.

Impact

This is a data-integrity issue affecting every chart, variation, and package manager. Flaky or timing-out commands can be published as deceptively fast, stable results.

Acceptance criteria

• Processed results preserve how many runs were attempted, succeeded, and dropped (for example, a dropped_runs field).
• The site visibly distinguishes partial results from complete results instead of presenting them as equally trustworthy.
• Any CI validation that needs raw exit codes runs before ./bench process.
• CPU fields are either recomputed from the same surviving runs or explicitly omitted/marked when that cannot be done consistently.
• Tests cover total failure, partial failure, and complete success.
• Historical/output schema compatibility is handled explicitly.

Source

Performance investigation: item 12 — partial-failure survivorship bias