Skip to content

docs(RESULTS): static-int8 paired CI — not separated; release bar met #218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ronaldtse merged 1 commit into from
Sep 17, 2026
Merged

docs(RESULTS): static-int8 paired CI — not separated; release bar met #218

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/RESULTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -805,3 +805,22 @@ release-scale bar: re-run this gate with --out so the delta ships
with a paired-bootstrap CI. The point estimate stands recorded; the
export path change is small and the calibration corpus recipe is in
scripts/static_int8_experiment.py.

## Static-int8 release bar met: the paired CI (2026-09-17)

Both artifacts scored full-set with per-row predictions saved
(dynamic 4.5619; static 4.5952 — its second run, inside the drift
band with the first's 4.6241). Sentence-level paired bootstrap
(seed 42, n=1000, the campaign's standard) on the per-item DER delta:

**static − dynamic = +2.71pp, CI95 [−1.54, +7.11] (n=1195)** — the
interval crosses zero: not separated. Micro aggregates differ by
0.03pp; the macro point is dominated by short rows where a few haraqat
swings are a large per-sentence percentage.

Verdict by the standing rule: static and dynamic are
quality-indistinguishable at our measurement power, and static
carries +8% CPU decode speed. The re-export decision (index-v6,
golden regen, browser-size composition gated separately) is now fully
informed; the export path is one command (modal_export::static,
merged in PR #217).
Loading