Skip to content

run-019 epoch-2 verdict: 2.7397, gate open, WO closed #243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ronaldtse merged 1 commit into from
Oct 5, 2026
Merged

run-019 epoch-2 verdict: 2.7397, gate open, WO closed #243

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions TODO.sota-2026/04-stoicheia-diffusion-wo.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,3 +89,21 @@ Gap to the teacher tier (2.2864): 1.30pp.

**Teacher-tier phase (run-019):** same recipe on the ByT5-LARGE
encoder — the SOTA-dedicated gate (beat 2.2864) is the target.

**run-019 epoch-2 (final, 2026-10-05):** full-set **Total DER 2.7397
/ Morph 1.6717** (epoch-1: 3.0078/1.8486). SOTA-dedicated gate (2.2864)
NOT cleared — +0.45pp — and the per-epoch gain curve (−1.17 small,
−0.27 here) makes a third epoch unfavorable. Podium: r7 2.2864 >
plane-large-e2 2.7397 > plane-large 3.0078 > Gemini-Flash 3.1926.
Teacher-tier phase CLOSED on evidence: the plane architecture's edge is
the on-device frontier (run-018), not the teacher tier. The conditional
run-020 full-corpus labeling pass is likewise closed (its trigger was
"only if epoch-2 falls short" — it did; but its own reading stands:
unique-data volume is the lever, and the frontier model run-018 already
holds the efficient tier).

**WO status: ENTERED, executed, closed.** Deliverables shipped:
haraqat_planes.py (byte-exact split/render), run-017/018/019/020
verdicts, secryst-py PlaneModel runtime (PR merged), ONNX export path.
Open packaging: plane v2/large artifact zips + index entries + HF
publish (after this verdict).
17 changes: 17 additions & 0 deletions docs/RESULTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1072,3 +1072,20 @@ loses to raw-text diversity at scale (run-018's 551k unique units →
supervision source. A full-corpus teacher labeling pass (~15-20h GPU)
is the conditional distill follow-up, only if the epoch-2 arm falls
short of the 2.2864 gate.

## run-019 epoch-2 — 2.7397: second epoch closes a third of the gap, SOTA gate still open (2026-10-05)

The epoch-2 extension of plane-large scores full-set **Total DER
2.7397 / Morph 1.6717 / Total WER 8.6083** — down from 3.0078/1.8486
at 1 epoch (−0.27pp, −9%). The dedicated podium becomes r7 2.2864,
plane-large-e2 2.7397, plane-large 3.0078: still ahead of Gemini-Flash
(3.1926), still behind the teacher (+0.45pp). A third epoch is not
scheduled — the per-epoch gain (−1.17pp small tier, −0.27pp here) is
shrinking faster than the 0.45pp gap.

Process note: the first epoch-2 eval was invalid — it resumed from the
epoch-1 eval progress file, found every window already "saved", and
re-scored epoch-1 predictions (bit-identical 3.0078). Fixed by a
per-marker progress file (EVAL_DONE_E2B) and re-run in full; the
number above is the corrected measurement. The 2026-10-04 epoch-1
verdict was a fresh eval and is unaffected.
Loading