Skip to content

Commit 6ec7ca8

Browse files
author
Ronald Tse
committed
docs: E3 verdict in publication notes — adopted, gate exceeded 9x
1 parent 1012dbc commit 6ec7ca8

1 file changed

Lines changed: 10 additions & 9 deletions

File tree

‎docs/PUBLICATION-NOTES.md‎

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -72,15 +72,16 @@ structure but not accuracy; teacher beam-search unnecessary for
7272
Arabic; per-channel int8 rejected on measurement; the 30 MiB tier
7373
closed as infeasible without pretraining.
7474

75-
## In flight — slots reserved, no claims yet
76-
77-
### 9. Muon optimizer A/B on the memory student (E3)
78-
Same architecture/data/seed as E2, optimizer swapped (Newton–Schulz
79-
orthogonalized momentum; embedding-like params on AdamW per the E1
80-
access-pattern policy). Training CE ~0.007 vs AdamW's ~0.02 at equal
81-
steps, ~1.2s/step vs ~3.4s — DER eval in flight; adopt gate ≥0.3pp.
82-
Lands: run-004-pkm-muon/final_eval.json → EXPERIMENTS.md E3 → paper
83-
training-methods note.
75+
### 9. Muon optimizer A/B on the memory student (E3) — LANDED 2026-08-28
76+
**4.8287 vs 7.5553 full-set (−2.727pp from the optimizer alone); adopt
77+
gate (≥0.3pp) exceeded 9×. ADOPTED.** Training CE ~0.007 vs ~0.02 at
78+
equal steps, ~1.2s/step vs ~3.4s, no stability events. With E2 this
79+
completes a controlled decomposition of the distillation gap at the
80+
ByT5-small rung: ≈0.70pp capacity + 2.73pp optimization + 2.25pp
81+
residual (domain coverage). The strongest training-methods result in
82+
the paper — first controlled Muon measurement on byte-level seq2seq
83+
distillation. Paper: frontier section. EXPERIMENTS.md E3, RESULTS.md
84+
run-004. Factorial cell 4 (vanilla+Muon, run-005) queued.
8485

8586
### 10. Arabic news-domain adaptation (r7) — ID LANDED 2026-08-28
8687
**2.2864 / 1.3343 full-set windowed zero-skip — new best dedicated

0 commit comments

Comments
 (0)