Skip to content

Commit 2c0cf86

Browse files
Ronald Tseronaldtse
authored andcommitted
docs: run-005-muon recorded — the 2x2 factorial closes at 5.2945
1 parent 006b6ad commit 2c0cf86

1 file changed

Lines changed: 15 additions & 0 deletions

File tree

‎docs/RESULTS.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -260,6 +260,21 @@ gap at this rung decomposes: ~0.70pp capacity + ~2.73pp optimization
260260
+ ~2.25pp residual (domain coverage). The vanilla+Muon factorial cell
261261
(run-005-muon) completes the decomposition.
262262

263+
### run-005-muon — factorial cell 4: vanilla + Muon (2026-08-28)
264+
265+
| Model | DER-CE (full 1,200) |
266+
|---|---|
267+
| ByT5-small + Muon (run-005) | 5.2945% |
268+
| ByT5-small + PKM + Muon (run-004) | 4.8287% |
269+
| ByT5-small + PKM + AdamW (run-003) | 7.5553% |
270+
| ByT5-small vanilla + AdamW (run-002) | 8.2590% |
271+
272+
The 2x2 closes cleanly: optimizer alone −2.96pp; memory alone −0.70pp
273+
(−0.47 under Muon); combined −3.43pp (60.4% of the 5.677pp canonical
274+
gap) — roughly additive, slightly sub-additive on memory. Residual
275+
~2.2pp is domain coverage. Optimization is the dominant recoverable
276+
term of the distillation gap at the ByT5-small rung.
277+
263278
Leaderboard context (SadeedDiac-25, Misraj evaluator, zero-skip,
264279
harakat-projected DER-CE): the teacher tier (r6, 580M) at 2.5793
265280
(reproduced at 2.5815, 2026-08-26) is the best dedicated model measured

0 commit comments

Comments
 (0)