Skip to content

Commit bed652e

Browse files
author
Ronald Tse
committed
docs: G2b verdict — 4.8231; the register-add neutralized the epoch gain
48k total with +18k classical Tashkeela at 6 epochs: flat vs the 3ep 4.8218 control, 0.25pp worse than news-only G2a 4.5701. Paired bootstrap +2.3717 CI 2.194-2.554; teacher reproduces 2.289. With E6 swap 5.8057 the domain-coverage attribution is negative in BOTH directions; Paper-B lever table gains E5/E6/G2b rows and the framing line is corrected. Verdict read from final_eval.json on the volume; recorded per TODO.substantiate item 04.
1 parent a042c5d commit bed652e

2 files changed

Lines changed: 39 additions & 8 deletions

File tree

‎docs/EXPERIMENTS.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -297,6 +297,27 @@ All rows passed the CER parity gate at release. Readings:
297297
shows a convergence-speed effect distinguishable from noise on the
298298
surviving curves; the DER gates remain the only verdicts.
299299

300+
## G2b — label-scale ADD rung (run-011-tashkeela-6ep) — verdict recorded 2026-09-04
301+
302+
- **Design** (ml #135): 48,000 total units — the control's news-mix
303+
plus 18,000 additional classical Tashkeela units — trained 6 epochs
304+
(G2a's epoch count). Spec `ara-diac-small-2-6ep-tashkeela`.
305+
- **Status:** COMPLETE — **4.8231** full-set windowed DER (n=1200;
306+
teacher reproduces 2.289; paired bootstrap student−teacher +2.3717,
307+
CI [2.194, 2.554]). From final_eval.json on the volume; run
308+
completed and evaluated by its owners; recorded here per
309+
TODO.substantiate item 04. Labels sha256 b59e2f5677693eab8968208e0
310+
0a70590d03ce743fbc43b2acdb0ac9bd508c8fb.
311+
- **Read:** the domain-coverage attribution is now negative in BOTH
312+
directions at matched epochs. Swapping news out (E6, 3ep): 5.8057.
313+
Adding classical units on top (G2b, 6ep): 4.8231 — flat vs the 3ep
314+
30k control (4.8218) and 0.25pp WORSE than news-only G2a (4.5701):
315+
the Tashkeela add neutralized the epoch gain entirely. The
316+
post-E3 residual is not register-shaped; the only levers that have
317+
moved this rung are optimizer (−2.96pp), fresher teacher labels
318+
(−0.47pp), and epochs (−0.25pp). What remains is GKD (on-policy
319+
exposure, run-012 in flight) — after which the ladder is closed.
320+
300321
## GKD — on-policy distillation rung (REGISTERED/ARMED 2026-09-03)
301322

302323
- **Status:** REGISTERED, launch pending owner ordering (the last

‎docs/PUBLICATION-NOTES.md‎

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -77,11 +77,16 @@ confound (fresh aux head for the final 23% of steps); E6
7777
constant-budget register swap (2026-09-02): 5.8057 — replacing news
7878
units with classical Tashkeela at constant total HURT (−0.98pp vs
7979
control), the domain-shaped-residual hypothesis's causal test
80-
failing in the swap direction; the add direction (G2b) remains
81-
open.** The E5/E6 pair is the paper's data-vs-architecture exhibit:
80+
failing in the swap direction; G2b label-scale ADD (2026-09-04,
81+
verdict recorded per TODO.substantiate 04): 4.8231 — 48k total
82+
(+18k classical) at 6 epochs is FLAT vs the 3ep control and 0.25pp
83+
worse than news-only G2a (4.5701): the add neutralized the epoch
84+
gain, closing the domain hypothesis negative in BOTH directions.**
85+
The E5/E6/G2b set is the paper's data-vs-architecture exhibit:
8286
two levers from the frontier-LLM literature (MTP, register
83-
diversification at constant budget) both regressed on byte-level
84-
student distillation — the levers that moved the rung were optimizer
87+
diversification — swap or add) plus the label-scale hypothesis all
88+
regressed or flat on byte-level student distillation — the levers
89+
that moved the rung were optimizer
8590
(E3), fresher teacher labels (E4), and epochs (G2a).
8691

8792
### 9. Muon optimizer A/B on the memory student (E3) — LANDED 2026-08-28
@@ -226,14 +231,19 @@ The decomposition for paper B, every line full-set with brackets:
226231
| + Muon (E3) | 5.2945 | — |
227232
| + r7 teacher | 4.8218 | — |
228233
| + 6 epochs (G2a) | 4.5701 | delta 2.12 [1.91, 2.35] |
234+
| register swap (E6, 3ep) | 5.8057 | negative |
235+
| register add (G2b, 6ep) | 4.8231 | delta 2.37 [2.19, 2.55] |
236+
| MTP-aux (E5, 3ep) | 5.0853 | negative (confound disclosed) |
229237
| depth halved (lite, 6ep) | 5.784 | delta 3.25 [3.03, 3.49] |
230238

231239
Paper-B framing: levers compose roughly additively (optimizer >>
232-
teacher > epochs), and the E6 causal test came back NEGATIVE —
240+
teacher > epochs), and BOTH register tests came back NEGATIVE —
233241
swapping 8k news units for classical-register Tashkeela at constant
234-
30k total scored 5.8057 (−0.98pp vs control), so the residual is
235-
not fixed by register mix at constant budget; the add direction
236-
(G2b, 48k total) is the live test, and capacity appears ONLY as
242+
30k total scored 5.8057 (−0.98pp vs control), and adding 18k
243+
classical units on top at 6 epochs scored 4.8231 (flat vs the 3ep
244+
control, 0.25pp worse than news-only G2a — the add neutralized the
245+
epoch gain): the residual is not register-shaped in either
246+
direction, and capacity appears ONLY as
237247
depth — width is load-bearing (both stitch ratios collapsed) while
238248
depth trades 1.21pp for 37% of the artifact. The subset-overstatement
239249
phenomenon (five instances, up to 3.2x inflation) is the

0 commit comments

Comments
 (0)