Skip to content

Commit 5745d80

Browse files
Ronald Tseronaldtse
authored andcommitted
docs: E6 verdict — gate FAILED at 5.8057; the register-swap causal test is negative
Constant-budget diversification (8k news units swapped for classical Tashkeela, 30k total, identical schedule) scored 5.8057 vs the 4.8218 control and gate 4.5218 — worse than E5. Teacher reproduced at 2.289 (n=1200). PUBLICATION-NOTES: E5+E6 recorded as the data-vs- architecture negative pair; Paper-B paragraph updated (residual is not fixed by register mix at constant budget; G2b add is the live test).
1 parent c8ad4ee commit 5745d80

2 files changed

Lines changed: 32 additions & 3 deletions

File tree

‎docs/EXPERIMENTS.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -271,6 +271,20 @@ All rows passed the CER parity gate at release. Readings:
271271
- **Prediction (registered):** 4.45-4.75 — a register-diversification
272272
effect at constant budget should be real but second-order; if the
273273
domain attribution is right, this should beat E5's MTP-aux.
274+
- **Status:** COMPLETE 2026-09-02 — **GATE FAILED: 5.8057** full-set
275+
windowed DER (gate <=4.5218; control 4.8218; E5's failed 5.0853).
276+
NOT ADOPTED. Teacher reproduces at 2.289 (n=1200, protocol
277+
consistent). Paired bootstrap student-teacher: +3.2388pp, CI
278+
[2.939, 3.575]. The prediction (4.45-4.75) missed badly. Read:
279+
swapping news-domain units OUT for classical-register units at
280+
constant budget HURT (−0.98pp vs control) — the news-domain
281+
teacher labels match the benchmark surface better than classical
282+
units do, and 30k total was already enough budget that register
283+
mix was not the binding constraint. Together with G2a (4.5701 at
284+
6 epochs) the ladder now reads: longer training and MORE total
285+
units move the rung; register swaps at constant budget do not.
286+
Clean negative; direction shifts to G2b (48k add). Labels sha256
287+
39b4213b5ab4c9f7273da90a9a1954f7701320b62870a954e6e2c02e810617a4.
274288

275289
## Parked
276290

‎docs/PUBLICATION-NOTES.md‎

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,19 @@ Paper: frontier section (fourth question). RESULTS.md run-003-pkm.
7070
RL teacher polishing flat/negative ×3; microkimi bridges improve
7171
structure but not accuracy; teacher beam-search unnecessary for
7272
Arabic; per-channel int8 rejected on measurement; the 30 MiB tier
73-
closed as infeasible without pretraining.
73+
closed as infeasible without pretraining; **E5 MTP-aux (2026-09-01):
74+
5.0853 vs the 4.8218 control — multi-token-prediction as a training
75+
auxiliary HURT at this scale (+0.26pp), with a disclosed preemption
76+
confound (fresh aux head for the final 23% of steps); E6
77+
constant-budget register swap (2026-09-02): 5.8057 — replacing news
78+
units with classical Tashkeela at constant total HURT (−0.98pp vs
79+
control), the domain-shaped-residual hypothesis's causal test
80+
failing in the swap direction; the add direction (G2b) remains
81+
open.** The E5/E6 pair is the paper's data-vs-architecture exhibit:
82+
two levers from the frontier-LLM literature (MTP, register
83+
diversification at constant budget) both regressed on byte-level
84+
student distillation — the levers that moved the rung were optimizer
85+
(E3), fresher teacher labels (E4), and epochs (G2a).
7486

7587
### 9. Muon optimizer A/B on the memory student (E3) — LANDED 2026-08-28
7688
**4.8287 vs 7.5553 full-set (−2.727pp from the optimizer alone); adopt
@@ -217,8 +229,11 @@ The decomposition for paper B, every line full-set with brackets:
217229
| depth halved (lite, 6ep) | 5.784 | delta 3.25 [3.03, 3.49] |
218230

219231
Paper-B framing: levers compose roughly additively (optimizer >>
220-
teacher > epochs), the remaining ~2.0-2.2pp is domain-shaped (the
221-
label-scale rung is the causal test), and capacity appears ONLY as
232+
teacher > epochs), and the E6 causal test came back NEGATIVE —
233+
swapping 8k news units for classical-register Tashkeela at constant
234+
30k total scored 5.8057 (−0.98pp vs control), so the residual is
235+
not fixed by register mix at constant budget; the add direction
236+
(G2b, 48k total) is the live test, and capacity appears ONLY as
222237
depth — width is load-bearing (both stitch ratios collapsed) while
223238
depth trades 1.21pp for 37% of the artifact. The subset-overstatement
224239
phenomenon (five instances, up to 3.2x inflation) is the

0 commit comments

Comments
 (0)