Skip to content

Commit 8911507

Browse files
author
Ronald Tse
committed
docs: run-003-pkm recorded in RESULTS.md; final gate engagement in E2
7.5553 full-set vs run-002's 8.2590 (0.704pp closed, 12.4% relative, below the pre-registered bar). Gates settled at 0.034-0.053.
1 parent 382b8c8 commit 8911507

2 files changed

Lines changed: 22 additions & 1 deletion

File tree

‎docs/EXPERIMENTS.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,9 @@ All rows passed the CER parity gate at release. Readings:
8585
coverage (the subset lesson from run-002). Reported exactly as
8686
measured — no threshold-moving.
8787
- Engagement was verified independent of outcome: gates moved off zero
88-
by step-500 (0.0008-0.0028), CE 2.07 → 0.02 over 10,995 steps.
88+
by step-500 (0.0008-0.0028) and settled at 0.034-0.053 by step-10,500
89+
(15-20x) — the memory branch contributed measurably but modestly,
90+
matching the 0.70pp outcome. CE 2.07 → 0.02 over 10,995 steps.
8991
- Launched 2026-08-27, A10G, labels reused from run-002 (single-variable
9092
design). Survived one mid-run eviction: resumed from step-2000.
9193
- **Hypothesis:** the 5.68pp teacher→student gap (r6 2.5815 → ByT5-small

‎docs/RESULTS.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -222,6 +222,25 @@ mojibake labels (byt5 decode_joined bug), the second silently resumed
222222
from the poisoned lineage's checkpoints (now guarded by labels.sha
223223
digest matching), this one clean end-to-end. CE plateaued at ~0.016.
224224

225+
### run-003-pkm — memory-layer student (2026-08-28, research run)
226+
227+
The qwen-next capacity experiment (EXPERIMENTS.md E2): identical to
228+
run-002 except three product-key memory layers (+85.9M lookup params,
229+
zero-init gates) on the ByT5-small decoder — single-variable.
230+
231+
| Model | DER-CE (full 1,200) |
232+
|---|---|
233+
| Teacher (r6, full-set) | 2.5815% |
234+
| **Student + PKM memory (run-003-pkm)** | **7.5553%** |
235+
| Student vanilla (run-002) | 8.2590% |
236+
237+
0.704pp of the 5.677pp teacher-student gap closed (12.4% relative) at
238+
near-zero added compute — below the pre-registered ≥1.0pp win bar, so
239+
the memory axis is real but not the dominant term of the gap. Gates
240+
verified engaged (0.034-0.053 at completion). Not shipped; the vanilla
241+
client rung stands. The optimizer A/B on the same architecture
242+
(run-004-pkm-muon) is recorded in EXPERIMENTS.md E3.
243+
225244
Leaderboard context (SadeedDiac-25, Misraj evaluator, zero-skip,
226245
harakat-projected DER-CE): the teacher tier (r6, 580M) at 2.5793
227246
(reproduced at 2.5815, 2026-08-26) is the best dedicated model measured

0 commit comments

Comments
 (0)