Skip to content

Commit bd3bc03

Browse files
author
Ronald Tse
committed
docs(EXPERIMENTS): GKD rung registered/armed — gate + prediction before any launch
The last untested reading of the domain attribution; spec design, gate (4.5218), and prediction (4.30-4.65) pre-registered. Launch is the owner call. Also records the run-006 re-eval discrepancy (5.0821 vs canonical 4.8218) so the next verdict compares within one eval pass.
1 parent 497b83c commit bd3bc03

1 file changed

Lines changed: 25 additions & 0 deletions

File tree

‎docs/EXPERIMENTS.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -297,6 +297,31 @@ All rows passed the CER parity gate at release. Readings:
297297
shows a convergence-speed effect distinguishable from noise on the
298298
surviving curves; the DER gates remain the only verdicts.
299299

300+
## GKD — on-policy distillation rung (REGISTERED/ARMED 2026-09-03)
301+
302+
- **Status:** REGISTERED, launch pending owner ordering (the last
303+
lever with no data; TODO.substantiate/03). Implementation follows
304+
this registration verbatim when launched.
305+
- **Hypothesis:** the client tier's remaining ~2.0pp over the teacher
306+
is domain-shaped exposure: the student never trains on its own
307+
decode distribution. On-policy GKD (sequences sampled from the
308+
student during training, scored against the frozen r7 teacher's
309+
targets/logits) attacks exactly that.
310+
- **Design:** spec `ara-diac-small-2-gkd`, control = run-006 verbatim
311+
(30k units, identical steps/schedule/optimizer); delta = GKD loss
312+
mixing — student-sampled sequences (temperature-matched to greedy
313+
inference) blended into the sequence-KD objective at a fixed ratio,
314+
annealed to zero over the final third.
315+
- **Pre-agreed gate:** adopt at <= 4.5218 full-set windowed DER;
316+
honest-report band [4.5218, 4.8218); investigate if worse.
317+
- **Prediction (registered):** 4.30-4.65 — the E6 swap-negative says
318+
register mix at constant budget is not the lever; on-policy
319+
exposure is the remaining untested reading of the domain
320+
attribution. Note: run-006's canonical control is 4.8218 (E4-era
321+
eval); a later auto-chain re-eval printed 5.0821 for the same run —
322+
the ledger's 4.8218 stays canonical; a GKD verdict compares within
323+
one eval pass.
324+
300325
## Parked
301326

302327
- **Speculative decoding** (LongCat converts sparsity→speed): revisit

0 commit comments

Comments
 (0)