You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 7647348
Browse filesBrowse the repository at this point in the historyBrowse files
@@ -180,7 +180,7 @@ All fp32 artifacts measured 0.0pp parity (torch reference vs ONNX, ≥500 sample
180
180
181
181
=== Capacity costs are real and are disclosed
182
182
183
-
The Hebrew student met its pre-registered budget (+5.58pp against a ≈5.6pp allowance, the widened budget reflecting the teacher's own headroom). The Arabic client student missed its strict +0.5pp budget by +2.34pp and ships with that number in its metadata, its model card, and the results log. The measured pattern across languages: ByT5-small → ByT5-base shrink costs are single-digit percentage points on diacritization but larger on G2P (Thai +7.63pp beam-4 / teacher-relative 2.85% greedy). Users choosing a tier choose a measured trade, not a mystery box.
183
+
The Hebrew student met its pre-registered budget (+5.58pp against a ≈5.6pp allowance, the widened budget reflecting the teacher's own headroom). The Arabic client student missed its strict +0.5pp budget by +2.34pp and ships with that number in its metadata, its model card, and the results log. The measured pattern across languages: ByT5-small → ByT5-base shrink costs are single-digit percentage points on diacritization, and — measured same-protocol at greedy — +1.60pp (client tier) and +2.28pp (server tier) on Thai G2P. Users choosing a tier choose a measured trade, not a mystery box.
184
184
185
185
=== The Persian teacher is competitive with published state of the art
186
186
@@ -199,7 +199,7 @@ The campaign's central engineering question: how small can a client artifact be?
199
199
|same + linear bridges |random |33M |~30 MiB |71.12
@@ -209,7 +209,7 @@ The campaign's central engineering question: how small can a client artifact be?
209
209
Three findings:
210
210
211
211
1. **Random initialization collapses regardless of capacity or structure.** The bridge projections (teacher activations ridge-regressed into the student's geometry) measurably improve internal structure — 75.80 → 71.12 — but cannot rescue task accuracy. Enlarging without pretraining does not help (70M scores *worse* than 33M).
212
-
2. **The pretrained rung is the whole cliff.** From 70M/78.51 to 300M/12.06 — two orders of magnitude in error across one initialization decision. ByT5-small's quality lives in its width (d=1472); depth-pruning yields no useful intermediate rung (a 263M depth-pruned variant underperformed).
212
+
2. **The pretrained rung is the whole cliff.** From 70M/78.51 to 300M/2.85 — two orders of magnitude in error across one initialization decision. ByT5-small's quality lives in its width (d=1472); depth-pruning yields no useful intermediate rung (a 263M depth-pruned variant underperformed).
213
213
3. **The collapse reproduces exactly in diacritization, with clean labels.** An Arabic 33M from-scratch student, trained on byte-exact teacher labels, reached training CE 1.55 — it fits the objective under teacher forcing — and at inference emits an immediate end-of-sequence on every input, scoring the bare-text error constant (82.87% DER-CE vs teacher 1.32%). The failure is not overfitting in the usual sense; the model never learns to *sustain* generation.
214
214
215
215
The engineering consequence: the client tier ships at ByT5-small, quantized. A true 30–70 MiB tier requires a *pretrained* backbone at that scale — byte-level pretraining of a narrow model — which remains open future work. The quantization ladder is the practical size lever today: 246 MiB at int8, 193 MiB at int4, with the 4-bit quality cost measured at +0.17pp CER and certified by the parity gate.
@@ -233,6 +233,19 @@ Porting beam search into the runtimes forced an apples-to-apples measurement on
233
233
234
234
Exact match barely moves; edit distance collapses by 4.2x. The signature is diagnostic: beam decoding is mangling precisely the sentences the model did not get exactly right.
235
235
236
+
The *teacher* is also affected, though less severely: measured at
237
+
greedy on the same harness and split, the umt5 teacher scores 1.25% PER
238
+
(EM 95.16%) against its published beam-4 4.43%, and the server-tier
239
+
ByT5-base student 3.53% (EM 90.48%) against its published 9.19%. The
240
+
pathology even reorders tiers: at greedy the quantized client student
241
+
(2.85%, through the runtime ONNX path) outperforms the server-tier
242
+
student (3.53%, torch harness) — a 0.68pp gap that the 0.08pp ONNX
243
+
parity delta cannot account for. Same-protocol, the
244
+
client tier's true shrink cost is +1.60pp — not the +7.63pp the
245
+
beam-vs-beam comparison suggested. Both numbers in that comparison were
246
+
decode artifacts; the distillation itself was substantially better than
247
+
either published figure implied.
248
+
236
249
[[fig-decode]]
237
250
.(a) Top-1 log-probabilities at successive positions of a representative greedy decode against the uniform baseline; the argmax is barely distinguished from the field. (b) PER by decode on a 150-sentence subset: no normalization exponent recovers greedy quality.
238
251
[image::paper-assets/decode.png[Decode distributions and alpha sweep]]
0 commit comments