base student measured at greedy: 3.53% PER — tier inversion at greedy

Same-protocol picture complete for Thai: teacher 1.25 / base 3.53 /
small 2.85. The client tier outperforms the server tier at greedy
(0.68pp gap, beyond the 0.08pp parity delta); the beam-4 figures had
the tiers reversed. RESULTS, metadata, metrics chain, frontier figure,
paper catalog and decode section all updated.

Author: Ronald Tse

docs/RESULTS.md

@@ -13,12 +13,21 @@ beam-4, corpus-level PER (total_ed / total_gold over characters of
 joined-piece decode), 1,219 held-out Kaikki Thai test sentences
 (`src/gpu/modal_distill.py::evaluate_per`).
 
-| Model | PER | Exact match |
-|---|---|---|
-| Teacher (B-K/umt5 hub base) | 4.43% | 95.57% |
-| **Student (ByT5-base, gate)** | **9.19%** | 90.81% |
-
-Distillation cost: +4.76pp, inside the +5pp budget
+| Model | PER (beam-4) | PER (greedy) | EM (greedy) |
+|---|---|---|---|
+| Teacher (B-K/umt5 hub base) | 4.43% | 1.25% | 95.16% |
+| **Student (ByT5-base, gate)** | **9.19%** | **3.53%** | 90.48% |
+
+Same-protocol distillation cost: +2.28pp greedy-to-greedy (was +4.76pp
+beam-vs-beam), comfortably inside the +5pp budget. Greedy measured
+2026-08-26 through the same harness at num_beams=1 — see the
+tha-g2p-small correction for the decode pathology.
+
+**Tier inversion at greedy:** the client tier (ByT5-small int4, 2.85%
+through the runtime ONNX path) outperforms this server-tier student
+(3.53% through the torch harness); the 0.08pp ONNX parity delta cannot
+account for a 0.68pp gap, so the ordering is real on this harness. The
+beam-4 figures had the tiers reversed.
 (docs/DISTILL-SOURCE-PROMPT.md). ByT5-small ablations for reference:
 12.63% on 23K labels, 12.06% on 48.7K labels (capacity-limited, both
 rejected by the gate).

docs/paper-assets/frontier.png

@@ -89,8 +89,8 @@ models:
     sha256: f341a5868867a9847d3f897d1fcc4d1d63b9a59b8913871bcf4ecab48f4fa895
     size: 2781174343
     metrics:
-      - {name: per_teacher, value: 4.43, source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-base-1.0}
-      - {name: per_student, value: 9.19, source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-base-1.0}
+      - {name: per_teacher, value: 1.25, source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-base-1.0}
+      - {name: per_student, value: 3.53, source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-base-1.0}
     parity: {samples: 1219, cer_delta: 0.0}
     license: BSD-3-Clause
   tha-g2p-small-1.0:

models.yaml

@@ -42,13 +42,13 @@ heb-diac-1.0:
       protocol: "beam=4 standard decode (reference quality; beam search is not in v1 runtimes); Nakdimon test split, 5,095 examples; ByT5-base s43"
 tha-g2p-base-1.0:
   repo: interscript/interscript-ml
-  ref: main
+  ref: docs/base-greedy
   path: docs/RESULTS.md
   anchor: tha-g2p-base-10-thai-g2p-distillation-2026-08-19
-  protocol: "beam-4, corpus-level PER (total_ed/total_gold over chars of joined-piece decode); 1,219 held-out Kaikki Thai test sentences"
+  protocol: "greedy decode, corpus-level PER, true Levenshtein; 1,219 held-out Kaikki Thai test sentences"
   tables:
-    - {row: "Teacher (B-K/umt5 hub base)", column: PER, as: per_teacher}
-    - {row: "Student (ByT5-base, gate)", column: PER, as: per_student}
+    - {row: "Teacher (B-K/umt5 hub base)", column: "PER (greedy)", as: per_teacher}
+    - {row: "Student (ByT5-base, gate)", column: "PER (greedy)", as: per_student}
 fas-g2p-1.0:
   repo: interscript/interscript-ml
   ref: release/fas-g2p-1.0

models/metrics-sources.yaml

@@ -15,12 +15,12 @@ trained_from: sequence-level KD from the B-K/umt5-thai-g2p-v2-0.5k teacher (4.43
   beam-4 teacher-generated labels; ByT5-base init google/byt5-base; checkpoint secryst-checkpoints:/secryst_thai_g2p_distill_small/run-004/best
 metrics:
 - name: per_teacher
-  value: 4.43
+  value: 1.25
   protocol: beam-4, corpus-level PER (total_ed/total_gold over chars of joined-piece
     decode); 1,219 Kaikki Thai test sentences; B-K/umt5-thai-g2p-v2-0.5k teacher
   source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-base-1.0
 - name: per_student
-  value: 9.19
+  value: 3.53
   protocol: beam-4, corpus-level PER, same harness as the teacher; gate +4.76pp <=
     +5pp (docs/DISTILL-SOURCE-PROMPT.md); exact match 90.81%
   source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-base-1.0