|
| 1 | += Morphological Supervision, Not Phonemic: Controlled Evidence and a Dual-Surface Teacher for Arabic Diacritization |
| 2 | +Ronald Tse |
| 3 | +v0.1 draft scaffold, 2026-08-31 |
| 4 | +:doctype: article |
| 5 | +:toc: macro |
| 6 | +:numbered: |
| 7 | + |
| 8 | +toc::[] |
| 9 | + |
| 10 | +== Abstract (draft) |
| 11 | + |
| 12 | +Arabic diacritization (haraqat restoration) is commonly assumed to |
| 13 | +benefit from phonemic supervision. Under a single-variable controlled |
| 14 | +ablation — identical teacher architecture, corpus, seed, and |
| 15 | +initialization, varying only the auxiliary stream's output |
| 16 | +representation — lexical-morphological supervision strictly dominates: |
| 17 | +morphology 2.5793 < phonemic IPA 2.6588 < none 2.6775 windowed DER-CE |
| 18 | +on the full 1,200-paragraph SadeedDiac-25 protocol. The IPA projection |
| 19 | +is itself learnable (2.3% CER probe), ruling out representation |
| 20 | +failure as the explanation: the strong form of "phonemes help |
| 21 | +diacritization" fails. A news-domain-adapted teacher (r7) then removes |
| 22 | +the lineage's in-domain/out-of-domain trade-off entirely — ID 2.2864 |
| 23 | +and OOD WikiNews-2024 17.38/11.83 multi-reference — making it, to our |
| 24 | +knowledge, the best dedicated model measured under this protocol. |
| 25 | + |
| 26 | +== 1. Introduction |
| 27 | +- authority-backed transliteration context; the phonological layer |
| 28 | +- the aux-supervision question and why it matters for low-resource tiers |
| 29 | +- contributions: (1) controlled ablation, (2) dual-surface teacher, |
| 30 | + (3) protocol-matched leaderboard, (4) full-set evaluation discipline |
| 31 | + |
| 32 | +== 2. Related work |
| 33 | +[Sadeed-1.5B; LLM baselines; aux-supervision literature] |
| 34 | + |
| 35 | +== 3. Protocol |
| 36 | +- windowed DER-CE (1,400-byte windows, word-boundary split, greedy, |
| 37 | + haraqat-projected, zero-skip), Misraj evaluator |
| 38 | +- full-set-only rule; the subset-overstatement phenomenon (3 quantified |
| 39 | + instances, 2-4x inflation — cross-ref paper B for the student tier) |
| 40 | + |
| 41 | +== 4. The controlled ablation (r8) |
| 42 | +| Arm | full-set DER-CE | |
| 43 | +| Morphological aux | 2.5793 | |
| 44 | +| Phonemic (IPA) aux | 2.6588 | |
| 45 | +| No aux | 2.6775 | |
| 46 | +[paired-bootstrap CIs — slot: task from PR #9 tooling] |
| 47 | +[IPA probe control: 2.3% CER] |
| 48 | + |
| 49 | +== 5. Dual-surface adaptation (r7) |
| 50 | +| Model | ID (SadeedDiac-25) | OOD (WikiNews-2024) | |
| 51 | +| r6 | 2.5793 | 19.82/12.46 | |
| 52 | +| r7 | 2.2864 | 17.38/11.83 | |
| 53 | + |
| 54 | +== 6. Leaderboard (protocol-matched) |
| 55 | +| Model | windowed DER-CE zero-skip | |
| 56 | +| Claude-3.7-Sonnet (published) | 1.3941 | |
| 57 | +| r7 teacher (ours, 580M dedicated) | 2.2864 | |
| 58 | +| GLM-5.2 zero-skip | 2.6911 | |
| 59 | +| [GLM-5.3-Flash, effort=low — RUN IN FLIGHT] | — | |
| 60 | +| Gemini-Flash-2.0 | 3.1926 | |
| 61 | +| GPT-4 | 3.8645 | |
| 62 | +| Sadeed-1.5B | 7.2915 | |
| 63 | + |
| 64 | +== 7. Limitations |
| 65 | +- single-seed arms; item-level bootstrap; multi-seed on the headline |
| 66 | + pair if compute permits |
| 67 | +- benchmark scope (SadeedDiac-25; WikiNews-2024 multi-ref) |
| 68 | + |
| 69 | +== 8. Artifacts |
| 70 | +r7/r8 checkpoints; evaluator harness; full per-run provenance |
| 71 | +(cross-ref paper B artifact appendix). |
| 72 | + |
| 73 | +== Allocation note |
| 74 | +This file is Paper A per the 2026-08-29 two-paper decision |
| 75 | +(PUBLICATION-NOTES.md): A = teacher (this paper); B = student |
| 76 | +(paper.adoc, restructured around the 2x2 factorial + frontier); |
| 77 | +C = systems (benchmarks/imf-runtime, gated on E3). |
0 commit comments