Skip to content

Commit 0925d8f

Browse files
author
Ronald Tse
committed
docs: paper A scaffold — controlled ablation + dual-surface teacher
Filled tables from RESULTS.md; CI slots for the paired bootstrap; GLM-5.3 leaderboard row pending the in-flight eval; allocation note tying the two-paper decision together.
1 parent 4311a84 commit 0925d8f

1 file changed

Lines changed: 77 additions & 0 deletions

File tree

‎docs/paper-a.adoc‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
= Morphological Supervision, Not Phonemic: Controlled Evidence and a Dual-Surface Teacher for Arabic Diacritization
2+
Ronald Tse
3+
v0.1 draft scaffold, 2026-08-31
4+
:doctype: article
5+
:toc: macro
6+
:numbered:
7+
8+
toc::[]
9+
10+
== Abstract (draft)
11+
12+
Arabic diacritization (haraqat restoration) is commonly assumed to
13+
benefit from phonemic supervision. Under a single-variable controlled
14+
ablation — identical teacher architecture, corpus, seed, and
15+
initialization, varying only the auxiliary stream's output
16+
representation — lexical-morphological supervision strictly dominates:
17+
morphology 2.5793 < phonemic IPA 2.6588 < none 2.6775 windowed DER-CE
18+
on the full 1,200-paragraph SadeedDiac-25 protocol. The IPA projection
19+
is itself learnable (2.3% CER probe), ruling out representation
20+
failure as the explanation: the strong form of "phonemes help
21+
diacritization" fails. A news-domain-adapted teacher (r7) then removes
22+
the lineage's in-domain/out-of-domain trade-off entirely — ID 2.2864
23+
and OOD WikiNews-2024 17.38/11.83 multi-reference — making it, to our
24+
knowledge, the best dedicated model measured under this protocol.
25+
26+
== 1. Introduction
27+
- authority-backed transliteration context; the phonological layer
28+
- the aux-supervision question and why it matters for low-resource tiers
29+
- contributions: (1) controlled ablation, (2) dual-surface teacher,
30+
(3) protocol-matched leaderboard, (4) full-set evaluation discipline
31+
32+
== 2. Related work
33+
[Sadeed-1.5B; LLM baselines; aux-supervision literature]
34+
35+
== 3. Protocol
36+
- windowed DER-CE (1,400-byte windows, word-boundary split, greedy,
37+
haraqat-projected, zero-skip), Misraj evaluator
38+
- full-set-only rule; the subset-overstatement phenomenon (3 quantified
39+
instances, 2-4x inflation — cross-ref paper B for the student tier)
40+
41+
== 4. The controlled ablation (r8)
42+
| Arm | full-set DER-CE |
43+
| Morphological aux | 2.5793 |
44+
| Phonemic (IPA) aux | 2.6588 |
45+
| No aux | 2.6775 |
46+
[paired-bootstrap CIs — slot: task from PR #9 tooling]
47+
[IPA probe control: 2.3% CER]
48+
49+
== 5. Dual-surface adaptation (r7)
50+
| Model | ID (SadeedDiac-25) | OOD (WikiNews-2024) |
51+
| r6 | 2.5793 | 19.82/12.46 |
52+
| r7 | 2.2864 | 17.38/11.83 |
53+
54+
== 6. Leaderboard (protocol-matched)
55+
| Model | windowed DER-CE zero-skip |
56+
| Claude-3.7-Sonnet (published) | 1.3941 |
57+
| r7 teacher (ours, 580M dedicated) | 2.2864 |
58+
| GLM-5.2 zero-skip | 2.6911 |
59+
| [GLM-5.3-Flash, effort=low — RUN IN FLIGHT] | — |
60+
| Gemini-Flash-2.0 | 3.1926 |
61+
| GPT-4 | 3.8645 |
62+
| Sadeed-1.5B | 7.2915 |
63+
64+
== 7. Limitations
65+
- single-seed arms; item-level bootstrap; multi-seed on the headline
66+
pair if compute permits
67+
- benchmark scope (SadeedDiac-25; WikiNews-2024 multi-ref)
68+
69+
== 8. Artifacts
70+
r7/r8 checkpoints; evaluator harness; full per-run provenance
71+
(cross-ref paper B artifact appendix).
72+
73+
== Allocation note
74+
This file is Paper A per the 2026-08-29 two-paper decision
75+
(PUBLICATION-NOTES.md): A = teacher (this paper); B = student
76+
(paper.adoc, restructured around the 2x2 factorial + frontier);
77+
C = systems (benchmarks/imf-runtime, gated on E3).

0 commit comments

Comments
 (0)