Skip to content

Commit 165cecf

Browse files
ronaldtseRonald Tse
andauthored
docs: matched-architecture program — audit clean, seq2seq converged, both arms in flight (#290)
Co-authored-by: Ronald Tse <ronald.tse@gmail.com>
1 parent 9e9ef54 commit 165cecf

3 files changed

Lines changed: 39 additions & 0 deletions

File tree

‎TODO.sota/00-README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,7 @@ hf jobs run --flavor a100-large -d \
6767
| 28 | 28-oracle-complementarity.md | P2 | — | CLOSED — ceiling 96.78 real; naive voting catastrophic |
6868
| 30 | 30-heb-learned-ipa-v0.md | P1 | 09 | NEGATIVE — CER 1.18; ASR teacher is the bottleneck |
6969
| 31 | 31-heb-phoneme-asr-v1.md | P1 | 30 | SPEC'D — tune the phoneme CTC on Hebrew first (owner block) |
70+
| 32 | 32-matched-architecture-arms.md | P1 | 17 | RUNNING — run-033 plane-large + run-034 BiLSTM on their corpus |
7071
| 26 | 26-heb-noisy-student.md | P1 | 04 | NEGATIVE — 8.72 vs 8.18; all three text levers closed |
7172

7273
## Schedule
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# WO32 — the matched-architecture program (same corpus MUST win)
2+
3+
Audit: no contamination either side; seq2seq converged at 10.03 → gap
4+
is architecture + teacher. Arms:
5+
6+
- run-033: byt5-large plane + full 900K silver (news-pure, K=3) —
7+
their recipe class, stronger backbone. Gate: WN-2024 multiref < 10
8+
(stretch: single digits).
9+
- run-034: literal BiLSTM tagger on the same corpus — reproduces their
10+
class from scratch. Gate: WN-2024 < 15 validates the class at our
11+
scale; comparison anchor.
12+
- SOTA-stack layer above both: K-pass conditioning, byte-pretrained
13+
backbones, preserve mode (already in the plane family).
14+
15+
Ship rule: whichever arm dominates WikiNews-2024 becomes the
16+
ara-diac-news successor; seq2seq line is closed (converged at 10.03).

‎docs/RESULTS.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1397,3 +1397,25 @@ NOT triggered — scaling a weak teacher wastes it. The audio path
13971397
remains open but requires: fine-tune the phoneme CTC on Hebrew audio
13981398
(FLEURS he ~10h is exactly that training set), then relabel, then
13991399
v1. Spec'd as WO31; needs owner green-light for the next compute block.
1400+
1401+
## The "same corpus must win" program — convergence CLOSED, both matched arms in flight (2026-10-09)
1402+
1403+
**Contamination audit: CLEAN** — 0/356 WikiNews-2024 test sentences
1404+
share even one 8-gram with the QCRI 5M-word corpus; their 2.70 is
1405+
honest generalization, ours is too. **run-029c (+2 epochs, seq2seq
1406+
specialist): 10.03/8.95** (from 10.13) — the seq2seq line has
1407+
CONVERGED; doubling training bought 0.10 WER. The remaining 7.3-point
1408+
gap is therefore ARCHITECTURE (tagging vs generation) + teacher quality,
1409+
and two arms now attack it directly on their exact recipe:
1410+
1411+
- **run-033** (byt5-large PLANE = our tagging family + full 900K-unit
1412+
corpus, news-pure, 3ep, K=3, gradient checkpointing) — their class
1413+
with a far stronger backbone.
1414+
- **run-034** (literal BiLSTM tagger, Fadel/QCRI-style: char-emb →
1415+
3×BiLSTM-512 → per-position haraqat head, from scratch, same corpus)
1416+
— their architecture, rebuilt and run at our data scale.
1417+
1418+
Fixes en route (train#128-#131): haraqat split_planes mark-only crash
1419+
(QCRI silver has pure-combining-mark windows — latent library bug, now
1420+
tested); byt5-large OOM → PLANE_CHECKPOINT gradient checkpointing;
1421+
LSTM OOM → BIL_BS; padded-label loss smearing → ignore_index=-100.

0 commit comments

Comments
 (0)