Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions TODO.sota/00-README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ hf jobs run --flavor a100-large -d \
| 28 | 28-oracle-complementarity.md | P2 | — | CLOSED — ceiling 96.78 real; naive voting catastrophic |
| 30 | 30-heb-learned-ipa-v0.md | P1 | 09 | NEGATIVE — CER 1.18; ASR teacher is the bottleneck |
| 31 | 31-heb-phoneme-asr-v1.md | P1 | 30 | SPEC'D — tune the phoneme CTC on Hebrew first (owner block) |
| 32 | 32-matched-architecture-arms.md | P1 | 17 | RUNNING — run-033 plane-large + run-034 BiLSTM on their corpus |
| 26 | 26-heb-noisy-student.md | P1 | 04 | NEGATIVE — 8.72 vs 8.18; all three text levers closed |

## Schedule
Expand Down
16 changes: 16 additions & 0 deletions TODO.sota/32-matched-architecture-arms.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# WO32 — the matched-architecture program (same corpus MUST win)

Audit: no contamination either side; seq2seq converged at 10.03 → gap
is architecture + teacher. Arms:

- run-033: byt5-large plane + full 900K silver (news-pure, K=3) —
their recipe class, stronger backbone. Gate: WN-2024 multiref < 10
(stretch: single digits).
- run-034: literal BiLSTM tagger on the same corpus — reproduces their
class from scratch. Gate: WN-2024 < 15 validates the class at our
scale; comparison anchor.
- SOTA-stack layer above both: K-pass conditioning, byte-pretrained
backbones, preserve mode (already in the plane family).

Ship rule: whichever arm dominates WikiNews-2024 becomes the
ara-diac-news successor; seq2seq line is closed (converged at 10.03).
22 changes: 22 additions & 0 deletions docs/RESULTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -1397,3 +1397,25 @@ NOT triggered — scaling a weak teacher wastes it. The audio path
remains open but requires: fine-tune the phoneme CTC on Hebrew audio
(FLEURS he ~10h is exactly that training set), then relabel, then
v1. Spec'd as WO31; needs owner green-light for the next compute block.

## The "same corpus must win" program — convergence CLOSED, both matched arms in flight (2026-10-09)

**Contamination audit: CLEAN** — 0/356 WikiNews-2024 test sentences
share even one 8-gram with the QCRI 5M-word corpus; their 2.70 is
honest generalization, ours is too. **run-029c (+2 epochs, seq2seq
specialist): 10.03/8.95** (from 10.13) — the seq2seq line has
CONVERGED; doubling training bought 0.10 WER. The remaining 7.3-point
gap is therefore ARCHITECTURE (tagging vs generation) + teacher quality,
and two arms now attack it directly on their exact recipe:

- **run-033** (byt5-large PLANE = our tagging family + full 900K-unit
corpus, news-pure, 3ep, K=3, gradient checkpointing) — their class
with a far stronger backbone.
- **run-034** (literal BiLSTM tagger, Fadel/QCRI-style: char-emb →
3×BiLSTM-512 → per-position haraqat head, from scratch, same corpus)
— their architecture, rebuilt and run at our data scale.

Fixes en route (train#128-#131): haraqat split_planes mark-only crash
(QCRI silver has pure-combining-mark windows — latent library bug, now
tested); byt5-large OOM → PLANE_CHECKPOINT gradient checkpointing;
LSTM OOM → BIL_BS; padded-label loss smearing → ignore_index=-100.
Loading