Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion TODO.sota/00-README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,8 @@ hf jobs run --flavor a100-large -d \
| 25 | 25-ar-news-specialist.md | P1 | 17 | VERDICT: OOD 10.13/8.98 (crown), ID 5.50; export staged |
| 27 | 27-ar-large-id-arm.md | P1 | — | NEGATIVE — 2.5765 parity with r7; capacity closed on AR too |
| 28 | 28-oracle-complementarity.md | P2 | — | CLOSED — ceiling 96.78 real; naive voting catastrophic |
| 30 | 30-heb-learned-ipa-v0.md | P1 | 09 | RUNNING (run-032, FLEURS audio-supervised student) |
| 30 | 30-heb-learned-ipa-v0.md | P1 | 09 | NEGATIVE — CER 1.18; ASR teacher is the bottleneck |
| 31 | 31-heb-phoneme-asr-v1.md | P1 | 30 | SPEC'D — tune the phoneme CTC on Hebrew first (owner block) |
| 26 | 26-heb-noisy-student.md | P1 | 04 | NEGATIVE — 8.72 vs 8.18; all three text levers closed |

## Schedule
Expand Down
17 changes: 17 additions & 0 deletions TODO.sota/31-heb-phoneme-asr-v1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# WO31 — Hebrew-tuned phoneme ASR (the v0 lesson, spec'd)

WO30 v0 proved the pipeline but the teacher is the bottleneck: the
universal espeak-cv-ft CTC phonemizes Hebrew poorly. The fix (the
ReNikud recipe's first stage):

1. Fine-tune wav2vec2-lv-60-espeak-cv-ft on FLEURS he_il (~10h,
~3.6K train utterances) with espeak-ng reference phonemizations of
the TRANSCRIPTS as targets (self-supervised bootstrap: espeak reads
the text; the CTC learns to hear it in real audio).
2. Re-run the stage-2 alignment probe with the tuned ASR (expect PER
well under 0.395's already-passing mark).
3. Relabel FLEURS + retrain the v1 student; gate unchanged (<0.2393).
4. Only then does ivrit.ai scale ($500) make sense.

Cost: one fine-tune (~2-4 GPU-hours) + relabel + v1 train. Owner
green-light for the block.
17 changes: 17 additions & 0 deletions docs/RESULTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -1380,3 +1380,20 @@ three closed levers: crown **8.18** (data scale ✗ at 50K→90K, capacity
✗ byt5-large 8.48, noisy-student ✗ 8.72). The remaining Hebrew lever
is the AUDIO campaign — WO30 v0 student RUNNING now on the GO-grade
supervision.

## WO30 verdict — Hebrew learned-IPA v0 FAILED: CER 1.18; the weak link is the ASR teacher (2026-10-09)

run-032 v0 (FLEURS 4,359 audio-labeled pairs → byt5-small): training
converged on its labels (loss 0.11) but scores **CER 1.18** on the
phonikud board (gate <0.2393). Diagnosis, recorded before any scale
spend: (1) format divergence — the student faithfully reproduces the
ASR's space-separated phone style vs the board's compact stressed
strings; (2) the deeper cause — the universal phoneme CTC
(wav2vec2-espeak-cv-ft) is a weak Hebrew teacher (stage-2 PER 0.395
was alignment-viability, not label quality; its FLEURS outputs showed
non-Hebrew phonemizations). ReNikud's pipeline builds a Hebrew-capable
phoneme recognizer FIRST. The $500 ivrit.ai scale decision is therefore
NOT triggered — scaling a weak teacher wastes it. The audio path
remains open but requires: fine-tune the phoneme CTC on Hebrew audio
(FLEURS he ~10h is exactly that training set), then relabel, then
v1. Spec'd as WO31; needs owner green-light for the next compute block.
Loading