Skip to content

Commit cb5e49e

Browse files
author
Ronald Tse
committed
docs: TODO.sota 17-19 — register closure arms, parity runs, lexicon loader
1 parent f76e7bb commit cb5e49e

4 files changed

Lines changed: 37 additions & 0 deletions

File tree

‎TODO.sota/00-README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,9 @@ hf jobs run --flavor a100-large -d \
5454
| 14 | 14-ts-plane-port.md | P2 | — | DONE (ts#99/#100/#101, CI leg models#265) |
5555
| 15 | 15-api-edge.md | P2 | 14 | DONE (api#29: edge-first /v1/infer, kind in index) |
5656
| 16 | 16-parking-lot.md | — | — | parked (not now) |
57+
| 17 | 17-arabic-register-closure.md | P1 | 11 | r8a RUNNING; r8b launching |
58+
| 18 | 18-parity-runs.md | P1 | 10 | pending |
59+
| 19 | 19-lexicon-loader.md | P2 | 13 | pending |
5760

5861
## Schedule
5962

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# WO17 — Arabic register closure (the WikiNews gap attack)
2+
3+
**Why:** WO01 proved the 17.38-vs-2.70 WikiNews-2024 gap is register,
4+
not vocabulary (OOV 1.61%). QCRI's own README confirms their news model
5+
is trained on 5M words of Wikipedia SILVER labels from their BiLSTM —
6+
the same noisy-teacher technique as our r8. Two arms decide which
7+
teacher's silver closes our gap:
8+
9+
- **r8a (self)**: r7 pseudo-labels arwiki (RUNNING, label-v4; keep_frac
10+
filter at train time). Our teacher: 2.2864 on the harder
11+
expert-reviewed SadeedDiac-25.
12+
- **r8b (QCRI silver)**: their published Wikipedia_20240420.diac.jsonl
13+
(95MB, ~5M words, machine-labeled by their BiLSTM ~3% WER). Published
14+
dataset — same standing as Tashkeela/Nakdimon in our stack; NOT an LLM
15+
(no-LLM rule intact). Provenance + license recorded in the dataset
16+
repo.
17+
18+
**Dual-surface gate (both arms, in-job):** SadeedDiac-25 DER (hold the
19+
2.2864 line) AND WikiNews-2024 multiref WER/DER (move 17.38/11.83).
20+
A model that wins OOD but collapses ID is a regression, not a win.
21+
Ship the arm that dominates both; if neither does, negative verdict.

‎TODO.sota/18-parity-runs.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# WO18 — parity runs for the shipped artifacts
2+
3+
1. Attach golden/heb-diac-plane-2.0.jsonl to the golden-v2 release.
4+
2. Dispatch neural-parity (3 legs: ruby, python, ts) for
5+
heb-diac-plane-2.0 — first model validated across ALL runtimes at
6+
ship time.
7+
3. Green runs recorded in RESULTS.md.

‎TODO.sota/19-lexicon-loader.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# WO19 — lexicon fetch helper (client convenience)
2+
3+
`thai_hybrid.fetch_lexicon()`: resolve the canonical release asset
4+
(tha-lexicon-kaikki-1.0) with sha256 verification into a Lexicon —
5+
same trust contract as model artifacts (no unverified data paths).
6+
TDD; py first (ts/ruby follow on demand).

0 commit comments

Comments
 (0)