Skip to content

docs: run-022 verdict — Hebrew plane 8.18 DER (gate cleared) #262

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ronaldtse merged 1 commit into from
Oct 7, 2026
Merged

docs: run-022 verdict — Hebrew plane 8.18 DER (gate cleared) #262

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion TODO.sota/00-README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ hf jobs run --flavor a100-large -d \
| 01 | 01-wikinews-multiref-reconciliation.md | P1 | — | DONE (RESULTS #257) |
| 02 | 02-user-diacritics-preservation.md | P1 | — | DONE (py#28, ruby#801) |
| 03 | 03-thai-hybrid-latency.md | P1 | — | DONE (py#31, RESULTS #258) |
| 04 | 04-hebrew-plane-scaleup.md | P2 | #44 | gate CLEARED (8.31 in-job); artifact eval + owner-gated release remain |
| 04 | 04-hebrew-plane-scaleup.md | P2 | #44 | gate CLEARED — artifact DER 8.18 (int8, runtime protocol); release pending owner version |
| 05 | 05-external-benchmarks.md | P2 | 01, 07 | DONE (RESULTS #259; MILIM awaiting release) |
| 06 | 06-arabic-r8-noisy-student.md | P2 | #44 | jobs LAUNCHED (label + train, HF queue) |
| 07 | 07-nikud-ipa-rule-layer.md | P3 | — | DONE (py#29, py#30) |
Expand Down
21 changes: 21 additions & 0 deletions docs/RESULTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1195,3 +1195,24 @@ Dicta/Nakdimon-class G2P on CER; the remaining gap to Phonikud/ReNikud
is exactly stress + learned spoken-norm phonology — the audio-supervised
direction (WO09/2027). Bench: benchmarks/heb_g2p_bench.py; gold vendored
with sha256 and attribution.

## run-022 heb-plane-base — 8.18 DER at the artifact level: gate CLEARED, Hebrew -4.30pp (2026-10-07)

WO04 (TODO.sota/04): run-021 recipe scaled to byt5-base, 4 epochs, K=3,
trained on HF Jobs (a100-large, 1h03m wall). In-job nakdimon verdict
8.31 (bf16, exact run-021 protocol). Exported int8 (dynamic QInt8,
415.9 MB) and measured at the artifact level under the runtime protocol
(py PlaneModel, CPU, skeleton windows, original-separator stitching,
seq2seq_der; benchmarks/heb_plane_eval.py):

| artifact (nakdimon test, 1,864 lines / 223,872 positions) | DER | size |
|---|---|---|
| heb-diac-plane-1.0 (byt5-small, K=2) | 12.48 | 219 MB |
| **heb-diac-plane-2.0 (byt5-base, K=3, int8)** | **8.18** | 416 MB |

Harness note: the first artifact-eval run read 55 DER — the harness was
feeding diacritized text to the runtime (marks treated as base
characters); fixed by stripping via nikud_planes before windowing. The
trainer's in-job eval was always correct. Staged zip:
heb-diac-plane-2.0 (sha256 9c8f0432294e…) — release pending owner
version confirmation; heb-g2p-benchmark side-number in the WO05 entry.
Loading