Skip to content

Commit 20e7cf1

Browse files
author
Ronald Tse
committed
docs: seed variance confirms eval bug; run-062 corrected chain launched
1 parent 34ee049 commit 20e7cf1

1 file changed

Lines changed: 14 additions & 0 deletions

File tree

‎docs/RESULTS.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1932,3 +1932,17 @@ student gate (phonikud, vowel-full gold) is then CONSISTENT
19321932
end-to-end for the first time. The WO31 lesson generalizes: the
19331933
pointed-vs-unpointed convention must be asserted at EVERY stage —
19341934
targets, gates, and labels — not just the first espeak call.
1935+
1936+
Seed-variance result (run-061b, seed 1337 vs run-061 seed 42):
1937+
FLEURS 0.2055 vs 0.2069 · ivrit dev 0.8745 vs 0.8722 — seeds agree to
1938+
±0.002, so BOTH numbers are stable measurements. Conclusion: the
1939+
FLEURS gate pass is real (teacher improved 0.319 → 0.206 at 200h),
1940+
and the 0.87 ivrit-dev is seed-independent → confirmed as the EVAL
1941+
BUG hypothesized earlier: asr_phones' dict branch ignores the
1942+
recorded 32000 Hz sample rate (2×-time-warped audio → garbage decode).
1943+
The training path resamples correctly (pcm_to_16k uses r["sr"]).
1944+
run-062 LAUNCHED: labeler+student with the v3 teacher in a FRESH run
1945+
dir (run-062, not run-052 — the stale consonant-skeleton labels.jsonl
1946+
from the wrong-convention teacher cannot resume). Gate tonight:
1947+
phonikud nCER < 0.2393 on vowel-full labels — the first
1948+
end-to-end-consistent Hebrew g2p measurement.

0 commit comments

Comments
 (0)