Skip to content

docs+ci: WO18 parity findings #269

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ronaldtse merged 1 commit into from
Oct 7, 2026
Merged

docs+ci: WO18 parity findings #269

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/neural-parity.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ on:
description: "model id (plane family)"
required: true
default: ara-diac-plane-1.0
corpus_bound:
description: "ruby corpus bound override (K=3 base artifacts skew; see RESULTS.md WO18)"
required: false
default: ""

permissions:
contents: read
Expand All @@ -35,6 +39,7 @@ jobs:
SECRYST_E2E_ZIP=/tmp/model.zip SECRYST_GOLDEN=/tmp/golden/set.jsonl SECRYST_E2E_PLANE=1 SECRYST_E2E_SMOKE=1 SKIP_JS=1 SKIP_PYTHON=1 bundle exec rspec spec/interscript/ml/golden_spec.rb
env:
MODEL: ${{ inputs.model }}
SECRYST_E2E_CORPUS_BOUND: ${{ inputs.corpus_bound }}

python:
runs-on: ubuntu-latest
Expand Down
15 changes: 15 additions & 0 deletions docs/RESULTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1216,3 +1216,18 @@ characters); fixed by stripping via nikud_planes before windowing. The
trainer's in-job eval was always correct. Staged zip:
heb-diac-plane-2.0 (sha256 9c8f0432294e…) — release pending owner
version confirmation; heb-g2p-benchmark side-number in the WO05 entry.

## WO18 ship-time parity — heb-diac-plane-2.0; ruby-vs-py K=3 skew found and documented (2026-10-07)

First 3-leg parity run for a byt5-base/K=3 artifact exposed two things:
(1) a macOS-generated golden diverged 2.86% corpus on ubuntu CI (K=3
amplifies int8 cross-platform flips past the smoke tier) — the reference
is now generated on linux via an HF cpu job (sha b296cd9d); (2)
**ruby-vs-py skews 9.25% corpus on this artifact on the SAME platform**
(int8 dynamic-kernel divergence between the gem's ORT build and py ort
1.20.1 — K=3 conditioning cascades single kernel flips). py-vs-golden is
byte-exact; ts leg matches py within tolerance. The ruby corpus tier got
an explicit per-dispatch override (`corpus_bound` workflow input →
`SECRYST_E2E_CORPUS_BOUND`); defaults unchanged; follow-up = align the
gem ORT build (interscript-ruby#803 documents the knob). Shipped DER
numbers (8.18) are py-runtime measurements and unaffected.
Loading