From bbdd91204944d788edab097626ab3bff678d8610 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Tue, 6 Oct 2026 20:51:44 +0800 Subject: [PATCH] docs: TODO.final - the consolidation campaign (13 items, dependencies, priorities) --- TODO.final/01-pypi-release-blocked.md | 22 ++++++++++++++++ TODO.final/02-website-collapse.md | 16 ++++++++++++ TODO.final/03-trainer-rename.md | 15 +++++++++++ TODO.final/04-corpora-consolidation.md | 16 ++++++++++++ TODO.final/05-secryst-deprecation-shims.md | 13 ++++++++++ TODO.final/06-legacy-archival.md | 13 ++++++++++ TODO.final/07-hub-cleanup.md | 12 +++++++++ TODO.final/08-ruby-provisioning-paths.md | 11 ++++++++ .../09-catalog-rename-interscript-models.md | 21 +++++++++++++++ TODO.final/10-version-generation-policy.md | 14 ++++++++++ TODO.final/11-neural-parity-goldens.md | 12 +++++++++ TODO.final/12-plane-family-expansion.md | 13 ++++++++++ TODO.final/13-api-edge-unification.md | 12 +++++++++ TODO.final/README.md | 26 +++++++++++++++++++ 14 files changed, 216 insertions(+) create mode 100644 TODO.final/01-pypi-release-blocked.md create mode 100644 TODO.final/02-website-collapse.md create mode 100644 TODO.final/03-trainer-rename.md create mode 100644 TODO.final/04-corpora-consolidation.md create mode 100644 TODO.final/05-secryst-deprecation-shims.md create mode 100644 TODO.final/06-legacy-archival.md create mode 100644 TODO.final/07-hub-cleanup.md create mode 100644 TODO.final/08-ruby-provisioning-paths.md create mode 100644 TODO.final/09-catalog-rename-interscript-models.md create mode 100644 TODO.final/10-version-generation-policy.md create mode 100644 TODO.final/11-neural-parity-goldens.md create mode 100644 TODO.final/12-plane-family-expansion.md create mode 100644 TODO.final/13-api-edge-unification.md create mode 100644 TODO.final/README.md diff --git a/TODO.final/01-pypi-release-blocked.md b/TODO.final/01-pypi-release-blocked.md new file mode 100644 index 0000000..b952203 --- /dev/null +++ b/TODO.final/01-pypi-release-blocked.md @@ -0,0 +1,22 @@ +# 01 — pip `interscript` release (BLOCKED: external) + +**Status: BLOCKED** on the previous developer fixing the PyPI trusted +publisher for the `interscript` project (the workflow's OIDC claim is +rejected `environment: MISSING` vs the registered publisher; pypi.org +side only). Everything else is ready. + +## Already done +- interscript-py main carries the unified surface: `interscript.ml` + ([ml] extra), transliterate dispatch by index kind, index-v7 pin, + INTERSCRIPT_ML_* env vars (PR #24). +- v0.2.0 tag is burned (a failed release run, never published) — the + debut MUST NOT reuse it. + +## Steps when unblocked +1. Owner states the debut version (0.3.0 per plan, or 1.0.0 to signal + maturity — owner decision). +2. If the TP entry expects a GitHub environment: either the pypi.org + entry drops it, or the release job declares `environment: `. +3. Tag/ship via the repo release path; verify PyPI listing + `pip + install interscript[ml]` smoke. +4. Close with the deprecation shim (TODO 05) once shipped. diff --git a/TODO.final/02-website-collapse.md b/TODO.final/02-website-collapse.md new file mode 100644 index 0000000..71b5e2c --- /dev/null +++ b/TODO.final/02-website-collapse.md @@ -0,0 +1,16 @@ +# 02 — Website collapse: one site, one model page + +**Status: pending.** interscript.org/ml is live but predates the plane +family and flagship leaderboard; secryst.github.io still serves its own +zoo — two model pages exist (split-brain confirmed). + +## Steps +1. Port the secryst.github.io/models content that /ml lacks: flagship + section (ara-diac-2.0, SadeedDiac-25 leaderboard), plane rows + (large 2.74 / small 3.59 + on-device numbers), subset-retraction + disclosure, HF org + collection links. +2. Verify /ml catalog completeness against models.yaml index-v7 (26 + entries; script the diff, never hand-count). +3. secryst.github.io becomes a redirect stub to interscript.org (repo + kept, never deleted). +4. Cross-link runtimes' READMEs to /ml. diff --git a/TODO.final/03-trainer-rename.md b/TODO.final/03-trainer-rename.md new file mode 100644 index 0000000..37d9955 --- /dev/null +++ b/TODO.final/03-trainer-rename.md @@ -0,0 +1,15 @@ +# 03 — rababa → interscript-train + +**Status: owner-approved direction; execution pending.** The trainer +absorbed all languages (Arabic/Hebrew/Urdu + plane + evals); the +Arabic-specific name is the last incoherence. + +## Steps +1. Rename GitHub repo (redirects follow automatically). +2. Update local clone remotes. +3. Sweep live references: CI, docs, bootstrap-style scripts, + cross-repo READMEs. Historical records (RESULTS.md "rababa run-0NN", + published cards' provenance lines) stay AS WRITTEN — they are the + record of what happened. +4. Fold secryst-train's still-relevant Thai-era bits in; archive + secryst-train (never delete). diff --git a/TODO.final/04-corpora-consolidation.md b/TODO.final/04-corpora-consolidation.md new file mode 100644 index 0000000..03cd2ed --- /dev/null +++ b/TODO.final/04-corpora-consolidation.md @@ -0,0 +1,16 @@ +# 04 — rababa-* corpora family consolidation + +**Status: pending.** 11 corpus/data repos in the interscript org +(rababa-farsi, -urdu, -urdu-corpus, -persian-corpus, -tashkeela-full, +-hebrew-distilled, -hewiki, -arwiki, -tashkeela, -sefaria) plus +rababa-models and data-* in the secryst org. + +## Steps +1. Inventory each: size, last activity, which training runs consume it. +2. Corpora with no code become directories under interscript-train + (data/), preserving history via git subtree or archived mirror + + pointer. Keep any repo that external processes push to. +3. Migrate secryst org data repos (data-arabic-pointing, + data-thai-interscript, data-khmer-translit, rababa-models) into the + same structure. +4. Archive emptied repos (never delete). diff --git a/TODO.final/05-secryst-deprecation-shims.md b/TODO.final/05-secryst-deprecation-shims.md new file mode 100644 index 0000000..9d67ecb --- /dev/null +++ b/TODO.final/05-secryst-deprecation-shims.md @@ -0,0 +1,13 @@ +# 05 — secryst package deprecation shims (final releases) + +**Status: pending.** The `secryst` names are frozen at their last +releases (npm 0.3.1, PyPI 0.2.1, gem 1.1.1) with index-v7 pins. Old +users keep working; nobody is told where the line continues. + +## Steps +1. Final shim releases that only change metadata/message to point at + `interscript` (+[ml] where applicable): npm secryst 0.3.2, PyPI + secryst 0.2.2, gem secryst 1.1.2. Owner confirms numbers. +2. Mark npm package deprecated (`npm deprecate` message), PyPI via + the shim description, RubyGems via the gemspec description. +3. Archive secryst/{secryst,secryst-py,secryst-ts,secryst-train} after. diff --git a/TODO.final/06-legacy-archival.md b/TODO.final/06-legacy-archival.md new file mode 100644 index 0000000..2502209 --- /dev/null +++ b/TODO.final/06-legacy-archival.md @@ -0,0 +1,13 @@ +# 06 — Legacy repo archival + +**Status: pending.** Open-but-dead repos dilute the org. + +## List (verify each is truly dead before archiving) +- interscript/interscript-js (dead name; hub still clones it — see 07) +- interscript/interscript-py-mvp +- interscript/interscript-ts-old +- secryst org repos after 04/05 complete (crystalseq, + transliteration-learner-from-graph-models, onnxruntime fork, etc.) + +Archive (read-only), never delete. Anything with unmerged unique work +gets that work salvaged first. diff --git a/TODO.final/07-hub-cleanup.md b/TODO.final/07-hub-cleanup.md new file mode 100644 index 0000000..5c0600b --- /dev/null +++ b/TODO.final/07-hub-cleanup.md @@ -0,0 +1,12 @@ +# 07 — Release hub cleanup (interscript/interscript) + +**Status: pending.** The hub now works (gem 3.0.0 via OIDC) but clones +dead repos and carries campaign TODO dirs. + +## Steps +1. bootstrap.rb: drop interscript-js (dead) from the package set; keep + ruby + maps (+ ts if the JS leg is ever revived — it should consume + interscript-ts instead). +2. Move TODO.complete/TODO.rababa/TODO.secryst into docs/archive/ (they + are campaign records, not live plans). +3. Confirm the OIDC gem path stays the single release entry point. diff --git a/TODO.final/08-ruby-provisioning-paths.md b/TODO.final/08-ruby-provisioning-paths.md new file mode 100644 index 0000000..ddf5b38 --- /dev/null +++ b/TODO.final/08-ruby-provisioning-paths.md @@ -0,0 +1,11 @@ +# 08 — Ruby ML provisioning path migration + +**Status: pending.** The ported Interscript::ML::Provisioning still +writes secryst-named paths (/var/lib/secryst, ~/.local/share/secryst). + +## Steps (TDD) +1. Failing spec: provisioning writes under interscript-ml names. +2. Write paths become interscript-ml/*; legacy secryst paths remain + READ-ONLY fallbacks for one transition period. +3. INTERSCRIPT_ML_DATA primary, SECRYST_DATA alias (already done — + verify coverage in specs). diff --git a/TODO.final/09-catalog-rename-interscript-models.md b/TODO.final/09-catalog-rename-interscript-models.md new file mode 100644 index 0000000..d638903 --- /dev/null +++ b/TODO.final/09-catalog-rename-interscript-models.md @@ -0,0 +1,21 @@ +# 09 — interscript-ml → interscript-models (+ index-v8) + +**Status: owner direction agreed (plural form recommended to mirror +interscript-maps).** The repo is the model catalog/contract layer; the +"ml" name misleads (it is the STABLE side, not the ML code). + +## Why v8 must ride along +Three shipped runtimes pin index URLs at +github.com/interscript/interscript-ml/releases/download/... — after a +repo rename those depend on redirects. The clean fix: + +## Steps +1. Rename repo → interscript-models. +2. Cut index-v8 with canonical interscript-models URLs (models.yaml + unchanged in content). +3. Runtime patch bumps pinning v8: npm 5.6.x, gem 3.0.x, pip with its + debut. Exact-pin specs updated test-first. +4. pip tools package naming (interscript-ml 0.1.1 / interscript-ml-tools) + follows the owner's call in the same window. +5. HF workflows and READMEs re-pointed; historical RESULTS/anchors stay + as written. diff --git a/TODO.final/10-version-generation-policy.md b/TODO.final/10-version-generation-policy.md new file mode 100644 index 0000000..ae17194 --- /dev/null +++ b/TODO.final/10-version-generation-policy.md @@ -0,0 +1,14 @@ +# 10 — Version-generation policy across runtimes + +**Status: owner decision pending.** The same runtime generation ships +as npm 5.6.0 / gem 3.0.0 / pip 0.1.0(stale) — three major lines for one +product. + +## Options +A. Independent per-registry versions; publish the policy that cross- + runtime conformance (goldens + index pin) is the real contract. +B. Aligned generation majors (e.g. all 3.x = unified-dispatch era) with + per-registry minors. + +Recommend A (documented) unless the owner wants the marketing clarity +of B. Either way: /ml and READMEs state which index each release pins. diff --git a/TODO.final/11-neural-parity-goldens.md b/TODO.final/11-neural-parity-goldens.md new file mode 100644 index 0000000..b29c844 --- /dev/null +++ b/TODO.final/11-neural-parity-goldens.md @@ -0,0 +1,12 @@ +# 11 — Neural cross-runtime parity goldens + +**Status: pending.** Goldens cover maps (7388/7388 parity). Model +inference (IMF seq2seq + plane K-pass) has no cross-runtime corpus: +each runtime implements decode independently. + +## Steps +1. Extend golden-v1 with model-inference vectors: fixed inputs → + expected outputs per runtime (tolerance policy for fp variants). +2. CI job diffs py/ts/ruby on the neural goldens (the maps-parity job + pattern). +3. Gate: no runtime release with a neural-golden divergence. diff --git a/TODO.final/12-plane-family-expansion.md b/TODO.final/12-plane-family-expansion.md new file mode 100644 index 0000000..f49ac7b --- /dev/null +++ b/TODO.final/12-plane-family-expansion.md @@ -0,0 +1,13 @@ +# 12 — Plane family expansion (Hebrew nikud, Urdu) + +**Status: pending (architecture proven).** The plane factorization is +language-agnostic; corpora already sit in the data layer (hewiki, +rababa-hebrew-distilled, urdu corpora). + +## Steps +1. Hebrew nikud plane: inventory (54-class analog for nikud combos), + train (rababa/interscript-train run), full-set eval vs heb-diac-1.1 + (16.44 DER), gates, artifact, index entry, HF publish. +2. Urdu plane: same shape vs urd-diac-1.0 (3.74 CER). +3. Each lands through the standard gates; no index entry without + provenance. diff --git a/TODO.final/13-api-edge-unification.md b/TODO.final/13-api-edge-unification.md new file mode 100644 index 0000000..9986bc5 --- /dev/null +++ b/TODO.final/13-api-edge-unification.md @@ -0,0 +1,12 @@ +# 13 — API edge on the unified runtime + +**Status: pending.** The Cloudflare worker (interscript-api) predates +the unified ts surface. + +## Steps +1. Worker consumes interscript-ts (one transliterate contract: maps + + models via index). +2. Bundled-maps artifact unchanged (data channel stays Releases); + models resolve on demand or pre-warm per region (cost decision). +3. Conformance: worker outputs byte-match local runtime on the shared + golden set. diff --git a/TODO.final/README.md b/TODO.final/README.md new file mode 100644 index 0000000..07b25be --- /dev/null +++ b/TODO.final/README.md @@ -0,0 +1,26 @@ +# TODO.final — the consolidation campaign (2026-10-06) + +Closes every remaining thread after the 2026 SOTA campaign and the +secryst→interscript migration. Predecessors: TODO.sota-2026 (closed), +interscript/interscript TODO.complete (closed), TODO.restructure +(closed via interscript-ruby #792). + +| # | item | priority | blocked on | +|---|------|----------|------------| +| 01 | pip interscript release | P0 | previous developer (PyPI TP env) | +| 02 | website collapse | P0 | — | +| 03 | trainer rename (rababa → interscript-train) | P0 | — | +| 04 | corpora consolidation | P1 | 03 | +| 05 | secryst deprecation shims | P1 | owner version numbers | +| 06 | legacy archival | P1 | 04, 05 | +| 07 | hub cleanup | P1 | — | +| 08 | ruby provisioning paths | P1 | — | +| 09 | catalog rename → interscript-models + index-v8 | P1 | owner go | +| 10 | version-generation policy | P2 | owner decision | +| 11 | neural parity goldens | P2 | — | +| 12 | plane family expansion (heb/urd) | P2 | — | +| 13 | API edge unification | P2 | — | + +Rules carried from prior campaigns: honest specs only (numbers from +measurement); artifacts via Releases+sha256 only; archive never delete; +versions are the owner's to state; TDD on all code paths.