feat(ml): windowed long-input translate (1400-byte protocol, client-side) - #83
Merged
Merged
Conversation
…nt-side) Models train on <=1400-byte windows; longer inputs were reaching them out-of-distribution and degrading. splitWindows reproduces the published Python harness split exactly (word-boundary, byte-budget, Python split semantics - the fixture is generated from the reference and caught the empty-token difference). translateWindowed routes both IMFModel.translate and SpeculativeModel.translate: single-window inputs (every golden-set size) pass straight through, so cross- runtime byte parity is untouched; long inputs decode as the join of their window decodes. translateDirect is the former body. 7 split-parity specs + 3 behavior specs; full suite 286.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Client-side windowing for long inputs — the runtime half of the 1400-byte protocol.
Models train on ≤1400-byte windows; the runtime previously fed whole inputs regardless, so long texts reached them out-of-distribution and degraded. Now:
splitWindowsreproduces the published Python harness split exactly (word-boundary, byte-budget). The parity fixture is generated from the reference implementation and caught the one semantic difference (Pythonstr.split()drops empty tokens on whitespace runs; JSsplit(" ")does not).translateWindowedroutes bothIMFModel.translateandSpeculativeModel.translate: single-window inputs — every golden-set size — pass straight through untouched, so cross-runtime byte parity is unaffected; long inputs decode as the join of their window decodes.translateDirectis the former decode body.10 new specs; full suite 286 passed. This also future-proofs the speculative path: its windows are now protocol windows, never raw long inputs.