feat(export): static-int8 artifact path (calibration + builder) - #217
Merged
Merged
Conversation
collect_decode_calibration: decoder feeds across BOTH framings
(prefill, incremental, 8-token windows) from a model's own eval
pairs - the shared recipe, DRY with the measured experiment.
quantize_int8_static: calibrated QUInt8 activations, MatMul-only,
head fp32, QOperator format.
export_int8_static (modal_export): builds {mid}-int8static.zip from
the fp32 zip - fp32 encoder + static decoder, the composition scored
4.6241 full-set vs dynamic 4.5701 with +8% CPU decode speed. The
docstring carries the honest composition note: browser-size needs an
int8-encoder variant gated by its own run. Smoke: khm-latn built
end-to-end (25 feeds, shas refreshed). Release swap remains a version
decision with the --out CI as the bar (running).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The export-side half of TODO.impl/11's positive branch: the artifact builder that makes the static-int8 re-export a one-command operation.
collect_decode_calibration(imf/export.py): decoder feeds across BOTH framings — prefills, incremental single-steps, 8-token windows — collected from the model's own eval pairs. The shared recipe, DRY with the experiment that measured the gate.quantize_int8_static: calibrated QUInt8 activations, MatMul-only, head fp32, QOperator — the validated node policy.export_int8_static(modal_export): builds{mid}-int8static.zipfrom the fp32 zip — fp32 encoder + static decoder, the exact composition scored 4.6241 full-set vs dynamic 4.5701 (+0.054pp, drift band) with +8% CPU decode speed. The docstring carries the honest composition note: a browser-sized artifact needs an int8-encoder variant gated by its own run.The release swap (index-v6, golden regen) remains a version decision; its stated bar — the paired-bootstrap CI from per-row predictions — is currently scoring on Modal.