docs(paper-c): decode framing parity section + static-int8/soup experiment scripts - #213
Merged
Merged
Conversation
added 4 commits
September 12, 2026 14:33
…+ soup probe scripts Paper C gains section 5.1: dynamically quantized graphs compute activation scales per fed tensor, so decode framing (batch shape) is a parity variable on the same machine/runtime/artifact - measured 0.03/element KV divergence, 33/66 self-acceptance, 0.99-vs-0.46 framing-relative acceptance. Quantized quality parity is framing-scoped; static scales are the proposed structural fix. scripts/static_int8_experiment.py: the pre-registered experiment (TODO.impl/11) - quantize_static on the 2.1 decoder calibrated over 365 real decode feeds in both framings; measures framing equality (gate), fp32 drift (proxy), and speed vs the shipped dynamic int8. modal_teacher_sadeed: --soup-both probes the r6+r7 50/50 weight average under the same windowed protocol (same-basin check; garbage output closes the axis on its own).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The framing finding, written into the paper and armed as an experiment.
Paper C §5.1 — decode framing is a parity axis. Dynamically quantized ONNX graphs compute activation scales per fed tensor (DynamicQuantizeLinear), so batch shape changes the decode on the same machine, runtime, and artifact — a variable cross-runtime and cross-hardware parity studies do not cover. Measured: KV divergence ~0.03/element (fp32 agrees to ~1e-6), drafter==verifier self-acceptance 33/66, speculative acceptance 0.99 under uniform framing vs 0.46 under runtime framing. Contract consequence: quantized quality parity is framing-scoped, and acceptance figures must disclose their framing.
The structural fix, pre-registered (rababa TODO.impl/11).
scripts/static_int8_experiment.pyre-quantizes the 2.1 decoder with static QUInt8 activation scales (MatMul-only, head fp32 — the shipped node policy), calibrated on 365 real decode feeds sampled across BOTH framings. Gates: framing-equality (identical single-step vs batched trajectories), fp32 drift no worse than dynamic int8; speed recorded. If both clear, all quantized artifacts re-export through the static path and the framing claim joins the contract.Seed-soup probe.
modal_teacher_sadeed --soup-bothweight-averages r6+r7 50/50 and runs the same windowed protocol — the free pre-r8 check (same-basin soup works, or garbage output closes the axis).