From a49c245c3a81a4cbac72a4706140ed0e75626205 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sat, 12 Sep 2026 18:57:00 +0800 Subject: [PATCH] docs: framing matrix complete - a distribution property, not quantization The single-vs-batched test run at every endpoint: Python ORT 480/480 at fp32, dynamic int8, AND static int8; onnxruntime-node 89-101/485 at every precision including fp32. Static scales do not repair the node build - its kernels differ across batch shapes before quantization enters. Shipped paths are single-framing so published byte parity is unaffected; framing is the parity variable of mixed-shape serving. Paper C 5.1 carries the final scoping; static int8 stays a candidate for the export path on speed and quality, decided by full-set DER. --- docs/RESULTS.md | 34 ++++++++++++++++++++++++++++++++++ docs/paper-c.adoc | 18 ++++++++++++------ 2 files changed, 46 insertions(+), 6 deletions(-) diff --git a/docs/RESULTS.md b/docs/RESULTS.md index 6e12d04..1955a07 100644 --- a/docs/RESULTS.md +++ b/docs/RESULTS.md @@ -752,3 +752,37 @@ Arabic adapted-init collapses under the strong one). The lite tier remains run-009 (5.78); init-source closes negative and the depth axis now reads 2-of-3 negative. Remaining architecture lever: TODO.impl/10 (lexical memory), gated as before. + +## Framing completes as a distribution×precision matrix (2026-09-12, final) + +The same single-vs-batched greedy test (5 golden rows x 96 steps, same +machine, ORT 1.23 everywhere), run to its endpoints: + +| runtime | precision | single==batched | +|---|---|---| +| Python ORT | fp32 | 480/480 | +| Python ORT | dynamic int8 | 480/480 | +| Python ORT | static int8 | 480/480 | +| onnxruntime-node | fp32 | **101/485** | +| onnxruntime-node | dynamic int8 | 97/485 | +| onnxruntime-node | static int8 | 89/485 | + +The framing instability is NOT a quantization property: the node build +diverges across batch shapes at fp32, and static activation scales +(pre-computed into the graph — TODO.impl/11's proposed fix) do not +repair it. It is a property of the onnxruntime-node kernel paths +(multi-token inputs take numerically different code paths than +single-token inputs), absent from the Python build at every precision. + +Contract consequences, final form: +- cross-FRAMING parity holds under the Python reference at all + precisions and does not hold under onnxruntime-node at ANY precision +- the shipped TS runtime is unaffected in practice: every shipped path + (translate, worker, CLI) is single-framing; framing becomes a + parity variable exactly when a runtime mixes batch shapes + (speculative decode, batched serving) — which is why the speculative + tier degraded and was pulled +- static int8 remains a POSITIVE byproduct: quality-clean vs fp32 on + this sample (0/480 drift) and ~8% faster than dynamic on CPU — + candidate for the export path on its own merits, decided by + full-set quality, not framing diff --git a/docs/paper-c.adoc b/docs/paper-c.adoc index d2ec734..580876c 100644 --- a/docs/paper-c.adoc +++ b/docs/paper-c.adoc @@ -96,12 +96,18 @@ cross-runtime nor cross-hardware studies cover: drafting, batched verification) — acceptance figures are framing-relative and must disclose the framing -Contract consequence: our quantized quality-parity guarantee is -framing-scoped (the framing is part of the protocol that a cer_delta -gate pins). The proposed structural fix is static activation scales -(quantize_static with calibration over both framings), which removes -the per-run scale computation entirely — pre-registered as -TODO.impl/11 with framing-equality as the gate. +Contract consequence, measured to its endpoints: framing stability is +a property of the RUNTIME DISTRIBUTION, not the graph. The same +single-vs-batched test passes 480/480 under the Python build at fp32, +dynamic-int8, AND static-int8 — and fails under onnxruntime-node at +every precision including fp32 (89-101/485). The proposed static-scale +fix (calibrated quantize_static) does not repair it: the node build's +kernels differ across batch shapes at float precision, before +quantization enters. Shipped paths are single-framing by construction, +so byte parity as published is unaffected; framing becomes a parity +variable precisely when a runtime mixes batch shapes (speculative +decode, batched serving) — and acceptance or parity figures measured +under one framing do not transfer to another. == 6. Reproducibility discipline - subset-overstatement: five instances, up to 3.2x inflation