diff --git a/TODO.impl/11-static-int8-framing.md b/TODO.impl/11-static-int8-framing.md index a76ef84..c1dd9bc 100644 --- a/TODO.impl/11-static-int8-framing.md +++ b/TODO.impl/11-static-int8-framing.md @@ -37,10 +37,31 @@ If A and B clear: re-export all quantized artifacts through the static path (modal_export gains the calibration stage), index-v6, golden rows regenerate, framing claim added to the contract. +## Verdict (2026-09-12, final): framing gate FAILED for the node fix — the instability is the node build itself + +The test run at every endpoint (RESULTS.md, PR #214): + +- Python ORT: 480/480 framing-stable at fp32, dynamic int8, AND + static int8 +- onnxruntime-node: 89-101/485 UNSTABLE at every precision INCLUDING + fp32 — batch-shape-dependent kernel numerics before quantization + enters; static scales cannot repair what is not a scale computation +- static int8 byproduct: quality-clean on sample (0/480 drift vs + fp32) and ~8% faster than dynamic on CPU — stays a candidate for + the export path on its own merits, gated by full-set DER, NOT by + framing + +Re-scoped conclusion: framing stability is a property of the runtime +distribution. Shipped TS paths are single-framing by construction, so +published byte parity is unaffected; framing is the parity variable +of mixed-shape serving (why the speculative tier degraded and was +pulled — TODO.impl/01). + ## Status - [x] Pre-registered (this file) - [x] fp32 artifact fetched, sha-verified against index-v5 - [x] Calibration corpus collected (365 feeds, both framings) -- [ ] quantize_static result measured (A/B/C) -- [ ] Verdict recorded in RESULTS.md; ledger cross-links updated +- [x] quantize_static measured (A/B/C) across BOTH runtime builds +- [x] Verdict recorded in RESULTS.md + Paper C 5.1; framing-fix + closed negative; static-int8 speed/quality kept open diff --git a/TODO.impl/README.md b/TODO.impl/README.md index cdf4493..60da5c5 100644 --- a/TODO.impl/README.md +++ b/TODO.impl/README.md @@ -24,7 +24,7 @@ Priorities: | 08 | golden-closeout.md | golden test PR, paper-c wording, release README note | P0 | closed 2026-09-12 (PR #211 merged) | | 09 | doc-corrections.md | Engram mechanism fix, mHC single-pass note | P2 | landed in PR #95 | | 10 | engram-lexical-memory.md | hashed n-gram memory module | P3 | gated behind 04 | -| 11 | static-int8-framing.md | static activation scales: fix framing at the source | P0 | pre-registered; experiment running | +| 11 | static-int8-framing.md | static scales as the framing fix | P0 | closed: node-kernel finding; static stays open on speed | Standing rules that apply to every item: full-set measurement or no claim; quantized parity is quality-level, not byte-level; LLM teachers