Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 43 additions & 10 deletions TODO.impl/10-engram-lexical-memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,21 +19,54 @@ a hashed n-gram lookup table is a direct fit for the idiom memory a
- Corrected mechanism per 09: hash-addressed tables in the forward
pass (DeepSeek config), not example retrieval.

## Gate: OPENED (2026-09-12)

02 (teacher mixture) and 04 (trained-init lite) both closed negative;
by this file's own pre-registered condition, the lexical-memory lever
is now the remaining architecture move for the student-side residual
(news/wiki-skewed, interaction-level).

## If built

- Table as a parameter module (addresses = n-gram hashes of the input
bytes; embeddings summed at the two chosen layers); trained with
the distill recipe; Sinkhorn-balanced table updates (06) are the
natural pairing, as in the report.
- Export probe first: a table-augmented student exported to ONNX with
decode-health smoke before any quality run.
BYTES - the corpus is byte-level, so orders {2,3,4} are byte
n-grams); embeddings summed into the encoder stream at ONE layer
(DeepSeek uses two at 552B; at 300M one is the proportionate dose).
- Table updates pair with the Sinkhorn-balanced rule (06), as in the
report.

## Size budget (honest arithmetic, int8 storage)

| table | size |
|---|---|
| 1M entries x 64-dim int8 | 64 MiB |
| 2M entries x 32-dim int8 | 64 MiB |
| 4M entries x 64-dim int8 | 256 MiB (over budget) |

The browser tier ships 95 MB today (lite int4). A 64 MiB table on top
is a ~168 MB tier - a NEW tier between lite and 2.1-int8 (264 MB),
defensible. Ceiling: ONE table, <= 64 MiB, int8-in-zip (fp16 at
inference, dequantized at load: the zip stays small, RAM grows 2x).
Primary config: 2M x 32 int8 (more lexical coverage per byte).

## Export feasibility probe (before any training)

Gather ops over an embedding table must survive torch.onnx.export ->
ORT -> the IMF zip pipeline. Probe: tiny table (1k x 8) attached to
the tiny T5 fixture, exported, loaded in ORT, decode-health smoke.

## Acceptance (only after the gate)
## Acceptance

- [ ] Size/latency budget for the browser tier documented
- [ ] Table module + export probe
- [ ] One run, full-set verdict with intervals
- [x] Size/latency budget documented (above)
- [x] Table module (src/gpu/engram.py) + 6 unit tests (PR #215)
- [x] Export probe: gather graph survives ONNX+ORT within 1e-4; the
hash becomes a per-runtime 20-line function (addresses as an
IMF input); int64 remainder addressing (primes available)
- [ ] One distill run, full-set verdict with intervals (GPU spend:
owner call on the run, the module and probe are free)

## Status

- [ ] Gated behind 02/04 results; not built
- [x] Gate opened (02 and 04 closed negative)
- [x] Module + probe landed
- [ ] Run (owner-gated)
10 changes: 10 additions & 0 deletions TODO.impl/11-static-int8-framing.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,16 @@ published byte parity is unaffected; framing is the parity variable
of mixed-shape serving (why the speculative tier degraded and was
pulled — TODO.impl/01).

## Speed branch: POSITIVE (2026-09-12, full-set)

The static artifact's quality gate PASSED: **4.6241** full-set (n=1200)
vs the shipped dynamic's 4.5701 — +0.054pp, the artifact-drift band —
while carrying +8% CPU decode speed (78 vs 72 tok/s). Re-exporting all
quantized artifacts through the static path is a release decision:
modal_export gains the calibration stage, index-v6, golden regen, and
the gate re-runs with --out so the delta ships with a CI. Point
estimate recorded in RESULTS.md.

## Status

- [x] Pre-registered (this file)
Expand Down
4 changes: 2 additions & 2 deletions TODO.impl/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ Priorities:
| 07 | diacritization-depth-knob.md | depth-conditioned vocalization | P3 | product decision |
| 08 | golden-closeout.md | golden test PR, paper-c wording, release README note | P0 | closed 2026-09-12 (PR #211 merged) |
| 09 | doc-corrections.md | Engram mechanism fix, mHC single-pass note | P2 | landed in PR #95 |
| 10 | engram-lexical-memory.md | hashed n-gram memory module | P3 | gated behind 04 |
| 11 | static-int8-framing.md | static scales as the framing fix | P0 | closed: node-kernel finding; static stays open on speed |
| 10 | engram-lexical-memory.md | hashed n-gram memory module | P3 | gate opened; module + export probe landed (PR #215); run owner-gated |
| 11 | static-int8-framing.md | static scales as the framing fix | P0 | framing closed (node-kernel); speed branch POSITIVE (4.6241, +8%) |

Standing rules that apply to every item: full-set measurement or no
claim; quantized parity is quality-level, not byte-level; LLM teachers
Expand Down