feat(gpu): Engram byte-n-gram conditional memory + validated framing protocol - #215
Merged
Merged
Conversation
added 2 commits
September 12, 2026 21:34
…opened) The lexical-memory lever: one embedding table addressed by hashes of the byte n-grams ending at each position (orders 2/3/4), summed into the encoder stream at one layer. Zero-init projection leaves the backbone untouched at step 0 (the PKM rule). int8 table + fp16 projection as the artifact form (64 MiB at 2M x 32). Export contract settled by the probe: hashing is NOT graph-exportable (python data-dependence) and int64 bitwise-and is rejected by ONNX - from_addresses is the pure-graph path (Gather + mean + Linear with remainder addressing, which also allows the report's prime table sizes), and the hash becomes a 20-line portable function per runtime, with addresses as an IMF input. Probe: export -> ORT -> torch match within 1e-4. 6 specs: determinism and order separation, byte-not-token addressing, zero-init identity, context sensitivity, int8 roundtrip, ONNX gather survival. Training run remains owner-gated.
Two protocol bugs invalidated the first framing runs: a cache-rebuild loop that re-fed window tokens onto a cache already containing them, and a window that dropped the PAD prefix (batch=1 degenerated to an order-1 Markov decode - fp32 'diverging' with itself was the tell). v3 windows over the full consumed sequence [PAD]+tokens and uses the judge call's own presents as the advanced cache. Sanity: fp32 framing 480/480 under Python ORT (the physics baseline).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements TODO.impl/10's free portion — the lexical-memory lever whose gate opened when 02 (teacher mixture) and 04 (trained-init lite) both closed negative.
The module (
src/gpu/engram.py): one embedding table addressed by hashes of the byte n-grams ending at each position (orders 2/3/4), summed into the encoder stream at one layer — the proportionate dose of DeepSeek's Engram for a 300M byte-level backbone. Zero-init projection leaves the backbone untouched at step 0 (the PKM rule). Artifact form: int8 table + fp16 projection — 64 MiB at 2M x 32, a defensible tier between lite (95 MB) and 2.1-int8 (264 MB).Export contract, settled by the probe: python-side hashing is not graph-exportable, and ONNX rejects int64 bitwise-AND — so
from_addressesis the pure-graph path (Gather + mean + Linear with remainder addressing, which also re-enables the report's prime table sizes), and the hash becomes a 20-line portable function per runtime with addresses as an IMF input. Probe passes: export → ORT → torch within 1e-4.Also: the validated framing protocol (greedy v3) that today's matrix was measured with — two protocol bugs invalidated the first runs (a cache-rebuild corruption and a PAD-dropping window); main still carried the buggy script.
6 Engram specs green. The training run remains owner-gated (one distill run with the table attached).