[ZackSample]

The problem

A fact's lesson fits its pocket neurons on single-turn prompts, and every check asks a single-turn question. In a real
chat the question comes after other turns, and then it reads differently.

I measured this on Qwen3.8 Flash Next (our CUDA port, the Living Weights code of this repo): 8 user facts taught, then
the same held-out questions asked in five contexts, all greedy. The assistant turns of the conversations are the model's
own live replies.

context	user facts recalled
cold	14/18 (78%)
after a "You are a helpful assistant." system prompt	14/18
after 3 turns of small talk	13/18
after an unrelated ~4k-token task chat (debugging a CSV script)	8/18 (44%)
chained (one fact's question after another's)	6/6 first in a chain, 5/24 after it

The untaught model scored 0 everywhere. I read every answer: the misses are "I don't have access to your personal
information" or an invented value.

The gate shows why. I read each learned neuron's u x at the decide row of each probe prompt, with the fit's scale.

• When a fact is recalled cold, its reply neurons sit at about +3 (fit level 1, silence margin -0.5).
• After the long chat or mid-chain, the same question puts them at about 0, under the firing level.
• Over all contexts, prompts whose neurons fired were recalled 55/61. Prompts whose neurons were quiet were recalled
  2/27.

So the conversation moves the residual under the gate, and nothing in a lesson ever saw a conversation.

What this changes

1. quiet() moves to pocket_quiet.zig, unchanged (6921253a). pocket.zig is at 593 lines. This is the same
patch as the first commit of the refit PR (#876), with the same patch-id, so either can land first.

2. Facts are learned and checked in conversation (cc7c89f0).

• The conversations (server/slide_context.zig, server/slide_talks.zig). A lesson also asks its questions after
  four conversations:

  • three turns of small talk;
  • a 3-turn JavaScript task;
  • a 6-turn bash and git task (about 2.8k tokens on Nemotron);
  • a chain: an opener ("How many legs does a spider have?", then "What did I eat for breakfast today?"), then the
    latest two learned facts' questions with the answers their lessons wrote.

  The assistant turns are the model's own greedy replies, written once and kept in the graph file (talks, and a
  fact's chain pair), so every later lesson and a restarted server read the same text. The scripted lines are
  generic, and none of them is a probe below.

• What a lesson learns there. Per fact and per conversation:

  • six of its questions, fitted to fire;
  • four of its near misses, and two questions about the user that no fact answers, fitted to stay silent;
  • up to three of its answers, learned there too, when the whole answer fits the trainer's pass.

• The checks. Each fact's two held-out questions are also asked after two conversations that no lesson learns in:
  a Docker task, and a chain's opening that ends on a question the model cannot answer. A fact that only comes back
  cold does not pass.

• Reading long prompts (families/nemotron/pocket_read.zig, cuda_train_tap.zig). An example now carries
  Example.context: the rows of its conversation, which every question asked after it shares. The pocket reads such an
  example through an optional trainer hook, tapRows.

  • Nemotron's CUDA hook runs the prompt through the engine's prompt chunks (cuda_forward.zig's chunkPre,
    chunkMixer and chunkPost, the states carried from chunk to chunk as serving's prefill carries them), with the
    keys pass's MoE (cuda_train_fwd.experts, the same pocket kernel). It copies the MoE input at the asked rows as
    the keys pass does, so any length works.
  • It uses a sequence of its own. Each conversation runs once. Its Mamba states are kept, and its KV rows stay in
    place, for the next question after it.
  • Every eighth row of a conversation is read as a row to stay silent on.
  • TF_SLIDE_TAP_CHECK=1 compares each tapped prompt with a fresh run (nothing kept, cut in 1,024-row pieces instead
    of at the conversation's end), and with the trainer's own keys pass when one holds it.

• Learners without the hook. A learner without tapRows keeps the conversation prompts its passes hold and leaves
  out the rest (lesson_examples.readable), logging how many. That is Metal today: the chains fit in its 256 rows, the
  task conversations do not. Answers in a conversation are always read by passes, so a learner leaves out the ones
  longer than its passes, on CUDA too.

• The pocket's log line (layer N at the reply fires on ...) adds the fact's own questions in a conversation and
  the most the neuron reads on other prompts there.

3. An earlier fact's refit holds the conversation rows it fires on (4a63c4bb). A fact learned after another is
also asked after a chain that holds the earlier fact's own question and answer, and the earlier fact's neurons fire
there.

• quiet() read those rows as rows to push below zero. It could not, and so it undid the refit.
• On Flash Next, that was 7 of 7 refits in the first run: the new fact was learned with the earlier neurons still firing
  on its questions.
• Now a conversation's rows where the earlier neuron fires are held as they read (.none), beside the rows its own
  lesson stored.
• A test builds the case. Before the change the refit is undone (expected 1, found 0). After it, the neuron is silent
  on the new fact's questions and reads its own question in the chain as before.
• cuda.zig and tensorfold.zig import pocket_quiet.zig in their tests, as Living Weights: a quiet refit holds an earlier fact's neuron on its own questions as read now #876 does. So this test and the pocket's
  earlier tests run in the Linux zig build test.

Receipt (Nemotron)

• Environment: lw-ctx-main on main c7137bb (v1.0.11 + 19), Zig 0.17.0; NVIDIA GB10 (DGX Spark, sm_121), driver
  580.178.04, CUDA 13.0.88, Ubuntu 24.04.5; checkpoint TensorFold/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit
  revision d9d758fb. Each run used a fresh cp -RL copy:
  tensorfold-native serve <copy> --no-thinking --temperature 0 --slide --slide-graph <graph.json>.
• The runs:
  • main and this branch each taught the ten facts of 1.0.8's receipt ("I like blue." to "I am allergic to
    peanuts."), one learn request each.
  • After every lesson, I asked the two fixed wordings per fact and every learned fact's stored held-out questions.
  • Before the first lesson and after the last, I ran a context matrix over the 20 fixed wordings, greedy, 120 tokens:
    • cold;
    • after a system prompt;
    • after 3 small-talk turns;
    • after an 8-turn unrelated task chat (4,171-4,232 prompt tokens with the question);
    • 4 chains of 5 questions.
  • The small talk and the task chat are fixed user lines (not the lesson's). The assistant turns are generated live
    and greedy by each build's model.
  • A reply counts when it names the fact's value and does not open with a refusal. I read every reply; the counts are
    the same as plain keyword hits, except for one main reply that names sushi as an example after "I don't know".
  • Before teaching, both builds gave the same 100 matrix replies and 12 unrelated replies byte for byte, and 0 hits.
    main was built at c7137bb. The branch binary was built at a586a277, which differs from the head 4a63c4b by one
    comment's wording.

context	main, 10 facts taught	this branch	main, the 6 facts both kept	branch, the same 6
cold	13/20	11/20	11/12	11/12
system prompt	14/20	12/20	12/12	12/12
3 turns of small talk	14/20	12/20	12/12	12/12
~4k-token task chat	6/20	10/20	6/12	10/12
chained	14/20	12/20	12/12	12/12

• Of the wordings recalled cold, after the task chat: main 6/13, this branch 9/11.
• Per fact, after the task chat:
  • sushi 0/2 → 2/2;
  • Leo 1/2 → 2/2 (main says "Your son is called Hadi");
  • 14 March 0/2 → 1/2.
  • On main, "When is my birthday?" and "Where do I live?" after the chat answer "My name is Nemotron. I am created
    by NVIDIA researchers."
• Facts kept. main kept 7 of 10, this branch 6.
  • Both took out Ginger ("it kept answering questions about something else"), and both missed nurse and Volvo.
  • The branch also missed Bristol. Its lesson failed the same check for 5 rounds: the held-out "Do you know which city
    is my home?" asked cold with thinking on ("I remember you mentioned that you know which city is your home, but I
    don't have that information stored"). No conversation check failed it.
  • That costs the branch Bristol's 2 wordings in every context. The fixed questions after lesson 10 are 11/20 on the
    branch against 13/20 on main, and 13/20 is also the A2 receipt's main.
• Chains. On Nemotron chains were already fine on main. Every chained miss on both builds is a fact that was not
  kept.
• In-conversation checks. Over the 10 lessons they failed 14 times. 10 were Volvo's yes/no held-outs ("Is my car a
  Volvo?" → "Yes, you do.", which names no value). 3 were nurse ("your work is running a small Go web server in
  Docker"). 1 was Ana in round 1, recalled in round 2.
• Unrelated prompts (11 general ones and "What is my cat called?"): the branch 12/12 identical to before teaching;
  main 11/12 (a reworded haiku).
• Restarts: plain and --slide, on both builds, gave the 20 cold replies and 12 unrelated replies identical to
  after lesson 10. The base checkpoint's sha256 is unchanged.
• Lesson time: 1,628 s for the 10 lessons against 1,502 s on main (+8%).
  • The first lesson writes the conversations once: +26.5 s.
  • A round-1 read takes 9.9 s against 5.9 s (the taps: about 110 reads of up to 2.9k tokens, against 37 passes).
  • A round's checks take 8.3 s against 7.3 s.
• Taps equal the trainer's pass, bit for bit. A 2-lesson run with TF_SLIDE_TAP_CHECK=1 read 102 prompts in a
  conversation (2,180 tapped rows, prompts up to 2,855 tokens).
  • Every tapped value at every pocket layer equalled a fresh run's.
  • The 30 prompts a keys pass holds (the chains, up to 205 tokens, conversation cut at 146-271) equalled the pass's
    own values: 0 values differ, most 0.0000.
  • So kept conversation states, cuts at any row, and the engine's chunks against the trainer's pass give the same
    bits.
• The refit fix (commit 3) was not exercised here. In the branch run, 13 refits were kept and 2 undone, and none
  met a conversation row its neuron fires on (0 conversation rows held). Its test covers it, and Flash Next below
  needed it.
• Tests: zig build test on Linux:
  • main 743/748 (5 skipped); commit 1 743/748; commit 2 746/751; commit 3 750/755. Commit 3's four more are its own
    test and the pocket's three that now run on Linux.
  • zig build test-server-cpu: 209/209 (main 207; +2: the conversations persist across a reopen of the graph, and
    the learned and checked conversations differ).
  • Every commit builds and passes. zig fmt --check build.zig zig is clean. python3 tools/zig/lean_check.py gives
    527 problems against 527 on main (0 new).
• Metal: not run on a Mac (I have none). zig build native and zig build test-native with -Dtarget=aarch64-macos
  and a stub xcrun compile every Zig module and stop only at linking Apple's frameworks, as on main. Metal's
  trainer has no tapRows, so it learns the chains, leaves out the task conversations, and logs how many it left out.
  It runs the same checks, so it checks in conversation. A Mac lesson is still needed.

Supporting evidence: the same fix on two other models

The fix was developed on Qwen3.8 Flash Next and carried to Qwen3.8-27B in our CUDA ports. Neither is in this repo yet,
so these numbers support the design and are not this PR's receipt. Each port has its own tapRows, through its own
engine's batched pass. On both, TF_SLIDE_TAP_CHECK found 0 values different from the trainer's pass: Flash Next on 48
prompts, and the 27B on a 128-row conversation, run and found kept.

• Flash Next, the same 7 facts before and after:

  context	before	after
  task chat ~4k	7/16	11/16
  chained	7/14	10/14
  small talk	12/16	15/16
  thinking on, after the task chat	4/7	7/7

  • Of the wordings recalled cold, after the task chat: 8/14 → 11/13.
  • Gate after the 4k chat, recalled probes: median u x +2.44 → +3.68 (cold +3.77).
  • Lesson time for 8 facts: 5,351 → 5,410 s.
  • Before the refit fix, the first run's earlier-fact refits all failed (7 of 7). With it, 72 succeeded and 56 failed,
    against 116 and 199 in the study before this work.

• Qwen3.8-27B, with the fix: cold 15/18, task chat ~4k 15/18, chained 14/16. Of the wordings recalled cold:

  • after the task chat 14/15;
  • chained 14/14.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off. (Serving is
  untouched. The taps run inside a lesson, on the trainer's own sequence, as its passes do.)
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Not
  touched. Both builds gave identical replies before teaching.)
• No precision traded for speed. If the bits change, the description says which sums change and why. (The tapped
  rows equal the trainer's pass's bits.)
• Prompt processing is no slower than the last release. (Not touched.)
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not. (GB10; Metal
  type-checked only.)
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

Not in this PR

• Asking earlier facts again in conversation. main has no re-check of earlier facts after a lesson. When the Living Weights: a later lesson's quiet refit can silence an earlier fact on its own questions, and nothing asks it again #870
  fix adds one, it should ask each fact's conversation wordings too. In our build that was one line in the re-check
  (Asking.before from the wording's conversation). The graph would also need to keep each wording's conversation
  name, which this PR leaves out (questions keeps the cold wordings only).
• The take-out restore and checks on the rounded pocket (our 0ffc5167, in Living Weights: every earlier fact asked again after a lesson, and a take-out that puts the pocket back bit for bit #877). This PR does not need it, and I
  expect your Living Weights: a later lesson's quiet refit can silence an earlier fact on its own questions, and nothing asks it again #870 fix to cover that ground.
• A Metal tapRows. Metal reads the chains and leaves the long conversations out, as above.
• Gate readings on Nemotron after teaching. The u x table above is from Flash Next's diagnostic tool, which I did
  not port. Nemotron's lessons log only the fit-time reading. On the branch, in all 60 of those lines (every lesson,
  every reply neuron), the fact's neurons fire on all 27 of its questions in a conversation, and other prompts there
  read at most -0.20.
• Thinking on in a conversation, and document facts in a conversation. Neither is learned or checked.

How it relates to #878 and your #870 work

• Living Weights as a core module: the learner, pocket and fits in zig/src/living, a family described as data (Nemotron only) #878 (draft) moves the learner and pocket from families/nemotron to zig/src/living. This PR changes
  learner.zig, lesson_examples.zig, pocket.zig, pocket_read.zig and pocket_quiet.zig where they are on
  main. Whichever lands second rebases by path. tapRows is one more optional trainer hook beside Living Weights as a core module: the learner, pocket and fits in zig/src/living, a family described as data (Nemotron only) #878's list, and
  cuda_train_tap.zig stays Nemotron's.
• Living Weights: a later lesson's quiet refit can silence an earlier fact on its own questions, and nothing asks it again #870 / Living Weights: a quiet refit holds an earlier fact's neuron on its own questions as read now #876. The first commit here is Living Weights: a quiet refit holds an earlier fact's neuron on its own questions as read now #876's first commit, the same patch. Commit 3 holds conversation rows in
  quiet(). If your Living Weights: a later lesson's quiet refit can silence an earlier fact on its own questions, and nothing asks it again #870 fix rewrites quiet() (Living Weights: a quiet refit holds an earlier fact's neuron on its own questions as read now #876 holds an earlier fact's own rows as read now), the conversation
  hold sits beside that in the same place: one more condition on which rows are held.
• If you would rather take the server half and the hook separately, commit 2 is self-contained. Without commit 3, the
  chain's rows only make earlier refits fail safe (undone), as described above.

[ashhart]

Thank you, @ZackSample, for testing taught facts after conversations as well as cold questions, and for showing the retention tradeoff.

This work moves with Living Weights to 1.1.1, the next patch release. The reviewed integration stores conversation names with answers, deduplicates shared tap rows and keeps thinking checks cold.

Before release, the combined learner needs both four- and ten-fact runs, unrelated-reply comparisons, lesson take-out and save/reopen checks on CUDA and Metal. The current results do not justify treating conversation recall as improved on every backend.

Posted by Ash's AI Assistant Synthena

[ashhart]

Thank you, @ZackSample, for testing taught facts after conversations as well as cold questions, and for showing the retention tradeoff.

This work moves with Living Weights to the next patch release that passes its gates, 1.1.1 if they pass in time. The reviewed integration stores conversation names with answers, deduplicates shared tap rows and keeps thinking checks cold.

Before release, the combined learner needs both four- and ten-fact runs, unrelated-reply comparisons, lesson take-out and save/reopen checks on CUDA and Metal. The current results do not justify treating conversation recall as improved on every backend.

Nemotron on Apple M3 Ultra kept 3 of 4 facts and Apple M5 Ultra kept 2 of 4, both failing at the same rounding check, so Living Weights stays held for a repaired 4/4 rerun plus the NVIDIA GB10 27B and persistence/retention gates.

Posted by Ash's AI Assistant Synthena