Repository navigation
Living Weights: a quiet refit holds an earlier fact's neuron on its own questions as read now - #876
Conversation
pocket.zig is at 593 lines and the refit fix adds to quiet(), so quiet() and its two thresholds move to pocket_quiet.zig as they are, beside pocket_joint.zig and pocket_values.zig. No change in what it does.
…'s rows as read now, not only as stored quiet() refits an earlier fact's neuron so that it stays silent on a new fact's questions, and holds the rows its own lesson stored fixed. Those rows were read before any pocket neuron sat below its layer. The same questions read today carry the lower neurons' push, and when they are among the new lesson's other prompts (a fact's question among the steady prompts, an answer refresh() read again, a document subject's replayed answers) the refit labelled them rows to push below zero. A kept refit then silenced the neuron on its own fact's questions while its check, which reads the stored rows only, still passed (ashhart#870). Every other prompt's row where the earlier neuron fires now is held fixed beside the stored ones, so a refit changes it only on the new fact's rows; if those fall inside the held span the refit fails its check and is undone, as before. The log line says how many rows were held and how many of them were read now. A test builds the case: before this change the refit is kept and moves all 12 of the earlier fact's rows as read now, after it none move. The Nemotron CUDA module's host tests now import pocket_quiet.zig, so this test and the pocket's earlier ones also run on Linux.
|
Thank you, @ZackSample. Your receipt shows the bug and the fix cleanly. With the reply key, Bristol's lesson took Ginger's held out answers from 3 of 3 to 0 of 3 on main, and with #876 all three still come back. Your test runs #876 and #877 land right after 1.0.12. They will be reconciled with #872's graph format, which ships in 1.0.12 first. One question before then. We served the same refit design on Nemotron and saw what it costs. Over ten facts on an M5 Ultra it kept a fact the old refit lost, but replies to unrelated prompts changed more often, 6 of 12 against 3 of 12. Over four facts on an M3 Ultra it was 5 of 12 against 3 of 12. Our guess is that the held rows include weak firings on unrelated prompts, which the old refit pushed below zero. Would holding only the rows where the neuron fires above Posted by Ash's AI Assistant Synthena |
|
Thanks, @ashhart. Yes: holding only the rows where the earlier neuron fires above The Ginger repro (
The ten facts in the default keying, with the receipt's 20 fixed questions, and 12 unrelated prompts (the ones in
The two drift columns are not the same size: our #876 run asked only 5 of the 12 unrelated prompts, so its 2 of 5 and #876's own test still passes with the threshold: in it the earlier fact's rows read now fire well above 0.5, so the Why I agree. We saw the same mechanism grow on a Qwen3.8-27B port (dense MLP pockets, the same learner) over 21 One related note: #891's refit ( |
|
I validated the agreed The additional fixture calls
Environment: M5 Max, 128 GiB, macOS 26.6.2 (25G83), Zig 0.17.0. Native binary SHA256: Astra also reviewed the example-index selection, fixed span and rollback without a remaining source blocker. These are host tests and a compile receipt. Recall and unrelated-answer drift still need paired model runs. This branch contains #876 plus the threshold; #877's graph reconciliation remains with the maintainer. |
|
Thank you, @ZackSample, the threshold boundary controls and paired fact results make the refit tradeoff explicit. We will keep refit_above in the integrated learner and carry the strict-boundary regression with its authorship. Rounded-state retention and whole-pocket rollback come together; #891's conversational judgment is enabled where CUDA supplies taps, while Metal keeps its earlier acceptance. The threshold and platform choice are settled. We will now run the integrated before/after/restart and retained-fact gates, including the Mac arm, with each workload's recall/drift results kept distinct. Your unchanged PR and the offered host patch remain the sources for that composition. Posted by Ash's AI Assistant Synthena |
|
Thank you, @ZackSample, for holding earlier neurons on their own questions as later lessons change the model's activations. This work moves with Living Weights to 1.1.1, the next patch release. The reviewed threshold keeps protection for activations above 0.5, with weak- and strong-activation controls. Before it ships, the combined learner needs the short and long lesson runs, unrelated-reply comparisons and persistence checks across CUDA and Metal. We will keep the retention-versus-drift results visible rather than treating the smaller workload as a universal improvement. Posted by Ash's AI Assistant Synthena |
…ires above refit_above (#870) #876 holds every row outside the new fact's own where an earlier neuron fires at all, so its weak firings on unrelated prompts stay as they read, where the refit before it pushed them below zero. That kept the earlier facts but doubled the replies to unrelated prompts that changed in #870's runs (5 of 12 against 3 of 12 over four facts). A refit now holds only the rows where the neuron fires above refit_above (0.5), the level that makes a refit due: the earlier fact's own questions as read today fire there, and its weaker firings are quieted as before. Test: prompts leaning the earlier fact's way; where it fired above 0.5 it reads as before, where it fired weakly it is at zero or below. Co-authored-by: ZackSample <134018356+ZackSample@users.noreply.github.com>
@ZackSample's #878 (head 75978a1): the learner, pocket and fits move to zig/src/living as a module, then a family is described as data (layout, neuron kind, shard format), Nemotron the one registered. Reconciled with what landed after its base: - #876 and the refit_above threshold: living/pocket_quiet.zig carries both (the extraction held the refit from before #876); - #872 and #877: the delete (pocket_forget.zig), the lesson's mark (pocket_mark.zig) and their tests move into living; their snapshots and put-backs hold a SwiGLU pocket's gate rows; the round a save writes is shards.roundSlot over living's affine (the codes affine4_host writes, tested), so the hosts' pocketRound hooks go; - the delete's tests now run in the living module, on Linux hosts too; - the HIP, XPU and fuzz builds' servers get the living module (server.zig), which the server's model view now imports; - 1.0.12's json module, heat, radio and GLM build wiring kept beside living's.
…sked cold @ZackSample's #891 (head 4a63c4b): a lesson also learns a fact's questions after scripted conversations (talk, task, long, and a chain of the latest learned facts' questions) and checks its two held-out questions after two conversations no lesson learns in. The model writes each conversation once; the fact graph keeps it. On CUDA a prompt in a conversation is read at any length through the trainer's taps; Metal leaves out what its passes cannot hold. Reconciled with what landed after its base: - the quiet refit holds both #876's rows (other prompts') and #891's (inside a conversation, where a chain carries an earlier fact's question and answer), above refit_above; - #877's rounds, recheck, repair and take-out kept; the conversation checks join each round's verdict; - #872's graph format kept, talks and chain beside it; checks asked in conversation stay out of its answer records here (the next commits give them their conversation); - after #878's move: the tap path in living/pocket_read.zig, the example checks in lesson_examples.readable, cuda_train_tap.zig's lesson_route from living; - askIn takes `shipped`, so a delete's checks can ask the model as shipped after a conversation; json as a module.
…ollow-ups named as ours After review of the fourteen lines that overstated: #710's fresh-quantization gain is one measured case; #902's stop and group cleanup hold where the watch lines are open and through the shared helper; #887 says its chunks change the attention bits below 8,192, on M5 too, with no larger error against float64; #899 records the ring start and the wider refusal; #802 keeps the author's B70 receipts apart from our host-checked lifetime fixes; #836's equalities are the GB10's and its Windows build the author's; #888 says a failed restore's pinned memory is kept, never freed; #734's tree rounds are checked against the host's, not float64; the Living Weights lines (#876, #877, #878, #891) say which runs their numbers come from, the ten-fact costs, and the M3 Ultra's four-fact result; #946 and #910 read plainer.
… in conversation (#876, #877, #878, #891) This reverts the merge bafca587d (its first parent kept): the learner moves back from zig/src/living to the Nemotron family as 1.0.12 ships it, and the server, the build and the families lose the shared module. 1.1.1 takes the series again. Kept from the commits that came after it: the lean moves (shard_edit_link_test.zig, the kernel list in cuda_kernels.zig), the KV pages, which now read the lesson route from the Nemotron family, the required-tools lanes import in the server build, and Drama's and Qwen-VL's build wiring.
…Next on CUDA's lines say what was measured The six Living Weights rows (#876, #877, #878, #891, #905 and #992) leave Unreleased with the series. The EXL3 operators' line names GLM-5.3-Flash's two-rank engine as the family that serves them; #995's and #979's launches have their records on nvcc 13.0 and 13.2; and Flash Next on CUDA states what a GB10 served from the MLX 4-bit checkpoint, text and PNG images.
|
Thank you, @ZackSample, for holding earlier neurons on their own questions as later lessons change the model's activations. This work moves with Living Weights to the next patch release that passes its gates, 1.1.1 if they pass in time. The reviewed threshold keeps protection for activations above 0.5, with weak- and strong-activation controls. Before it ships, the combined learner needs the short and long lesson runs, unrelated-reply comparisons and persistence checks across CUDA and Metal. We will keep the retention-versus-drift results visible rather than treating the smaller workload as a universal improvement. Nemotron on Apple M3 Ultra kept 3 of 4 facts and Apple M5 Ultra kept 2 of 4, both failing at the same rounding check, so Living Weights stays held for a repaired 4/4 rerun plus the NVIDIA GB10 27B and persistence/retention gates. Posted by Ash's AI Assistant Synthena |
…act's rows as read now @ZackSample's ashhart#876 (head 3723ab5): pocket_quiet.zig takes the quiet refit out of pocket.zig unchanged, then holds every row an earlier neuron fires on above zero at its current value, not only its stored rows. Conflict: pocket.zig's imports keep ashhart#872's pocket_forget.zig beside the new pocket_quiet.zig.
…ires above refit_above (ashhart#870) ashhart#876 holds every row outside the new fact's own where an earlier neuron fires at all, so its weak firings on unrelated prompts stay as they read, where the refit before it pushed them below zero. That kept the earlier facts but doubled the replies to unrelated prompts that changed in ashhart#870's runs (5 of 12 against 3 of 12 over four facts). A refit now holds only the rows where the neuron fires above refit_above (0.5), the level that makes a refit due: the earlier fact's own questions as read today fire there, and its weaker firings are quieted as before. Test: prompts leaning the earlier fact's way; where it fired above 0.5 it reads as before, where it fired weakly it is at zero or below. Co-authored-by: ZackSample <134018356+ZackSample@users.noreply.github.com>
… data @ZackSample's ashhart#878 (head 0bf1178): the learner, pocket and fits move to zig/src/living as a module, then a family is described as data (layout, neuron kind, shard format), Nemotron the one registered. Reconciled with what landed after its base: - ashhart#876 and the refit_above threshold: living/pocket_quiet.zig carries both (the extraction held the refit from before ashhart#876); - ashhart#872 and ashhart#877: the delete (pocket_forget.zig), the lesson's mark (pocket_mark.zig) and their tests move into living; their snapshots and put-backs hold a SwiGLU pocket's gate rows; the round a save writes is shards.roundSlot over living's affine (the codes affine4_host writes, tested), so the hosts' pocketRound hooks go; - the delete's tests now run in the living module, on Linux hosts too; - the HIP, XPU and fuzz builds' servers get the living module (server.zig), which the server's model view now imports; - 1.0.12's json module, heat, radio and GLM build wiring kept beside living's.
… only asked cold @ZackSample's ashhart#891 (head a203094): a lesson also learns a fact's questions after scripted conversations (talk, task, long, and a chain of the latest learned facts' questions) and checks its two held-out questions after two conversations no lesson learns in. The model writes each conversation once; the fact graph keeps it. On CUDA a prompt in a conversation is read at any length through the trainer's taps; Metal leaves out what its passes cannot hold. Reconciled with what landed after its base: - the quiet refit holds both ashhart#876's rows (other prompts') and ashhart#891's (inside a conversation, where a chain carries an earlier fact's question and answer), above refit_above; - ashhart#877's rounds, recheck, repair and take-out kept; the conversation checks join each round's verdict; - ashhart#872's graph format kept, talks and chain beside it; checks asked in conversation stay out of its answer records here (the next commits give them their conversation); - after ashhart#878's move: the tap path in living/pocket_read.zig, the example checks in lesson_examples.readable, cuda_train_tap.zig's lesson_route from living; - askIn takes `shipped`, so a delete's checks can ask the model as shipped after a conversation; json as a module.
…rt#877, ashhart#878, ashhart#891) One line each, with the GB10 measurements; the M3 Ultra numbers follow its gate.
…ollow-ups named as ours After review of the fourteen lines that overstated: ashhart#710's fresh-quantization gain is one measured case; ashhart#902's stop and group cleanup hold where the watch lines are open and through the shared helper; ashhart#887 says its chunks change the attention bits below 8,192, on M5 too, with no larger error against float64; ashhart#899 records the ring start and the wider refusal; ashhart#802 keeps the author's B70 receipts apart from our host-checked lifetime fixes; ashhart#836's equalities are the GB10's and its Windows build the author's; ashhart#888 says a failed restore's pinned memory is kept, never freed; ashhart#734's tree rounds are checked against the host's, not float64; the Living Weights lines (ashhart#876, ashhart#877, ashhart#878, ashhart#891) say which runs their numbers come from, the ten-fact costs, and the M3 Ultra's four-fact result; ashhart#946 and ashhart#910 read plainer.
… in conversation (ashhart#876, ashhart#877, ashhart#878, ashhart#891) This reverts the merge bafca587d (its first parent kept): the learner moves back from zig/src/living to the Nemotron family as 1.0.12 ships it, and the server, the build and the families lose the shared module. 1.1.1 takes the series again. Kept from the commits that came after it: the lean moves (shard_edit_link_test.zig, the kernel list in cuda_kernels.zig), the KV pages, which now read the lesson route from the Nemotron family, the required-tools lanes import in the server build, and Drama's and Qwen-VL's build wiring.
…Next on CUDA's lines say what was measured The six Living Weights rows (ashhart#876, ashhart#877, ashhart#878, ashhart#891, ashhart#905 and ashhart#992) leave Unreleased with the series. The EXL3 operators' line names GLM-5.3-Flash's two-rank engine as the family that serves them; ashhart#995's and ashhart#979's launches have their records on nvcc 13.0 and 13.2; and Flash Next on CUDA states what a GB10 served from the MLX 4-bit checkpoint, text and PNG images.
Fixes part of #870 (the refit). The other part, asking earlier facts again after a lesson, is #877.
What this changes
When a new fact is taught,
quiet()refits each earlier neuron that fires on the new fact's questions, so that it stayssilent there. The refit holds the earlier fact's stored rows fixed: its key rows as its own lesson read them. But the
same questions read today carry the push of every pocket neuron below, so they are different rows. When the new
lesson reads them among its other prompts (an earlier fact's question among the steady prompts, an answer
refresh()read again, a document subject's replayed answers), they are labelled
.steadyand pushed below the margin. The refitcan then be kept while it silences the neuron on its own fact's questions, and its check, which reads only the stored
rows, still passes.
Now every row outside the new fact's own examples where the earlier neuron fires as read now (
u x > 0) is held fixedbeside the stored ones, labelled
.none. Adam's step is projected off their span, as it already is for the storedrows (
pocket_fit.zig), so they read exactly as before. A refit therefore changes an earlier neuron only on the newfact's rows. If those fall inside the held span, the refit fails its check and is undone, as before. The log line adds
(N rows held as they read, M of them read now).Commits:
quiet()and its two thresholds move unchanged tofamilies/nemotron/pocket_quiet.zig, sincepocket.zigis at 593lines.
plus a new fact whose rows lean the same way, and runs
quiet()on them. Before the fix, the refit is kept and movesall 12 of the earlier fact's rows as read now (
expected 0, found 12). After it, none move, and the new fact's rowsare at or below zero. The Nemotron CUDA module's host tests now import
pocket_quiet.zig, so this test and thepocket's three earlier tests (
pocket.zig,pocket_fit.zig) also run in the Linuxzig build test. Until now onlythe Metal test root reached them.
Receipt
Environment:
lw-a1-refit2c2f4c6 onmainc7137bb (v1.0.11 + 19), Zig 0.17.0; NVIDIA GB10 (DGX Spark, sm_121),driver 580.178.04, CUDA 13.0.88, Ubuntu 24.04.5; checkpoint
TensorFold/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bitrevision d9d758fb, served from a freshcp -RLcopyfor each run:
tensorfold-native serve <copy> --no-thinking --temperature 0 --slide --slide-graph <graph.json>.The setup for both Nemotron receipts below, before (
main) and after (this branch):from the graph, at
"temperature": 0and 60 tokens. A reply counts when it names the fact's value.--slideand one with it.The bug on
main, and the fix. The first five facts withTF_SLIDE_POCKET_KEY=reply, the experiment flag thatputs every fact's neurons at the reply's first row, as Flash Next's facts were in Living Weights: a later lesson's quiet refit can silence an earlier fact on its own questions, and nothing asks it again #870:
main: "My dog is called Ginger." (fixed, held-out)"recalled": true, saved)On
main, lesson 4 refitted 5 of Ginger's 10 neurons, and each refit passed its check on Ginger's stored rows:The lesson ended
{"id": 4, "recalled": true, "steps": 2}and was saved. Before it, Ginger's held-out questions allanswered "Your dog's name is Ginger." After it:
On this branch, the same lesson refitted 6 of Ginger's neurons, each holding the rows it fires on as read now:
(130 rows held as they read, 21 of them read now)at layer 24, then 6, 16, 25, 10 and 10 at layers 36, 34, 31, 45and 43. Two more were left as they were. All three held-out questions still answered "Your dog's name is Ginger."
The shards were identical to
main's after lessons 1 and 2, before any refit. On both builds, a plain restart and a--sliderestart gave the same replies as after lesson 5.The ten facts with the default keying (no flag; 20 fixed questions):
main--slideThe refit log lines where the two differ:
main, two of Bristol's neurons fired on the nurse questions. Layer 22, neuron 2 was refitted. Layer17, neuron 1 was left as it was:
fires on this fact's questions (up to 0.84); fitted again its own would read at least 0.95 and other rows at most 10.53, so it stays as it was. With that neuron still firing on its questions,the nurse lesson never came back.
(165 rows held as they read, 8 of them read now)and(171 rows held as they read, 14 of them read now), and the second one's other rows reached at most -0.25. So the rowsmaincould notpush below zero were among the 14 rows where Bristol's neuron fires as read now, which this branch holds as they
read.
main, blue's layer 22, neuron 0 was left as it was (other rows at most 3.21). On this branch itwas refitted with 8 rows read now held.
With the default keying no kept fact was lost on either build. The same mechanism shows from the other side:
main, a refit that must silence an earlier fact's questions as read now either is kept, which loses the fact(above), or fails and leaves the earlier neuron firing on the new fact's questions, which here cost the nurse fact.
Fixed wordings (the first ten for the five-fact run): "What colour do I like?", "Which colour is my favourite?",
"What is my sister called?", "What's my sister's name?", "What is my dog called?", "What's my dog's name?", "Where
do I live?", "Which city do I live in?",
"When is my birthday?", "What date is my birthday?", "What do I do for work?", "What is my job?", "What is my
favourite food?", "Which food do I like most?", "What car do I drive?", "What kind of car do I have?", "What is my
son called?", "What's my son's name?", "What am I allergic to?", "Do I have any food allergies?".
Exactness, decode and prompt speed: n/a. Nothing in the forward or in serving changes. A lesson in which no earlier
neuron is refitted writes the same shards, byte for byte (receipt above).
Tests:
zig build teston Linux: 747/752 pass, 5 skipped (main: 743/748, 5 skipped; the 4 more are the new testand the 3 pocket tests now reached on Linux). Both commits build and pass.
zig fmt --check build.zig zigis clean,and
python3 tools/zig/lean_check.pygives 527 problems against 527 onmain(0 new).Not run: Metal (no Mac here).
zig build test-native -Dtarget=aarch64-macoswith a stubxcruncompiles and stopsonly at linking Apple's frameworks, as on
main. The change is host-side pocket code, shared by both backends.Checklist
change.)
"draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Nottouched.)
refit's fixed rows change.)
type-checked only.)