Repository navigation
Living Weights: every earlier fact asked again after a lesson, and a take-out that puts the pocket back bit for bit - #877
Conversation
pocket.zig is at 593 lines and the refit fix adds to quiet(), so quiet() and its two thresholds move to pocket_quiet.zig as they are, beside pocket_joint.zig and pocket_values.zig. No change in what it does.
…'s rows as read now, not only as stored quiet() refits an earlier fact's neuron so that it stays silent on a new fact's questions, and holds the rows its own lesson stored fixed. Those rows were read before any pocket neuron sat below its layer. The same questions read today carry the lower neurons' push, and when they are among the new lesson's other prompts (a fact's question among the steady prompts, an answer refresh() read again, a document subject's replayed answers) the refit labelled them rows to push below zero. A kept refit then silenced the neuron on its own fact's questions while its check, which reads the stored rows only, still passed (ashhart#870). Every other prompt's row where the earlier neuron fires now is held fixed beside the stored ones, so a refit changes it only on the new fact's rows; if those fall inside the held span the refit fails its check and is undone, as before. The log line says how many rows were held and how many of them were read now. A test builds the case: before this change the refit is kept and moves all 12 of the earlier fact's rows as read now, after it none move. The Nemotron CUDA module's host tests now import pocket_quiet.zig, so this test and the pocket's earlier ones also run on Linux.
slide_graph.zig is at 552 lines and the next commit adds the wordings' answers to it, so its three tests move to slide_graph_test.zig as they are.
… it made forget is repaired, else it is taken out A lesson's checks asked only the fact just taught, so a lesson that made the model forget an earlier fact was kept and saved (ashhart#870). Once a lesson's fact comes back, every fact learned before is asked again on its stored held-out wordings, side by side (slide_recheck.zig), and judged as its own lesson judged it: its short answer or answer words, now kept in the graph beside its questions. A fact from an older graph is judged by its words, and its first good reply is kept as its answer. A fact that came back when the lesson began and does not now is repaired: the lesson is fitted again (a reroute) with that fact's held-out answers among the answers it must leave as they were, then checked again. After two repairs, or if a repair is never checked, the lesson is taken out whole and the earlier facts are asked once more, before anything is saved. Facts learned before the server started are asked once at the first learn request, so only a fact the lesson broke counts. The stream reports each earlier fact (event recheck: fact, text, recalled, was), a summary with its seconds (rechecked), and any repair (repair).
…checks read it as a save writes it A lesson now marks the pocket as it finds it (every neuron's weights, the neurons in use, what was learned so far), and taking it out (a drop, an undo, a repair's reroute, an unrecalled lesson settled) puts exactly that back and hands the changed layers to the forward again, logging the pocket's hash before and after. The old undo zeroed the lesson's own neurons and restored the earlier ones it refitted, but nothing else the lesson changed. A save rounds every layer's neurons to the shards' 4-bit groups, and each output row's group spans all 64 neurons of a layer's pocket, whatever fact they hold, so a new fact's neurons coarsen an earlier fact's outputs at the layers they share. Nothing checked that: a later lesson's recheck could find the fact forgotten, blame that lesson, and a take-out could not bring back what the save had changed. Now a lesson's layers are rounded as the shards hold them (pocket_shards round(), which gives what write() hands back, bit for bit) before its held-out answers are judged, so every check, the lesson's own and the earlier facts' recheck, reads what a save writes, and a save of a checked pocket changes nothing. After a take-out the facts learned before are asked again, and the server logs, and streams as "restored", how many of the facts it made forget came back. TF_SLIDE_TAKE_OUT_CHECK takes every kept lesson out again, to check that the pocket and the earlier facts come back as they were. A lesson that is not kept also puts back the earlier facts' verdicts it began with, unless they were asked again after its take-out. Its re-checks may have found a fact forgotten before a repair or a later check took it out, and the next lesson must still count that fact as one that came back.
|
Thank you, @ZackSample. #877 does more than we had planned for #870. A lesson taken out puts the pocket back bit for bit, and every check reads the pocket as a save rounds it. Your receipt shows both working. The pocket came back with the same bits all five times it was put back, and with #876 it kept 8 of 10 facts and 15 of 20 fixed questions after the tenth lesson, against main's 7 of 10 and 13 of 20. It lands right after 1.0.12, together with #876. #872 ships in 1.0.12 first, and it also keeps a fact's answers in the graph under Posted by Ash's AI Assistant Synthena |
|
Thank you, @ZackSample, for rechecking earlier facts and restoring the whole pocket when a lesson is taken out. This work moves with Living Weights to 1.1.1, the next patch release. We have kept one answer-record and rounding path alongside #872's deletion handling, including the check that an unfinished lesson cannot restore a fact deleted meanwhile. The combined build still needs lesson, repair, take-out, delete and save/reopen comparisons, with pocket bytes checked as well as recalled answers. Posted by Ash's AI Assistant Synthena |
@ZackSample's #878 (head 75978a1): the learner, pocket and fits move to zig/src/living as a module, then a family is described as data (layout, neuron kind, shard format), Nemotron the one registered. Reconciled with what landed after its base: - #876 and the refit_above threshold: living/pocket_quiet.zig carries both (the extraction held the refit from before #876); - #872 and #877: the delete (pocket_forget.zig), the lesson's mark (pocket_mark.zig) and their tests move into living; their snapshots and put-backs hold a SwiGLU pocket's gate rows; the round a save writes is shards.roundSlot over living's affine (the codes affine4_host writes, tested), so the hosts' pocketRound hooks go; - the delete's tests now run in the living module, on Linux hosts too; - the HIP, XPU and fuzz builds' servers get the living module (server.zig), which the server's model view now imports; - 1.0.12's json module, heat, radio and GLM build wiring kept beside living's.
…sked cold @ZackSample's #891 (head 4a63c4b): a lesson also learns a fact's questions after scripted conversations (talk, task, long, and a chain of the latest learned facts' questions) and checks its two held-out questions after two conversations no lesson learns in. The model writes each conversation once; the fact graph keeps it. On CUDA a prompt in a conversation is read at any length through the trainer's taps; Metal leaves out what its passes cannot hold. Reconciled with what landed after its base: - the quiet refit holds both #876's rows (other prompts') and #891's (inside a conversation, where a chain carries an earlier fact's question and answer), above refit_above; - #877's rounds, recheck, repair and take-out kept; the conversation checks join each round's verdict; - #872's graph format kept, talks and chain beside it; checks asked in conversation stay out of its answer records here (the next commits give them their conversation); - after #878's move: the tap path in living/pocket_read.zig, the example checks in lesson_examples.readable, cuda_train_tap.zig's lesson_route from living; - askIn takes `shipped`, so a delete's checks can ask the model as shipped after a conversation; json as a module.
…ollow-ups named as ours After review of the fourteen lines that overstated: #710's fresh-quantization gain is one measured case; #902's stop and group cleanup hold where the watch lines are open and through the shared helper; #887 says its chunks change the attention bits below 8,192, on M5 too, with no larger error against float64; #899 records the ring start and the wider refusal; #802 keeps the author's B70 receipts apart from our host-checked lifetime fixes; #836's equalities are the GB10's and its Windows build the author's; #888 says a failed restore's pinned memory is kept, never freed; #734's tree rounds are checked against the host's, not float64; the Living Weights lines (#876, #877, #878, #891) say which runs their numbers come from, the ten-fact costs, and the M3 Ultra's four-fact result; #946 and #910 read plainer.
… in conversation (#876, #877, #878, #891) This reverts the merge bafca587d (its first parent kept): the learner moves back from zig/src/living to the Nemotron family as 1.0.12 ships it, and the server, the build and the families lose the shared module. 1.1.1 takes the series again. Kept from the commits that came after it: the lean moves (shard_edit_link_test.zig, the kernel list in cuda_kernels.zig), the KV pages, which now read the lesson route from the Nemotron family, the required-tools lanes import in the server build, and Drama's and Qwen-VL's build wiring.
…Next on CUDA's lines say what was measured The six Living Weights rows (#876, #877, #878, #891, #905 and #992) leave Unreleased with the series. The EXL3 operators' line names GLM-5.3-Flash's two-rank engine as the family that serves them; #995's and #979's launches have their records on nvcc 13.0 and 13.2; and Flash Next on CUDA states what a GB10 served from the MLX 4-bit checkpoint, text and PNG images.
|
Thank you, @ZackSample, for rechecking earlier facts and restoring the whole pocket when a lesson is taken out. This work moves with Living Weights to the next patch release that passes its gates, 1.1.1 if they pass in time. We have kept one answer-record and rounding path alongside #872's deletion handling, including the check that an unfinished lesson cannot restore a fact deleted meanwhile. The combined build still needs lesson, repair, take-out, delete and save/reopen comparisons, with pocket bytes checked as well as recalled answers. Nemotron on Apple M3 Ultra kept 3 of 4 facts and Apple M5 Ultra kept 2 of 4, both failing at the same rounding check, so Living Weights stays held for a repaired 4/4 rerun plus the NVIDIA GB10 27B and persistence/retention gates. Posted by Ash's AI Assistant Synthena |
…red or taken out bit for bit @ZackSample's ashhart#877 (head eb2db9f): after a lesson every earlier fact is asked again; one it made forget is fitted again with its answers held (two repairs), else the lesson is taken out and the pocket put back bit for bit; checks read the pocket as a save writes it. Reconciled with ashhart#872, which shipped first: - the fact graph keeps ashhart#872's answer records on disk; Asked is that same record, and one asked() sets a fact's questions and answers (expects() folded in); - one slide_recheck.zig: ashhart#872's delete recheck and ashhart#877's lesson recheck, which judge a reply the same way; - one PocketShards.round, ashhart#872's (neurons from the slice, lengths checked); - a delete invalidates an open lesson's mark, so a take-out after it never brings back what the delete cut (new test).
… data @ZackSample's ashhart#878 (head 0bf1178): the learner, pocket and fits move to zig/src/living as a module, then a family is described as data (layout, neuron kind, shard format), Nemotron the one registered. Reconciled with what landed after its base: - ashhart#876 and the refit_above threshold: living/pocket_quiet.zig carries both (the extraction held the refit from before ashhart#876); - ashhart#872 and ashhart#877: the delete (pocket_forget.zig), the lesson's mark (pocket_mark.zig) and their tests move into living; their snapshots and put-backs hold a SwiGLU pocket's gate rows; the round a save writes is shards.roundSlot over living's affine (the codes affine4_host writes, tested), so the hosts' pocketRound hooks go; - the delete's tests now run in the living module, on Linux hosts too; - the HIP, XPU and fuzz builds' servers get the living module (server.zig), which the server's model view now imports; - 1.0.12's json module, heat, radio and GLM build wiring kept beside living's.
… only asked cold @ZackSample's ashhart#891 (head a203094): a lesson also learns a fact's questions after scripted conversations (talk, task, long, and a chain of the latest learned facts' questions) and checks its two held-out questions after two conversations no lesson learns in. The model writes each conversation once; the fact graph keeps it. On CUDA a prompt in a conversation is read at any length through the trainer's taps; Metal leaves out what its passes cannot hold. Reconciled with what landed after its base: - the quiet refit holds both ashhart#876's rows (other prompts') and ashhart#891's (inside a conversation, where a chain carries an earlier fact's question and answer), above refit_above; - ashhart#877's rounds, recheck, repair and take-out kept; the conversation checks join each round's verdict; - ashhart#872's graph format kept, talks and chain beside it; checks asked in conversation stay out of its answer records here (the next commits give them their conversation); - after ashhart#878's move: the tap path in living/pocket_read.zig, the example checks in lesson_examples.readable, cuda_train_tap.zig's lesson_route from living; - askIn takes `shipped`, so a delete's checks can ask the model as shipped after a conversation; json as a module.
…rt#877, ashhart#878, ashhart#891) One line each, with the GB10 measurements; the M3 Ultra numbers follow its gate.
…ollow-ups named as ours After review of the fourteen lines that overstated: ashhart#710's fresh-quantization gain is one measured case; ashhart#902's stop and group cleanup hold where the watch lines are open and through the shared helper; ashhart#887 says its chunks change the attention bits below 8,192, on M5 too, with no larger error against float64; ashhart#899 records the ring start and the wider refusal; ashhart#802 keeps the author's B70 receipts apart from our host-checked lifetime fixes; ashhart#836's equalities are the GB10's and its Windows build the author's; ashhart#888 says a failed restore's pinned memory is kept, never freed; ashhart#734's tree rounds are checked against the host's, not float64; the Living Weights lines (ashhart#876, ashhart#877, ashhart#878, ashhart#891) say which runs their numbers come from, the ten-fact costs, and the M3 Ultra's four-fact result; ashhart#946 and ashhart#910 read plainer.
… in conversation (ashhart#876, ashhart#877, ashhart#878, ashhart#891) This reverts the merge bafca587d (its first parent kept): the learner moves back from zig/src/living to the Nemotron family as 1.0.12 ships it, and the server, the build and the families lose the shared module. 1.1.1 takes the series again. Kept from the commits that came after it: the lean moves (shard_edit_link_test.zig, the kernel list in cuda_kernels.zig), the KV pages, which now read the lesson route from the Nemotron family, the required-tools lanes import in the server build, and Drama's and Qwen-VL's build wiring.
…Next on CUDA's lines say what was measured The six Living Weights rows (ashhart#876, ashhart#877, ashhart#878, ashhart#891, ashhart#905 and ashhart#992) leave Unreleased with the series. The EXL3 operators' line names GLM-5.3-Flash's two-rank engine as the family that serves them; ashhart#995's and ashhart#979's launches have their records on nvcc 13.0 and 13.2; and Flash Next on CUDA states what a GB10 served from the MLX 4-bit checkpoint, text and PNG images.
Stacked on #876 (its first two commits are that PR's). Fixes the rest of #870 (with the refit PR, all of it). Review from
608bcfc4.What this changes
1. Earlier facts are asked again after every lesson (
83f5c59d). A lesson's checks asked only the fact just taught(
server/slide.zigrounds()), so a lesson that made the model forget an earlier fact was kept and saved. Now, oncethe new fact comes back, every fact learned before is asked again on its stored held-out wordings, side by side
(
server/slide_recheck.zig). Each is judged as its own lesson judged it: its short answer or answer words, which thegraph now keeps beside its questions. A fact from an older graph is judged by its words, and its first good reply is
kept as its answer.
that fact's held-out answers among the answers it must leave as they were, then checked again.
once more, all before anything is saved.
counts against it.
recheck(fact, text, recalled, was),rechecked(counts and seconds),repair, and, with part2,
restored.The graph's three tests move to
slide_graph_test.zigfirst (608bcfc4, unchanged), since the stored answers takeslide_graph.zigpast 600 lines.2. A take-out puts the pocket back exactly, and every check reads the pocket as a save writes it (
42c556c0).Taking it out (a drop, an undo, a repair's reroute, an unrecalled lesson settled) puts exactly that back, hands the
changed layers to the forward, and logs the pocket's hash before and after (
pocket_mark.zig). The old undo zeroedthe lesson's own neurons and restored the earlier ones it refitted, but nothing else.
down_proj's learnedcolumns is one group of 64, which is the whole pocket of that layer. So a new fact's neurons change the scale of
every earlier fact's outputs at the layers they share, and nothing checked that: a later lesson's recheck could
find a fact forgotten, blame that lesson, and a take-out could not bring back what the save had changed.
pocket_shardsround(), which a test shows gives whatwrite()hands back, bit for bit, and is a fixed point.Every check, the lesson's own and the earlier facts' recheck, then reads what a save will write, and a save of a
checked pocket changes nothing. The host gives the pocket this as
pocketRound, on Metal and CUDA alike.restoredevent says how many of the facts itmade forget came back. When a lesson ends without being kept in any other way (a round taken back, the last drop),
each earlier fact's last verdict goes back to what it was when the lesson began. Its re-checks may have marked a
fact forgotten before a repair failed, and the next lesson must still count that fact as one that came back.
TF_SLIDE_TAKE_OUT_CHECK=1takes every kept lesson out again, for checking that the pocket and the shards come backas they were.
Receipt
lw-a2-recheck42c556c onmainc7137bb (v1.0.11 + 19), Zig 0.17.0; NVIDIA GB10 (DGX Spark,sm_121), driver 580.178.04, CUDA 13.0.88, Ubuntu 24.04.5; checkpoint
TensorFold/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bitrevision d9d758fb, a freshcp -RLcopy for each run:tensorfold-native serve <copy> --no-thinking --temperature 0 --slide --slide-graph <graph.json>.learn request each. After every lesson I asked two fixed wordings per fact (Living Weights: a quiet refit holds an earlier fact's neuron on its own questions as read now #876 lists them) and every learned
fact's stored held-out questions, then restarted the server without
--slideand with it.5, 6 and 7 earlier facts took 0.2, 0.4, 0.8, 1.2, 1.4, 1.8 and 1.9 s. No lesson needed a repair.
fixed questions after lesson 10 came back 15/20 (
main13/20, Living Weights: a quiet refit holds an earlier fact's neuron on its own questions as read now #876 alone 16/20; the one fewer than therefit PR is "What's my son's name?"). A plain restart and a
--sliderestart gave the same replies as afterlesson 10.
5 times, each
the same bits, and never otherwise.main. Lesson 10 took 7 steps and 242 s, against 2-3 steps onthe other builds: its checks now read the pocket as rounded.
mainor here, so the repair and thetake-out for forgetting did not run. They do run in the next receipt.
TF_SLIDE_POCKET_KEY=reply. Onmain, lesson 4 ("I live in Bristol.") refits 5 of Ginger's 10 neurons, is kept andsaved, and Ginger's held-out questions go from 3/3 to 0/3 ("Your dog's name is Nala.", "My name is Nemotron
...").
mainplus this PR's two recheck commits, so the refit is stillmain's. After lesson 4came back, the re-check found Ginger forgotten in 1.0 s, with the same replies as
main:{"id": 4, "fact": 3, "text": "My dog is called Ginger.", "recalled": false, "was": true}. The log saidfact 4's lesson made the model forget "My dog is called Ginger.": fitted again with their held-out answers kept as they were (repair 1 of 2). Neither repair brought Ginger back: the reroute refits the same 5 neurons, since the refitis not this PR's. The next round's own check then took the lesson back (
it made up something about the user it was never told). Nothing forgotten was saved: after the request, Ginger answered 3/3 held-out and blue and Anaall of theirs.
lesson 4's repairs. Its re-check therefore reported Ginger
"was": false, so its "Sweetie" answer did not count."recalled": false, "was": true("Your dog's name is Sweetie."), tried two repairs, and took lesson 5 out(
taken out: it made the model forget "My dog is called Ginger.", still after 2 repairs). Asked once more,Ginger came back (
3 come back, 0 forgotten).forgotten (0.2, 0.4, 0.7 and 1.1 s). Ginger kept 3/3 held-out after lessons 4 and 5. 9/10 fixed questions after
lesson 5 (
main8/10), and both restarts gave the same replies.It was started again with
TF_SLIDE_TAKE_OUT_CHECK=1, and "My favourite food is sushi." was taught.changed: each shares those layers'output groups with blue's or Ana's neurons). It came back in 2 steps, and the re-check found blue and Ana recalled
(0.4 s).
the pocket put back as the lesson found it (c9d76667e86db64f -> c65eba95899d1e84, the same bits). That is the same hash as after blue and Ana in the ten-fact run.restoredwas{"forgotten": 0, "back": 0, "lost": 0}..safetensorsfile and ofconfig.jsonafter the request equalled its value before thelesson. Blue and Ana answered all their fixed and held-out questions after it (2/2 + 2/2, 2/2 + 3/3).
nothing in them: no re-check there found a fact forgotten, and the forced take-out asked the facts again.
pocket_mark.zig: a lesson's own neuron beside an earlier one, the earlier one refitted, the layer rounded, thentaken out. With the old undo, the pocket's hash differs from before the lesson. With the mark, it is the same, and
the neurons in use and what was learned are as before.
pocket_shards_test.zig:round()equals whatwrite()hands back, bit for bit, and rounding again changesnothing.
slide_graph_test.zig: a fact's verdict, marked forgotten by a re-check, goes back to the one the lesson beganwith.
slide_recheck.zig: a held-out answer judged by its short answer or words; a fact is forgotten only when it cameback before the lesson.
the re-check (seconds above).
zig build teston Linux: 749/754 pass, 5 skipped (main: 743/748, 5 skipped).zig build test-server-cpu:208/208 (
main: 207). Every commit builds and passes.zig fmt --check build.zig zigis clean, andpython3 tools/zig/lean_check.pygives 527 problems against 527 onmain(0 new).zig build test-nativeandzig build nativewith-Dtarget=aarch64-macosand astub
xcruncompile and stop only at linking Apple's frameworks, as onmain. The Metal host gainspocketRound,the same wrapper as CUDA's.
Checklist
change; the re-check asks through the same side-by-side path as the lesson's own checks.)
"draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Nottouched.)
is checked at the precision it is saved in, which is the same as before after a save.)
type-checked only.)
Notes
608bcfc4,83f5c59d) are server-only. They also cherry-pick cleanly ontomainwithoutLiving Weights: a quiet refit holds an earlier fact's neuron on its own questions as read now #876 (208/208 server tests), if you would rather take them alone. The take-out commit needs the refit PR's
split of
pocket.zigto stay under 600 lines.false positives, so I left it out.