fix(memory): self-heal missing entry markers so hot-layer memory persists#1443
fix(memory): self-heal missing entry markers so hot-layer memory persists#1443asdf8675309 wants to merge 1 commit into
Conversation
…ists The two hot-layer memory templates (USER/PRINCIPAL/PRINCIPAL_MEMORY.md and USER/DIGITAL_ASSISTANT/DA_MEMORY.md) ship without the <!-- BEGIN ENTRIES --> / <!-- END ENTRIES --> marker pair that MemoryWriter.ts requires. On a stock install this silently breaks the "memory that compounds" loop: - parseFile() hits its markers-missing branch and returns entries: [], so every reader (LoadMemory.hook, MemoryHealthCheck, Pulse) sees zero entries no matter how many the reviewer has written; - serializeFile() re-adds a lone END marker on each write, so the file accretes orphan END markers while remaining unreadable, and nothing alarms (setEntries returns ok). Two-layer fix: 1. Add the BEGIN/END marker pair to both shipped templates so fresh installs start well-formed. 2. Make parseFile() self-heal a marker-less or corrupted file: recover well-formed entry lines, drop stray markers, preserve the intro text, and rebuild a single clean BEGIN/END scaffold. This also repairs files already corrupted in the field, not just fresh installs. Refs danielmiessler#1409, danielmiessler#1426 Co-authored-by: Claude <noreply@anthropic.com>
|
Problem confirmed: stock templates ship marker-less → Production lesson we've extracted: marker-based parsing must self-heal when the template/install is incomplete. The failure mode here (silent loss of all entries + accumulating corruption) is the worst case — data is written but never read back, so the author sees success but the system has amnesia. Why this fix is architecturally sound:
The key insight: when a read/write loop depends on structural markers, the reader should be defensive (reconstruct valid structure from partial data), not just the writer (assume structure exists). Otherwise, installs that shipped broken stay broken. Minimal-abstraction verification path: # Before: 0 markers in shipped template
grep -c 'BEGIN ENTRIES' LifeOS/install/USER/PRINCIPAL/PRINCIPAL_MEMORY.md # -> 0
# After: 1 marker pair present
grep -c 'BEGIN ENTRIES' LifeOS/install/USER/PRINCIPAL/PRINCIPAL_MEMORY.md # -> 1
# Self-heal on corrupted file: reader recovers all entries, next write emits
# clean single BEGIN/END pair (no stray markers), round-trips idempotently.Our similar pattern: we hit the same class of problem in session transcript markers — when a hook fails mid-write, the marker structure breaks, and subsequent reads fail silently. Our fix: reader always rebuilds valid structure from line-level data (entries/events) rather than trusting markers to be present. This PR does exactly that. Tested in isolation (macOS, v6.0.5). No automated test added because Fixes #1409, #1426. Strongly recommend merge — silent memory loss is a trust-breaker for "memory that compounds." Hit identical marker-corruption in session checkpoints. Discussion: T-MEM: Memory is the Moat + T-6SX: Six Self-X Properties |
|
Thanks a lot for this, @asdf8675309 — really appreciate you flagging it and taking the time to write the fix. When I went back through the current source, this turns out to already be handled: hot-layer memory marker self-heal already shipped independently in both templates and MemoryWriter. The system has moved a fair bit since you opened this (the PAI → LifeOS rename and a few subsystem rewrites), so the gap you spotted has since been closed independently. Going to close this one out on that basis — but genuinely grateful for the contribution. 🙏 |
Problem
The two hot-layer memory templates —
USER/PRINCIPAL/PRINCIPAL_MEMORY.mdandUSER/DIGITAL_ASSISTANT/DA_MEMORY.md— ship without the<!-- BEGIN ENTRIES -->/<!-- END ENTRIES -->marker pair thatMemoryWriter.tsrequires. On a stock install the "memory that compounds" loop silently never persists:parseFile()hits its markers-missing branch and returnsentries: [], so every reader (LoadMemory.hook,MemoryHealthCheck, Pulse) sees zero entries no matter how many the reviewer has written.serializeFile()re-adds a loneENDmarker on each write, so the file accretes orphanENDmarkers while staying unreadable — and nothing alarms (setEntriesreturnsok).Reported in #1409 and #1426.
Fix
Minimal, no new abstractions:
BEGIN/ENDmarker pair to both shipped templates so fresh installs start well-formed.parseFile()self-heal — on a marker-less or corrupted file, recover well-formed entry lines, drop stray markers, preserve the intro text, and rebuild a single cleanBEGIN/ENDscaffold. This also repairs files already corrupted in the field, not just fresh installs — a serialize-sideBEGIN-guard alone would leave existing orphaned entries stranded outside the markers.Verified (macOS, LifeOS v6.0.5)
0BEGINmarkers; after the reviewer ran,PRINCIPAL_MEMORY.mdheld real entries interleaved with several stray<!-- END ENTRIES -->markers and noBEGIN;LoadMemoryinjected(no entries yet)whileread()returned0.BEGIN/ENDpairENDs, noBEGIN) →parseFilerecovers all N entries, serialize emits one marker pair, round-trips idempotentlyread()returns the correct entry count andLoadMemoryinjects the entries.Manual verification steps
Limitations
MemoryWriter.ts's built-insmokeTest()writes to the installed~/.claude/…path (viaALLOWED_FILESkeyed tohomedir()), not a repo fixture, so it can't run hermetically against the repo copy — I verified the parser logic in isolation instead. Happy to add a fixture-based test if you'd prefer one.Fixes #1409, #1426