Skip to content

fix(memory): hot-layer entry markers — scaffold both, heal a missing BEGIN, migrate existing installs#1427

Closed
rpriven wants to merge 1 commit into
danielmiessler:mainfrom
rpriven:fix/hot-layer-memory-markers
Closed

fix(memory): hot-layer entry markers — scaffold both, heal a missing BEGIN, migrate existing installs#1427
rpriven wants to merge 1 commit into
danielmiessler:mainfrom
rpriven:fix/hot-layer-memory-markers

Conversation

@rpriven

@rpriven rpriven commented Jul 5, 2026

Copy link
Copy Markdown

Fixes #1409. Closes #1426 (my duplicate report — @badosanjos reported the same root cause first in #1409).

The bug (two parts, one failure)

On a stock install the hot-layer memory files never load into a session. The reviewer
writes durable facts, but LoadMemory.hook.ts injects (no entries yet). Root cause is
two compounding defects:

  1. Templates ship with no entry markers. install/USER/PRINCIPAL/PRINCIPAL_MEMORY.md
    and install/USER/DIGITAL_ASSISTANT/DA_MEMORY.md had neither <!-- BEGIN ENTRIES -->
    nor <!-- END ENTRIES -->. The reader slices between those markers, finds no BEGIN,
    and returns empty.
  2. serializeFile heals a missing END but never a missing BEGIN. So every write to a
    marker-less file appended another END and never a BEGIN — the file accreted stray
    END markers, read as empty forever, and dedup/cap/eviction never ran.

The fix

Fresh installs — add both markers to the two shipped templates:

<!-- BEGIN ENTRIES -->
<!-- END ENTRIES -->

Can't recurserializeFile now adds a missing BEGIN_MARKER, symmetric to the
existing END_MARKER guard.

Existing data survivesparseFile, on a marker-less/malformed file, now recovers
the real PREFIX_PATTERN entries from the body (dropping stray markers) instead of
discarding them, so the next write re-emits them inside a canonical block. The recovery
scanner skips lines inside code fences and blockquotes, so a documented PREFERENCE: …
example is never adopted as a real entry.

Existing installs — self-heal, plus an on-demand accelerator. Once a user updates and
has the fixed MemoryWriter.ts, a broken file repairs automatically on the next reviewer
write
(the parseFile recovery above re-blocks the orphaned entries) — no manual step.
The Update workflow's ScaffoldUser.ts copyMissing deliberately never overwrites existing
USER files, so the template fix alone would not touch them; the writer fix is what covers
them. For an immediate repair without waiting for the next reviewer cycle, the new
standalone MigrateMemoryMarkers.ts fixes both files on demand — same invocation model as
the existing MigrateContextFreshness / MigrateTelosFreshness tools (run directly; those
are not auto-invoked by Update either). It's idempotent (canonical files skipped), backs up
before repair, supports --dry-run, and reuses the tested read()setEntries() path.
Maintainers may optionally wire the migrations into the Update workflow as a separate
enhancement.

Testing

Two runnable tests (no test runner is configured in the repo; run with an isolated HOME):

  • MemoryWriter.markers.test.ts — 12 checks: fresh-template round-trip, the previously
    untested cross-session path (write in session A → read in session B), recovery of
    an already-corrupted file through the real add-flow, and the fence/blockquote guard.
  • MigrateMemoryMarkers.test.ts — 9 checks: dry-run is side-effect-free, corrupted files
    heal (orphaned entries recovered), canonical files are left byte-identical, idempotent
    on re-run.
HOME=$(mktemp -d) bun LifeOS/install/LIFEOS/TOOLS/MemoryWriter.markers.test.ts
HOME=$(mktemp -d) bun LifeOS/install/LIFEOS/TOOLS/MigrateMemoryMarkers.test.ts

Both green. MemoryWriter.ts and MigrateMemoryMarkers.ts compile clean under
bun build. The happy path is provably unaffected: the new recovery branch only runs when
markers are missing/malformed, and the new BEGIN guard is a structural no-op on well-formed
files (their preEntriesBody already contains the marker).

For users already on a broken install

After updating, existing memory auto-repairs on the next reviewer write. To fix immediately
without waiting, run the migration:

bun ~/.claude/LIFEOS/TOOLS/MigrateMemoryMarkers.ts        # --dry-run to preview

Or hand-edit both ~/.claude/LIFEOS/USER/PRINCIPAL/PRINCIPAL_MEMORY.md and
.../DIGITAL_ASSISTANT/DA_MEMORY.md: put one <!-- BEGIN ENTRIES --> immediately above the
first entry and a single <!-- END ENTRIES --> after the last, removing any stray END
markers in between. No data is lost — the stranded entries are preserved, just re-blocked.

Out of scope (noted, not fixed here)

  • parseFile locates markers with indexOf, so prose that mentions the literal marker
    text before the real block could shadow entries. Pre-existing (reproduces on main
    un-patched); worth hardening to whole-line marker matching in a follow-up.
  • allowDrastic has no live wiring anywhere — a legitimate full-clear is structurally
    unreachable. Orthogonal; separate follow-up.

@rpriven rpriven force-pushed the fix/hot-layer-memory-markers branch from b818fb2 to 0876df3 Compare July 5, 2026 04:06
…BEGIN, migrate existing installs

Hot-layer memory never loads into a session on a stock install: the templates
ship without <!-- BEGIN ENTRIES -->/<!-- END ENTRIES --> markers, and
serializeFile re-adds a missing END but never a missing BEGIN — so every write to
a marker-less file accretes stray END markers and the reader (LoadMemory) slices
an empty block. The reviewer captures durable facts that are then never recalled.

- templates: ship both markers (fixes fresh installs)
- MemoryWriter.serializeFile: heal a missing BEGIN, symmetric to the END guard
- MemoryWriter.parseFile: recover orphaned entries from an already-broken file
  (skipping code-fence/blockquote lookalikes) so existing data survives next write
- MigrateMemoryMarkers.ts: idempotent one-time repair for existing installs
- tests: cover the previously-untested write-then-recall-next-session path

Co-Authored-By: Kai <noreply@anthropic.com>
@danielmiessler

Copy link
Copy Markdown
Owner

Thanks a lot for this, @rpriven — really appreciate you flagging it and taking the time to write the fix.

When I went back through the current source, this turns out to already be handled: hot-layer memory marker heal already implemented independently in current source. The system has moved a fair bit since you opened this (the PAI → LifeOS rename and a few subsystem rewrites), so the gap you spotted has since been closed independently.

Going to close this one out on that basis — but genuinely grateful for the contribution. 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants