feat(memory): compile the memories room into a family chronicle

.gitignore

@@ -68,3 +68,10 @@ impl/
 !/AGENT.md
 # Local planning workspace (Nimbalyst) — not product source
 nimbalyst-local/
+
+# Rendered test corpora — the spec is the source of truth, the audio
+# and images it renders are disposable artifacts.
+tools/family-memories/out/
+
+# Nimbalyst editor scratch (screenshots, transcript images)
+.nimbalyst/

docs/design/brain/memories-pipeline.md

@@ -0,0 +1,107 @@
+# Memories Processing Pipeline
+
+Status: **draft** — companion to [diary-journal.md](diary-journal.md)
+(which covers the timeline/diary/journal split and rendering). This doc
+covers the step before: turning a messy, already-populated memories
+room into clean timeline entries. Grounded in a structural probe of the
+real room (Sept 2026) and scored by the `tools/family-memories` corpus.
+
+## What the room actually contains
+
+| Pattern | Consequence for the pipeline |
+|---|---|
+| Voice memos with spoken date openings ("Hallo X, heute ist der …") | the date is in the audio, nowhere else |
+| Sync bursts: offline recordings uploaded together, out of order | `origin_server_ts` = sync time, days off; ordering within burst is meaningless |
+| True fragments: one recording split mid-sentence | must be joined before summarizing |
+| Kitchen-table dialogues (2+ speakers) | need dialogue-aware handling, not verbatim monologue treatment |
+| Images + late caption via reply relation | caption belongs to the image |
+| Texts referencing "the picture above" with no relation | implicit context, adjacency-based |
+| Long texts with `m.replace` edits | only the final version counts |
+
+Matrix stores **no compose time** — verified against the spec and the
+room's own events. Element strips filenames and metadata. A synced
+memo without a spoken date has an unrecoverable date.
+
+## Pipeline
+
+```
+room history (paginated, oldest-first)
+  → 1 resolve      edits collapsed, replies attached, bursts detected
+  → 2 transcribe   Whisper; cached in TRANSCRIPT_DIR (never re-pay GPU)
+  → 3 classify     per item, local LLM, structured output:
+                   {mode, spoken_date?, fragment_boundary?, addressee?}
+  → 4 join         fragment pairs merged (ends-mid-thought ⨯ continues)
+  → 5 date         spoken > live timestamp > burst ⇒ UNCERTAIN
+  → 6 compile      timeline entries (diary-journal.md takes over)
+```
+
+1. **Resolve** — pure Matrix mechanics, no AI: apply `m.replace`, attach
+   reply-captions to parents, group events <120s apart from the same
+   sender into candidate bursts. Keep event ids for idempotency.
+2. **Transcribe** — existing Whisper path + `TRANSCRIPT_DIR` cache
+   (core already reserves it for exactly this backfill).
+3. **Classify** — one structured-output call per transcript
+   (temperature 0): monologue/dialogue, spoken date if present,
+   starts/ends mid-thought, addressee. This worked cleanly in the probe.
+4. **Join** — a burst is NOT a fragment chain (the probe's key trap:
+   three same-second uploads were three independent memos). Join only
+   when A ends mid-thought AND B continues it — an LLM judgment on the
+   pair, not a timing heuristic.
+5. **Date** — the timestamp is usually right: most messages are sent
+   live, so `origin_server_ts` is the default. Spoken dates override
+   it when present. Only messages inside a detected sync burst get the
+   exception treatment — there the timestamp is days off, so without a
+   spoken date the entry is dated "week of <sync date>" and marked
+   uncertain rather than confidently wrong.
+6. **Compile** — entries carry `{date, date_confidence, kind, people,
+   transcript/summary, assets, source event ids}` into
+   `memory/timeline/`, per diary-journal.md.
+
+## Decisions
+
+- **Backfill-first.** The room is already populated; the compiler is a
+  rerunnable batch over full history, incremental later. Idempotency
+  via source event ids in each timeline entry.
+- **Diarization stays out of v1** (per diary-journal.md), but step 3
+  cheaply *labels* dialogues, so the renderer can mark them "captured
+  by X, conversation" and a later diarization pass knows exactly which
+  few recordings to touch.
+- **Goal is topics, not verbatim accuracy.** Whisper large-v3-turbo was
+  rated clean on real German memos; good enough. No model change needed.
+- **Pages are a chronicle/diary hybrid, not transcript dumps.** Detail
+  scales with the source: short memos stay verbatim, long recordings
+  get a gist, a short narrative, selected word-for-word quotes, and
+  the audio link. Quoted words are verified against the transcript;
+  narrative renders as narrative. The full transcript stays in the
+  transcript store; audio in Matrix is the archival original.
+- **Privacy shape:** content flows machine-to-machine (Synapse →
+  Whisper → oMLX → vault); only structure and compiled entries surface.
+
+## Open
+
+- Date fix at the source: a small upload path that stamps
+  `dev.famstack.recorded_ts` into event content would eliminate the
+  uncertain class for future memos. Family habit of speaking the date
+  covers the past.
+- Burst window (120s): settled differently than expected. Timing alone
+  turned out to be the wrong signal -- three memos recorded a minute
+  apart at the dinner table are not a sync burst, and calling them one
+  filed a normal evening as undateable. A run now has to contradict
+  itself (some memo says aloud it was made on a day its own timestamp
+  disagrees with) before its timestamps are distrusted, which leaves
+  the window doing nothing but grouping what arrived together.
+- Join thresholds: gone. The model names which message finishes which,
+  having both in front of it; the room checks the link is adjacent,
+  same sender, same kind.
+
+- The polish pass could take the household vocabulary too. Whisper now
+  decodes against the family's names and topics, which is where a
+  misheard name has to be fixed -- polish may not change words and
+  should not. But the same vocabulary would help polish decide where
+  sentences break around a proper noun it now knows is a name. Its
+  contract does not move: clean sentences out of an imperfect
+  transcription, same tone, same words.
+- Re-transcription when the vocabulary changes. A new family member
+  does not improve recordings already decoded, and re-running whisper
+  over years of audio to pick up one name is the wrong default. An
+  explicit `--retranscribe` would make it a choice.

docs/design/brain/memory-pipeline-architecture.md

@@ -0,0 +1,211 @@
+# Memory Pipeline Architecture
+
+Status: reference, September 2026. Describes the voice path from the
+Matrix memories room to the published diary, the ontology layer, and
+the mitigations this system needs because it runs on local models.
+
+## Context and constraints
+
+The pipeline runs on one Apple Silicon host (M1 Max, 64 GB). Models:
+Qwen3.6-35B-A3B 4-bit on oMLX for text, whisper.cpp large-v3-turbo
+for speech. Both are small compared to hosted models and show error
+classes that hosted APIs rarely surface. The output is a family
+archive: pages are read years later, by people who were in the
+recordings. An error in a published page misstates someone's life.
+
+Consequences for the design:
+
+- Generated text is verified mechanically or constrained to closed
+  answer sets. Prompt instructions alone did not prevent any of the
+  failures listed under Pitfalls.
+- Every model call has an output token cap derived from a measured
+  ratio. Uncapped local calls monopolize the single GPU.
+- All expensive results are cached, keyed by content, so a full
+  recompile is affordable and deterministic.
+
+## Component overview
+
+```
+Matrix memories room (source of truth, audio = archival original)
+        │
+        ▼
+  transcription (whisper.cpp, configured flags, quality capture)
+        │
+        ▼
+  transcript store   ~/famstack-data/core/transcripts/, 1 JSON/event
+        │              {raw, text, quality, passes[]}
+        ▼
+  pass chain         gate → polish → correct → structure
+        │              (lib/stack/ai/transcripts.py, versioned)
+        ▼
+  reading            facts per message: date, mode, addressee,
+        │            fragment links, gist, candidate quotes
+        ▼
+  compile            date precedence, fragment joins, reply/caption
+        │            attachment, entry assembly
+        ▼
+  render             language table, tiered entry formats
+        │
+        ▼
+  wiki (Quartz)      pages in memory/brain, committed by curator
+```
+
+The room is append-only and the compiler is a fold over the full
+history. There is no watermark: a reply or an edit can land on a
+year-old entry, and only a full pass attaches it. Caches make the
+fold cheap; they store cost, not position.
+
+## Transcription
+
+whisper.cpp flags, set in the LaunchAgent and reconciled on every
+`stack up ai`:
+
+| Flag | Value | Reason |
+|---|---|---|
+| `--language` | from `[core].language` | auto-detection fails on silence and noise; one incident recording of infant sounds was transcribed as CJK text |
+| `--max-context` | 0 | a hallucinated segment otherwise seeds the next segment |
+| `--suppress-nst` | on | drops non-speech tokens |
+
+Each flag was tested against the three recordings that produced the
+2026-09-14 incident. Each flag fixed loops the other two did not;
+only the combination produced zero repetition on all three.
+
+Transcription requests use `verbose_json` with word timestamps. The
+stored quality record keeps per-segment `avg_logprob`,
+`no_speech_prob`, `temperature`, a word count, and every word below
+0.5 confidence. whisper computes these values on every call; the
+plain `json` format discards them. Voice commands and chat notes use
+the plain format and skip all quality machinery — the cost is only
+paid where an archive is built.
+
+## Transcript store and pass framework
+
+One record per Matrix event, atomic writes, single-flight production.
+Record fields:
+
+- `raw` — whisper output, never modified by any pass
+- `text` — current output of the pass chain
+- `quality` — segment metrics and low-confidence words
+- `passes[]` — `{name, version, outcome, model?, replacements?}`
+
+The pass list makes model upgrades incremental: increasing a pass
+version marks affected records stale (`stale_passes`), and a sweep
+re-runs that single pass on stored raw text without re-running
+whisper. `--retranscribe` exists for the case where whisper itself
+changed.
+
+| Pass | Function | Failure class covered |
+|---|---|---|
+| gate | empty the text when the transcript is hallucinated | repetition loops (measured 5–147 repeats of one 6-gram vs 1–2 in real speech); majority of segments failing whisper's own thresholds (avg_logprob < −1.0, no_speech_prob > 0.6) |
+| polish | restore punctuation; word sequence verified unchanged | unreadable single-block output |
+| correct | map low-confidence words to household names, closed set | misheard names (measured: p=0.19 on the one confirmed case) |
+| structure | paragraph breaks at ≥1.5 s segment pauses, from word counts | wall-of-text rendering; no model call |
+
+The gate keeps two independent signals because the failure classes
+are disjoint: a repetition loop is a high-confidence failure the
+logprob check does not see, and mumble is a low-confidence failure
+the repetition check does not see.
+
+## Ontology layer
+
+Sources: person pages in the wiki (canonical spelling plus synonyms)
+and `ontology.toml` topics. Consumers:
+
+| Consumer | Use | Effect |
+|---|---|---|
+| whisper priming | names and topics as decoder prompt | fewer mishearings at the source |
+| correct pass | closed replacement set | a repaired word is always a real name |
+| summary prompt | list of family members | a name outside the list is treated as a mishearing and kept off the page |
+| archivist (documents domain) | tags and correspondents | same principle, pre-existing |
+
+The pattern in all four: the ontology converts an open generation
+problem into selection from a known set. Selection is the reliable
+operation at this model size.
+
+## Reading, compilation, attribution
+
+The reading returns facts per message, JSON, temperature 0: spoken
+date, mode (monologue/dialogue/note), addressee, fragment links,
+gist, candidate quotes. Rules with rationale:
+
+- **Quotes are verified.** A candidate quote renders only if it
+  matches one sentence or a consecutive run in the transcript
+  (case/punctuation-insensitive). The page shows the transcript's own
+  text. Quotes containing any low-confidence word are dropped.
+- **Date precedence:** spoken date > live timestamp > unrecoverable.
+  Matrix records only server receipt time; a synced message can be
+  days off. Sync-burst messages without a spoken date are filed under
+  the week they surfaced and labeled.
+- **Attribution is limited to structural facts.** Sender and spoken
+  addressee are known. Line-level attribution inside a conversation
+  is unknown until diarization exists; summaries may name a
+  conversation's participants and nothing finer.
+- **Fragment joins are content decisions.** Three uploads in one
+  second are usually three memos; a join requires one message to end
+  mid-sentence and the next to continue it. Timing alone misfiled
+  ordinary evenings as sync bursts before this rule.
+
+Summary input is preprocessed: words the pipeline knows are unclear
+are replaced with `[unclear]` before the model sees them. This
+replaced an instruction ("do not use unreliable words"), which the
+model had ignored.
+
+## Rendering
+
+- Entries under 120 words render verbatim.
+- Longer recordings render as gist, verified quotes, folded full
+  transcript, audio link.
+- Messages addressed to one person render whole at any length.
+- Recordings whose transcript the gate emptied render with a fixed
+  explanatory sentence and the audio link.
+- All reader-facing strings, month and weekday names come from a
+  language table (`en`/`de`), selected once per run from config.
+  Prompts that produce family-facing text name the target language
+  explicitly; a model otherwise answers in the prompt's language.
+
+## Pitfalls (incident-derived)
+
+| Incident | Cause | Mitigation |
+|---|---|---|
+| 100k-token generation, 25 min GPU monopoly (2026-09-14) | polish is an echo task; a looping transcript has no natural end; no output cap; server default max_tokens was 128000 | gate before any echo task; caps at input size + 10% (measured output ratio 0.98–1.01); server default lowered |
+| misheard name presented as a family member | whisper flagged the word (p=0.19) but the summary prompt's warning was ignored | correction pass (closed set), evidence redaction, ontology priming, quote filter |
+| English output in a German diary | prompts are English; models answer in the prompt language | target language stated in every prompt; rendered strings from the language table |
+| whisper dead after every `stack down ai` / `up ai` | stop hook unloads the LaunchAgent; nothing on the up path loaded it; `RunAtLoad` fires only at login | on_start reconciles the agent by content and loads it when absent |
+| undatable memories | Matrix has no compose-time field; offline recordings carry sync time | spoken-date extraction; honest "unrecoverable" state; recording habit: say the date aloud |
+| stale caches serving old wording | caches were keyed by content only; prompt changes did not change the keys | every cached artifact stores a fingerprint (hash) of the prompt or pass parameters that produced it; a prompt edit invalidates exactly the affected artifacts on the next compile |
+
+## Open items
+
+- Diarization for conversations (would upgrade attribution from
+  participant-level to line-level; mode labels already mark the
+  affected entries).
+- Episode grouping: a vacation spanning many entries currently
+  renders as independent entries plus one month summary.
+- Inline media on diary pages: images, and an audio player with a
+  play button per recording. Design: media export at compile time,
+  not a proxy. Matrix is capture transport and timeline anchor; the
+  archive is plain files, the same pattern the archivist uses for
+  documents (chat -> Paperless).
+  - The compiler writes each media original to
+    `{data_dir}/memory/media/<yyyy>/<mm>/<event-id>.<ext>`,
+    idempotent by event id. Audio gets an `.m4a` transcode beside
+    the original (Safari does not play Ogg/Opus reliably); images
+    get a page-weight thumbnail. ffmpeg is a compile-time
+    dependency.
+  - Serving is an open choice: a static mount in the wiki
+    container, or a thin proxy that serves from the same filesystem
+    and gives logical URLs independent of the on-disk layout. Either
+    way the renderer emits stable URLs and
+    `<audio controls preload="none">`, with no auth and no runtime
+    Synapse dependency.
+  - Long-term property: dated ordinary files beside Markdown pages,
+    readable without any famstack software. `data_dir/memory` is in
+    backup scope; the Synapse media copy becomes redundant.
+  - Rejected: serving from Synapse's authenticated media API
+    (1.160) at view time. It keeps the archive dependent on a
+    running homeserver, for no storage saving once transcode caches
+    exist. The export to plain files is the decided part.
+- Retranscription (`--retranscribe`) is manual; it is needed only
+  when whisper's configuration or vocabulary changes. Pass and prompt
+  changes regenerate automatically via fingerprints during any
+  compile, including the nightly one.