Close A1/A2 structurally; red-team the gold suite (A3+B2)

NEXT-STEPS.md

@@ -37,48 +37,63 @@ failure mode first.
 These are places where the structure does not (or cannot) catch the unwanted
 move, so it is held by a softer guard and owned openly.
 
-### A1. Routing-hint metadata travels unguarded
+### A1. Routing-hint metadata — the boundary is now structural; its edges named
 The type that crosses to the model (`RoutingHint` in `src/types.ts`) has no
 field for a note's body text, so the body cannot leak along that path
-(`src/no-leak.ts`). But two fields do travel: the **label** (currently the
-note's title, set in `buildPrivateNotes` in `src/corpus.ts`) and the
-**locator** (from frontmatter). A note with a sensitive title leaks through its
-label, and nothing in the type stops it.
-
-- **Trade-off:** richer labels and locators help the model route well; every
-  field that travels is also a leak surface.
-- **Current posture:** documented, with a loud warning at the population site
-  (`src/corpus.ts`) and in the README — keep titles and locators public-safe.
-  The guard is discipline, not structure.
-- **For a fork / contributor:** make the boundary structural instead of
-  advisory. Options: derive the label from a public-safe identifier rather than
-  the raw title; whitelist or sanitize the fields that may travel; or carry a
-  separate, explicitly-public "display label" distinct from the private title.
-  A build-time lint that flags obviously-private patterns in the traveling
-  fields is a cheap first step before any of these. Any of them moves this seam
-  from "owned" to "inexpressible," which is where it should end up.
-
-### A2. Related-material mode admits confabulation (provenance without backing)
-In related-material mode the answer cites a routing hint and is otherwise free
-prose. The hint is real and was retrieved, so a claim citing it **passes the
-structural grounding gate** (`assertCitationsGroundedInEvidence` in
-`src/answer.ts`) — it has provenance. But the hint carries no text, so there is
-**no backing** for any prose about the moment's actual contents. A model that
-fabricates substance and cites the hint anyway clears the gate. This is exactly
-the provenance-without-backing residue the grounding definition already declines
-to certify; it is not a hole in the gate, it is the edge the gate was honest
-about.
-
-- **Trade-off:** natural-language routing ("there's a relevant private passage
-  here") is useful and humane; it is also the freedom a confabulation hides in.
-- **Current posture:** held by a soft prompt instruction (route, don't restate —
-  `src/prompt.ts`) and a hand-written set of forbidden-answer patterns (see A3).
-  The model's unbacked claim is disavowable as such, and owned.
-- **For a fork / contributor:** close it structurally by **templating the
-  related-material answer from the hint's public-safe fields** (label, locator,
-  URL), so the mode can only point, never assert content. Confabulation then
-  becomes inexpressible in that mode rather than merely discouraged. This is the
-  highest-value structural ticket in the file.
+(`src/no-leak.ts`). Two fields do travel — the **label** and the **locator**
+— and they used to be raw frontmatter guarded by a warning comment: a note
+with a sensitive title leaked through its own label, and nothing in the type
+stopped it.
+
+- **Current posture:** structural, three layers deep. The traveling label is
+  an explicit `label:` frontmatter field, distinct from the private `title`
+  (which is embedded for retrieval and never travels) — a fork upgrading past
+  this change fails loudly until each note declares one, which is the point:
+  what travels is now an authored decision per note, not a default. Both
+  traveling fields are typed `PublicSafe`, whose only constructor is the
+  build-time lint (`assertPublicSafeField` in `src/public-safe.ts`): single
+  line, capped length, and no run of five consecutive words shared with the
+  note's private body — a field that quotes the note fails the build, not the
+  answer. The index schema versioned past the split (v3, `src/store.ts`), so
+  a stale artifact fails fast with the remedy.
+- **The residue, named:** the lint is a tripwire, not a classifier — a short
+  private phrase, or private meaning carried in public words, still passes
+  it. `url` (`about:`) travels unlinted as a declared public page. And the
+  brand erases at JSON boundaries: an index read from disk is trusted to have
+  been built through the lint, not re-checked. The gold canaries
+  (`eval/gold.yaml`) backstop all three at answer time.
+- **For a fork / contributor:** tune `PUBLIC_SAFE_NGRAM_WORDS` against your
+  corpus (5 is calibrated so bibliographic locators pass; see the constant's
+  comment), and if your private layer has a known sensitive vocabulary, add a
+  denylist check beside the n-gram tripwire — the lint is one function with
+  one call site, built to take it.
+
+### A2. Related-material confabulation — closed structurally; the residue moved
+In related-material mode the answer cites a routing hint, and a hint is real
+provenance with **no backing**: it carries no text, so prose about the
+moment's actual contents can never be certified. This was the
+provenance-without-backing edge the grounding gate
+(`assertCitationsGroundedInEvidence` in `src/answer.ts`) was honest about — a
+model that fabricated substance and cited the hint anyway cleared it, held
+only by a soft prompt instruction (route, don't restate).
+
+- **Current posture:** closed. The related-material answer is no longer model
+  prose: `finalizeAnswer` (`src/answer.ts`) replaces it with a fixed template
+  rendered only from the cited hints' public-safe fields
+  (`renderRelatedMaterialAnswer` in `src/public-safe.ts`), after grounding, so
+  the mode can point and never assert content. Confabulation in this mode is
+  now inexpressible rather than discouraged; the gold suite pins the template
+  (`expectAnswerPatterns` on q07 and the extraction queries) so a regression
+  cannot silently un-template the mode.
+- **What the closure is worth:** exactly the safety of the fields it renders.
+  The template's prose is label + locator — which is A1's seam. Closing A2
+  raised the stakes on closing A1.
+- **The residue, named:** `supported` mode still carries a hint citation under
+  free prose (a record backs the prose; the hint adds where else to look), so
+  a model could still confabulate a note's contents *there*. That residue is
+  owned by gold canary patterns (q15 and the canary comment in
+  `eval/gold.yaml`), not by structure — templating supported-mode prose would
+  mean templating record-backed answers, which is the product.
 
 ### A3. Forbidden-answer patterns are hand-written and partial
 The checks that catch a few specific bad outputs (for example, a raw URL where
@@ -90,10 +105,20 @@ the `forbidAnswerPatterns` field on a gold query, applied in `judgeAnswer`
   positives; broad ones catch more but start refusing good answers.
 - **Current posture:** partial coverage, openly. Treated as a regression guard
   for known failure shapes, not a soundness boundary.
-- **For a fork / contributor:** audit the pattern set against the modes; add
-  coverage for each mode's characteristic failure; consider replacing the most
-  fragile patterns with a structural check (A2 removes the need for several of
-  them outright).
+- **Current posture (updated):** audited against the modes. Each mode now
+  carries its characteristic-failure coverage in `eval/gold.yaml`: canon
+  answers forbid private-body canary phrases (q02, q05), refusals forbid
+  URL/citation-shaped debris (q08–q10, q14), the boundary queries forbid the
+  canaries outright and *require* the A2 template (`expectAnswerPatterns`),
+  and the extraction/injection queries (q11–q14) aim the attack directly at
+  the boundary. The A2 template did what was predicted — the fragile
+  "did the model restate the note?" patterns are now backstops behind a
+  structural check rather than the only line.
+- **For a fork / contributor:** the shape of the audit transfers, the
+  patterns don't. When you swap in your corpus, pick fresh canaries from your
+  own private bodies, verify they appear on no public page, and keep one
+  extraction query and one injection query aimed at whatever your private
+  layer actually is.
 
 ---
 

README.md

@@ -45,8 +45,9 @@ The corpus has two layers, and the distinction drives everything downstream
   The body travels all the way to the model, because you already published it.
 - **Private notes** are material you want *searchable but never quotable* —
   here, the songwriter's notebook in `example-content/notebook/`. Each note
-  declares the public page it routes to (`about`) and where the moment lives
-  (`locator`). Its text gets embedded, so retrieval can find it. It is never
+  declares the public page it routes to (`about`), where the moment lives
+  (`locator`), and a public-safe display name (`label`) — its `title` and
+  text stay private, embedded so retrieval can find the moment but never
   shown to the model.
 
 > In production ([Ask the Archive](https://lukefwalton.com/ask/)), published
@@ -97,9 +98,14 @@ One Responses API call (`src/answer.ts`), with the policy versioned in code
 (`src/prompt.ts`). Records render with their full bodies. Hints render as
 label, locator, and URL — `buildUserPrompt` couldn't leak a hint's text if it
 wanted to, because the field doesn't exist. **What does travel is the label
-and the locator: any frontmatter field that becomes either one reaches the
-model, so keep titles and locators public-safe.** (Making that boundary
-structural rather than advisory is [`NEXT-STEPS.md`](./NEXT-STEPS.md) A1.)
+and the locator, and both are typed `PublicSafe`:** the label comes from an
+explicit `label:` frontmatter field (the private `title` never travels), and
+the only way to construct the type is the build-time lint
+(`assertPublicSafeField` in `src/public-safe.ts`), which rejects a traveling
+field that quotes the note's own body. The lint is a tripwire, not a
+classifier — a short private phrase still passes it — so write labels and
+locators like captions; what the lint can and can't catch is owned in
+[`NEXT-STEPS.md`](./NEXT-STEPS.md) A1.
 The model is told what a hint *is*: the location of a relevant private
 moment, to be routed to, never restated. And if nothing cleared the score
 floor, the engine returns `not-found` without making the call at all —
@@ -128,6 +134,14 @@ Finally, `assertCitationsGroundedInEvidence` verifies every citation is the
 exact (id, url) pair of something actually retrieved. An invented source is
 an error, not a footnote.
 
+One mode gets a fourth layer. A `related-material` answer's prose is not the
+model's: after grounding, the engine replaces it with a fixed sentence
+rendered from the cited hints' label and locator
+(`renderRelatedMaterialAnswer` in `src/public-safe.ts`). A hint citation is
+provenance without backing — the hint carries no text — so free prose there
+was the one place a confabulated "summary" of private material could pass
+every gate. Now the mode can point, never assert content.
+
 One UI lesson: **retrieved is not cited**. Retrieved neighbors are
 candidates; final citations are evidence. If you build a web UI around this,
 render source cards from the final citation list, not from raw retrieval
@@ -216,12 +230,16 @@ get `temperature: 0`).
    files the way you want your citations to read. Frontmatter the engine
    reads: `title` (required), `description`/`summary`/`meaning`,
    `themes`/`keywords`/`topics`, `date`, `draft: true` to skip a file.
-3. Private notes additionally need `about` (the public URL to route to) and
-   `locator` (where the moment lives). One contract to respect: a note's
-   `title` and `locator` ARE public-safe surface — they travel into hints and
-   answers — so write them like captions, not like the note itself. Only the
-   body is private. No private layer? Remove `privateNotesDir` from the
-   config and the engine runs public-only.
+3. Private notes additionally need `about` (the public URL to route to),
+   `locator` (where the moment lives), and `label` — the display name that
+   travels into hints and answers. The `title` and body stay private (they
+   are embedded for search, never shown to the model); the `label` and
+   `locator` ARE public surface, so write them like captions, not like the
+   note itself. A build-time lint rejects a label or locator that quotes the
+   note's body — repeating the title as the label is fine *when the title is
+   safe to publish*, and declaring that per note is the point. No private
+   layer? Remove `privateNotesDir` from the config and the engine runs
+   public-only.
 4. Replace `example-content/` with your corpus and rerun `npm run index`.
 5. Rewrite `eval/gold.yaml` for your corpus — keep the refusals.
 

demo/artifacts.test.ts

@@ -1,8 +1,11 @@
 import assert from 'node:assert/strict';
+import { createHash } from 'node:crypto';
 import { test } from 'node:test';
 
+import { embedText, noteEmbedText } from '../src/corpus.js';
+import { truncateForEmbedding } from '../src/embedding.js';
 import { loadGold } from '../src/evaluate.js';
-import { readIndexFile } from '../src/store.js';
+import { entrySourceId, readIndexFile } from '../src/store.js';
 import { readQueryVectors } from './query-vectors.js';
 
 const EXPECTED_NATURAL_SOURCES = [
@@ -39,6 +42,27 @@ test('committed demo index matches the public-domain source allowlist', () => {
   );
 });
 
+test('committed demo hashes match the current embed-text derivation', () => {
+  // Mirrors contentHash in demo/build.ts (that file runs its keyed build on
+  // import, so it can't be imported here). This pins the schema-v3 migration
+  // invariant: the committed vectors were embedded from label+body at v2, and
+  // today's derivation (noteEmbedText = title+body, embedText for records)
+  // must reproduce the exact bytes those hashes were taken over. Drift in the
+  // migration or the derivation fails here, keylessly — not at re-embed time.
+  const hash = (text: string) =>
+    createHash('sha1').update(truncateForEmbedding(text)).digest('hex').slice(0, 16);
+  for (const path of ['demo/corpus/index.json', 'demo/corpus/index.synthetic.json']) {
+    for (const entry of readIndexFile(path)) {
+      const text = entry.sourceType === 'note' ? noteEmbedText(entry.note) : embedText(entry.record);
+      assert.equal(
+        entry.contentHash,
+        hash(text),
+        `contentHash drift for '${entrySourceId(entry)}' in ${path}`,
+      );
+    }
+  }
+});
+
 test('committed demo query vectors match the gold suite ids', () => {
   const gold = [
     ...loadGold('demo/gold.yaml', 'Smith Collection'),

demo/corpus/private/forgiveness-of-sins.md

@@ -1,5 +1,6 @@
 ---
 title: "Forgiveness of Sins sermon"
+label: "Forgiveness of Sins sermon"
 about: https://archive.org/details/forgivenessofsin00smitrich
 locator: "The Forgiveness of Sins, and Other Sermons (1904), sermon I"
 ---

demo/corpus/private/temptation.md

@@ -1,5 +1,6 @@
 ---
 title: "Temptation sermon"
+label: "Temptation sermon"
 about: https://archive.org/details/forgivenessofsin00smitrich
 locator: "The Forgiveness of Sins, and Other Sermons (1904), sermon III"
 ---

demo/corpus/private/word-of-god.md

@@ -1,5 +1,6 @@
 ---
 title: "The Word of God sermon"
+label: "The Word of God sermon"
 about: https://archive.org/details/forgivenessofsin00smitrich
 locator: "The Forgiveness of Sins, and Other Sermons (1904), sermon II"
 ---

demo/corpus/synthetic/syn-amos-justice-margin.md

@@ -1,5 +1,6 @@
 ---
 title: "Private Amos marginalia on divine justice"
+label: "Private Amos marginalia on divine justice"
 about: https://en.wikipedia.org/wiki/George_Adam_Smith
 locator: "study marginalia, Amos"
 synthetic: true

demo/quantize.test.ts

@@ -8,6 +8,7 @@ import assert from 'node:assert/strict';
 import { test } from 'node:test';
 
 import { cosine } from '../src/retrieve.js';
+import { assertPublicSafeField } from '../src/public-safe.js';
 import type { ArchiveRecord, IndexEntry, PrivateNote } from '../src/types.js';
 import type { GoldQuery } from '../src/evaluate.js';
 import { dequantize, levelFor, quantize, requantizeVector } from './quantize.js';
@@ -34,7 +35,15 @@ function makeRecord(id: string, extra: Partial<ArchiveRecord> = {}): ArchiveReco
 }
 
 function makeNote(id: string): PrivateNote {
-  return { id, label: id, url: 'https://en.wikipedia.org/wiki/George_Adam_Smith', locator: 'sermon', text: 'private' };
+  const context = { path: id, privateText: 'private' } as const;
+  return {
+    id,
+    title: id,
+    label: assertPublicSafeField(id, { field: 'label', ...context }),
+    url: 'https://en.wikipedia.org/wiki/George_Adam_Smith',
+    locator: assertPublicSafeField('sermon', { field: 'locator', ...context }),
+    text: 'private',
+  };
 }
 
 function recordEntry(id: string, vector: number[], extra: Partial<ArchiveRecord> = {}): IndexEntry {

docs/scaling-demo/build-handoff.md

@@ -42,7 +42,7 @@ Note the deliberate theme collision: Amos and Micah carry "justice," which Adam
 
 ### Private ledger — George sermons, dir `demo/corpus/private/`
 
-These are **real George minor works**, designated private (a layer assignment, not secrecy). Note frontmatter: `title` (the label that travels — keep it public-safe), `about` (a **real** public George page to route to, e.g. the work's Wikisource/IA page or `https://en.wikipedia.org/wiki/George_Adam_Smith`), `locator` (where the moment lives, e.g. "Forgiveness of Sins (1905), sermon II"). Body: the real sermon text. The id is `note:<slug>`.
+These are **real George minor works**, designated private (a layer assignment, not secrecy). Note frontmatter: `title` (private — embedded for retrieval, never travels), `label` (required since the core's A1 split: the display name that DOES travel; here it can simply repeat the title, since this "private" layer is public-domain by design), `about` (a **real** public George page to route to, e.g. the work's Wikisource/IA page or `https://en.wikipedia.org/wiki/George_Adam_Smith`), `locator` (where the moment lives, e.g. "Forgiveness of Sins (1905), sermon II"). `label` and `locator` must pass the build lint (`assertPublicSafeField` in `src/public-safe.ts`): single line, ≤120 chars, no 5-token run shared with the note body — a bibliographic locator that names the volume is fine (4-token overlap is the calibrated margin; see delta-log row 21). Body: the real sermon text. The id is `note:<slug>`.
 
 | slug (filename) | unit to extract |
 |---|---|