feat(learn): attribute lesson recurrence from session corrections

docs/docs/configure/learn.md

@@ -222,7 +222,21 @@ altimate-code learn disable
 
 ### status
 
-Shows whether learning is on, whether automatic promotion is on with its last promotion and last held-back reason, lesson counts (approved, candidate, retired), open signals, pending recoveries, the last reflection and its summary, and the resolved limits. `--json` also includes the last reflection's token use and estimated cost.
+Shows whether learning is on, whether automatic promotion is on with its last promotion and last held-back reason, lesson counts (approved, candidate, retired), open signals, pending recoveries, the last reflection and its summary, the resolved limits, and lesson recurrence. `--json` also includes the last reflection's token use and estimated cost.
+
+**Recurrence** answers "was this lesson shown, and did the user still correct the same thing?". It reads the newest 200 session snapshots and the recorded corrections. It makes no model calls and writes nothing. For each shown lesson it counts sessions where a matching correction came:
+
+| Count | Meaning |
+|---|---|
+| `recurred` | after the lesson was shown, before the agent started answering the corrected request |
+| `during` | while the agent was answering, for example a file-hook lesson added mid-turn; the lesson may have arrived too late |
+| `unknown` | in an order that cannot be told, for example older corrections or a request replayed after compaction |
+| `ambiguous` | when the correction matched more than one lesson the session was shown in its current text |
+| `earlier_version` | when the session was shown only an earlier version of the lesson text; it never counts for the current text |
+
+A correction matches a lesson when it names one of the lesson's full backtick identifiers plus another word from the lesson, or contains at least 3 of the lesson's words and 60% of them. A prefix or suffix identifier such as `_at` or `stg_` never matches on its own. Lessons first shown after the correction are ignored for it. The text output lists only lessons with a matching correction and summarizes the rest as "no observed correction". That is not proof the agent followed them. Corrections are recorded only while capture is on. Status also lists up to 5 identifiers that were corrected in 2 or more sessions with no matching current lesson. They are taken from your local corrections and never leave the project. If the analysis fails, status shows `Recurrence unavailable` (`recurrence_error` in JSON), and the rest of the status still prints.
+
+When the current text of a lesson has `recurred` in 2 or more sessions, reflection marks it `r:N` in the playbook it sends the model. The model is asked to make the lesson clearer or more specific when the feedback shows the wording is unclear, and to keep it when the agent simply did not follow a correct rule. It may narrow a rule only when the feedback shows the rule is too broad, and must keep the behavior the rule requires. The result is a candidate like any other and goes through the same review or automatic-promotion gates.
 
 ```bash
 altimate-code learn status

packages/opencode/src/altimate/learn/capture.ts

@@ -92,6 +92,39 @@ export interface MessageLike {
   time?: { created?: number; completed?: number }
 }
 
+export type TurnMessage = MessageLike & { synthetic?: boolean; compaction?: boolean; replay?: boolean }
+
+type TurnPart = { type: string; synthetic?: boolean; metadata?: Record<string, unknown> }
+
+/**
+ * Session history as turn boundaries. All-synthetic text continues a turn unless the message carries user input
+ * (an attachment or agent mention, whose text companions are synthetic). Compaction and its replay are marked.
+ */
+export function turnMessages(messages: readonly { info: MessageLike; parts: readonly TurnPart[] }[]): TurnMessage[] {
+  return messages.map(({ info, parts }) => {
+    const text = parts.filter((part) => part.type === "text")
+    return {
+      ...info,
+      synthetic: text.length > 0 && text.every((part) => part.synthetic) && !parts.some((part) => part.type === "file" || part.type === "agent"),
+      compaction: parts.some((part) => part.type === "compaction"),
+      replay: text.some((part) => typeof part.metadata?.learnOriginalMessage === "string"),
+    }
+  })
+}
+
+/**
+ * Start of the agent's response to the request the correction answers. Synthetic continuations are not
+ * requests. A request right after a compaction may be its replay, so its turn start is unknown.
+ */
+export function correctedTurnStart(messages: readonly TurnMessage[], messageID: string): number | undefined {
+  const end = messages.findIndex((message) => message.id === messageID && message.role === "user")
+  if (end < 0) return
+  const previous = messages.slice(0, end).findLastIndex((message) => message.role === "user" && !message.synthetic && !message.compaction)
+  if (previous < 0 || messages[previous].replay) return
+  if (messages.slice(0, previous).findLast((message) => message.role === "user")?.compaction) return
+  return messages.slice(previous + 1, end).find((message) => message.role === "assistant")?.time?.created
+}
+
 export interface PartLike {
   id: string
   sessionID: string
@@ -107,13 +140,15 @@ export interface PartLike {
 export interface CaptureDeps {
   /** True when the session has a completed assistant message before the user message being classified. */
   hasPriorAssistant: (sessionID: string, beforeMessageID: string) => Promise<boolean>
+  turnStart?: (sessionID: string, messageID: string) => Promise<number | undefined>
   record: (signal: NewSignal) => Promise<Signal | undefined>
 }
 
 export class Capture {
   private messages = new Map<string, MessageLike>()
   private pendingParts = new Map<string, PartLike>()
   private sessionsWithAssistant = new Map<string, number>()
+  private corrected = new Set<string>()
   private trackers = new Map<string, ToolRetryTracker>()
   private pending = new Set<Promise<unknown>>()
 
@@ -170,16 +205,25 @@ export class Capture {
     if (message.role !== "user") return
     const reason = correctionReason(part.text)
     if (!reason) return
+    // Later updates of a recorded correction deduplicate to its first signal; skip their history reads.
+    if (this.corrected.has(part.messageID)) return
     const prior = this.sessionsWithAssistant.get(part.sessionID)
     const known = prior !== undefined && message.time?.created !== undefined && prior < message.time.created
     if (!known && !(await this.deps.hasPriorAssistant(part.sessionID, part.messageID))) return
+    const start = await this.deps.turnStart?.(part.sessionID, part.messageID).catch((error) => {
+      log.warn("learn turn start unavailable", { error })
+      return undefined
+    })
     await this.deps.record({
       kind: "user_correction",
+      ...(start !== undefined && Number.isFinite(start) ? { turnStartedAt: new Date(start).toISOString() } : {}),
+      ...(message.time?.created !== undefined && Number.isFinite(message.time.created) ? { correctedAt: new Date(message.time.created).toISOString() } : {}),
       sessionID: part.sessionID,
       messageID: part.messageID,
       text: part.text,
       reason,
     })
+    remember(this.corrected, part.messageID, RECENT_ID_CAP)
   }
 
   private async toolPart(part: PartLike) {
@@ -239,6 +283,10 @@ export async function startCapture(ctx: { directory: string; worktree: string })
     const { SessionID } = await import("../../session/schema")
     const capture = new Capture({
       record: (signal) => appendSignal(root, signal),
+      turnStart: async (sessionID, messageID) => {
+        const messages = await Session.messages({ sessionID: SessionID.make(sessionID) })
+        return correctedTurnStart(turnMessages(messages), messageID)
+      },
       hasPriorAssistant: async (sessionID, beforeMessageID) => {
         const messages = await Session.messages({ sessionID: SessionID.make(sessionID) })
         const before = messages.findIndex((message) => message.info.id === beforeMessageID)

packages/opencode/src/altimate/learn/delivery.ts

@@ -79,6 +79,44 @@ async function read(root: string, file: string) {
   }
 }
 
+export const lessonTextHash = (text: string) => Store.sha256(singleLine(text))
+export type ShownSnapshot = {
+  session: string
+  shown: { name: string; id: string; textHash: string; tier: Shown["tier"]; at: string }[]
+}
+
+export async function readShownSnapshots(root: string, options: { limit?: number } = {}) {
+  const dir = path.join(root, ".altimate-code", "learn", ".sessions")
+  const coverage = { sessions_analyzed: 0, sessions_skipped_malformed: 0, truncated: false }
+  const snapshots: ShownSnapshot[] = []
+  await SafeFS.assertSafePath(root, dir)
+  const entries = await fs.readdir(dir).catch((error: NodeJS.ErrnoException) => {
+    if (error.code === "ENOENT") return []
+    throw error
+  })
+  const files = await Promise.all(entries.filter((name) => name.endsWith(".json")).map(async (name) => {
+    const file = path.join(dir, name)
+    return { file, mtime: await fs.lstat(file).then((stat) => stat.mtimeMs).catch(() => 0) }
+  }))
+  const limit = Math.max(0, Math.floor(options.limit ?? 200))
+  coverage.truncated = files.length > limit
+  for (const { file } of files.sort((a, b) => b.mtime - a.mtime || a.file.localeCompare(b.file)).slice(0, limit)) {
+    try {
+      const state = State.parse(JSON.parse((await read(root, file)) ?? ""))
+      if (path.basename(file) !== Store.sha256(state.session) + ".json") throw new Error("Learn session state belongs to another session")
+      snapshots.push({ session: state.session, shown: state.shown.map(({ name, lesson, tier, at }) => ({
+        name, id: lesson.id, textHash: lessonTextHash(lesson.text), tier, at,
+      })) })
+    } catch (error) {
+      if (error instanceof SafeFS.UnsafeLearnPathError) throw error
+      coverage.sessions_skipped_malformed++
+      log.warn("learn session snapshot skipped", { file, error })
+    }
+  }
+  coverage.sessions_analyzed = snapshots.length
+  return { snapshots, coverage }
+}
+
 /** Persisted text snapshots keep the prefix stable even after edits, ranking changes or a restart. */
 export class Delivery {
   private resolved?: Limits

packages/opencode/src/altimate/learn/prompt.txt

@@ -16,8 +16,8 @@ How to work:
 2. Infer general conventions. State one rule with the exact identifier it governs, without rationale or incident details. Keep the specific table, model, column, or file identifier when the correction depends on it; do not substitute a vague generality.
 3. Learn only from the external feedback. Do not treat the agent's own claims of success as evidence.
 4. For an existing bullet:
-   - HELPFUL when the feedback shows the agent followed it and the related checks passed.
-   - HARMFUL only with explicit evidence that the agent FOLLOWED the bullet and that doing so caused a failure or a reviewer objection. The agent not following a bullet, feedback that is unrelated to the bullet, or feedback that asks for exactly what the bullet says is never HARMFUL (the last is HELPFUL). If unsure, leave the bullet alone.
+   - HELPFUL only when the DIGEST shows the agent followed it and the feedback shows the related checks passed. When the feedback asks for exactly what the bullet says because the agent did not follow it, the bullet is right but was missed: it is neither HELPFUL nor HARMFUL. Leave its counters alone, and EDIT it only if its wording is unclear.
+   - HARMFUL only with explicit evidence that the agent FOLLOWED the bullet and that doing so caused a failure or a reviewer objection. The agent not following a bullet, feedback that is unrelated to the bullet, or feedback that asks for exactly what the bullet says is never HARMFUL. If unsure, leave the bullet alone.
    - When feedback states a different expected behaviour than a bullet, EDIT the bullet to that expected behaviour (or ADD with "supersedes"). HARMFUL alone loses the knowledge. Keep the corrected convention general.
    - REMOVE when the feedback shows it is plainly wrong or outdated.
 5. Before ADD or EDIT, compare its code identifiers, naming prefixes and suffixes in backticks with every existing bullet. If another bullet touches the same convention, explicitly resolve their relationship: EDIT the existing rule, REMOVE the outdated rule, ADD a replacement with "supersedes": "<existing id>", or ADD a compatible narrower rule with "coexists": ["<existing id>", ...]. For example, a rule that analyses keep integer `*_cents` columns can coexist with a rule that staging models convert `*_cents` columns to dollars. EDIT also needs "coexists" for every other overlapping bullet it remains compatible with. Refer only to existing bullet ids. Two bullets that disagree must never both remain. When feedback contradicts a bullet, that bullet is outdated: supersede or remove it; do not merely ADD a conflicting rule or mark the old one HARMFUL.
@@ -35,3 +35,5 @@ Bullet text rules (deterministic checks reject unsafe content and flag verificat
 
 Output: a JSON object {"deltas": [...]}. Each delta is {"op": "ADD"|"EDIT"|"REMOVE"|"HELPFUL"|"HARMFUL", "id": "<bullet id, for every op except ADD>", "text": "<bullet text, for ADD and EDIT>", "reason": "<one short sentence citing the feedback>"}.
 Optional fields: "supersedes": "<existing bullet id>" for ADD, and "coexists": ["<existing bullet id>", ...] for ADD or EDIT. Omit these fields when they do not apply.
+
+r = sessions where this rule was shown and the user still corrected the same topic. If the feedback shows the wording is unclear or incomplete, EDIT it to be clearer or more specific. If the agent simply did not follow a correct rule, keep it. Narrow a rule only when the feedback shows it is too broad; never drop the behavior it requires.