Skip to content

fix(summary-hook): prevent silent capture loss (tracker-first ordering + mid-processing appends) - #84

Open
catShaark wants to merge 1 commit into
supermemoryai:mainfrom
catShaark:fix/stop-capture-data-loss
Open

fix(summary-hook): prevent silent capture loss (tracker-first ordering + mid-processing appends)#84
catShaark wants to merge 1 commit into
supermemoryai:mainfrom
catShaark:fix/stop-capture-data-loss

Conversation

@catShaark

Copy link
Copy Markdown

Problem

Two independent silent data-loss paths in the Stop-hook capture (summary-hook). Both reproduced against a live server with a real 9-turn Claude Code session at regular-user pacing (~10s between turns), by diffing the stored session document against the transcript.

1. Tracker advances before the upload

formatNewEntries / formatSignalEntries call setLastCapturedUuid inside the formatter, before addMemory runs. If the upload then fails (network, 4xx/5xx, rate limit — and the hook intentionally swallows all errors so it never blocks the session), that delta is permanently lost: the next Stop resumes from the already-advanced cursor.

2. Mid-processing appends are dropped server-side

The session document upserts by customId, and the backend only appends a new revision once the previous one has reached a terminal status. A POST that arrives while the document is still queued/extracting/indexing returns 200 but the content is silently discarded. Fast consecutive turns hit this in normal use: in our reproduction, one entire turn vanished from the session document — the POST "succeeded", the tracker advanced, the content was never stored.

Fix

  • The formatters now return { content, cursor } and no longer touch the tracker. The hook calls setLastCapturedUuid(sessionId, capture.cursor) only after addMemory resolves — a failed upload leaves the cursor untouched and the delta retries on the next Stop.
  • Before uploading, the hook does a single status GET on the session document (5s abort budget). If it's still processing, the capture is deferred without advancing the tracker — the delta simply carries over into the next Stop's capture, which is exactly the formatter's existing carry-over semantics for <100-char captures. Status lookup failures fail open (capture proceeds), so this can never block or break capture.

Both changes reuse the existing carry-over mechanism rather than adding retries/queues, so the hook stays fire-and-forget and adds at most one bounded GET per Stop.

Testing

  • npm test — 5/5 pass.
  • Bundles rebuilt with node scripts/build.js (included in the commit; happy to drop them if you prefer bundles rebuilt on your side).
  • Behavior verified end-to-end against a self-hosted server (v0.0.7-rc.2): with the fix, a 9-turn scripted session stores all 9 turns; without it, turn 8 was lost to the mid-processing drop and failed uploads lost their delta permanently.

Marked as draft for maintainer review — particularly whether you'd rather gate the status check behind a setting, and whether failed should also defer rather than proceed.

…ering and mid-processing appends

Two data-loss paths in the Stop-hook capture, both observed live:

1. The uuid tracker advanced inside formatNewEntries/formatSignalEntries,
   BEFORE the upload. Any failed addMemory permanently lost that delta
   (the Stop hook swallows errors by design). The formatters now return
   { content, cursor } and the hook advances the tracker only after the
   capture is persisted, so a failed upload retries on the next Stop.

2. /v3/documents upserts by customId APPEND only once the previous
   revision reaches a terminal status; an append arriving while the
   session document is still processing (queued/extracting/indexing) is
   silently dropped server-side. Reproduced on a healthy server with a
   9-turn session at ~10s pacing: one entire turn vanished from the
   session document while the tracker advanced past it. The hook now
   does a single 5s-bounded status GET first and, if the document is
   still processing, defers WITHOUT advancing the tracker — the delta
   carries over into the next Stop capture. Lookup failures fail open.

Bundles rebuilt (node scripts/build.js); npm test 5/5.
@catShaark
catShaark marked this pull request as ready for review August 5, 2026 09:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant