choiyounggi · choiyounggi · Aug 5, 2026 · Aug 5, 2026 · Aug 5, 2026
@@ -151,9 +151,15 @@ function main() {
   fs.mkdirSync(queueDir, { recursive: true });
   const queueFile = path.join(queueDir, `${sessionId}.jsonl`);
 
+  // Seed the dedupe set from the session queue AND the processed store —
+  // knowledge-flush empties the queue file when it retires rows, and the next
+  // Stop re-parses the unchanged transcript, so without the second source every
+  // flushed insight would be re-queued. Read-only: only the flush writes there.
+  const processedFile = path.join(queueDir, '.processed.jsonl');
   const seen = new Set();
-  if (fs.existsSync(queueFile)) {
-    for (const line of fs.readFileSync(queueFile, 'utf8').split('\n')) {
+  for (const src of [queueFile, processedFile]) {
+    if (!fs.existsSync(src)) continue;
+    for (const line of fs.readFileSync(src, 'utf8').split('\n')) {
       const o = safeJson(line);
       if (o.hash) seen.add(o.hash);
     }

@@ -53,7 +53,12 @@ printf '%s' "$CMD" | grep -Eq -- 'INGEST_REPORT' && IS_FLUSH=1
 
 # Locate the --body-file path referenced by the command. Portable (grep/sed only,
 # no node) so the enforcement path never depends on node being installed.
-BODY_FILE="$(printf '%s' "$CMD" | grep -oE -- "--body-file[= ]+[^ '\"\`]+" | head -1 | sed -E 's/^--body-file[= ]+//')"
+# The path may be bare, or wrapped in single or double quotes (the flush skill's
+# own example quotes it) — accept all three and strip the quotes.
+BODY_FILE="$(printf '%s' "$CMD" \
+  | grep -oE -- "--body-file[= ]+(\"[^\"]+\"|'[^']+'|[^ '\"\`]+)" | head -1 \
+  | sed -E "s/^--body-file[= ]+//" \
+  | sed -E "s/^\"(.*)\"$/\\1/" | sed -E "s/^'(.*)'$/\\1/")"
 
 fail() {
   echo "dev-loop knowledge-flush gate: $1" >&2
@@ -69,8 +74,13 @@ fail() {
 }
 
 [ -n "$BODY_FILE" ] || fail "no --body-file found on the gh pr create command."
-# Expand a leading ~ if present.
-case "$BODY_FILE" in "~"/*) BODY_FILE="$HOME/${BODY_FILE#~/}" ;; esac
+# Expand a leading ~, $HOME, or ${HOME} — the only variable forms the gate can
+# resolve safely; anything else must be written as a literal path.
+case "$BODY_FILE" in
+  "~"/*)        BODY_FILE="$HOME/${BODY_FILE#~/}" ;;
+  '$HOME'/*)    BODY_FILE="$HOME/${BODY_FILE#\$HOME/}" ;;
+  '${HOME}'/*)  BODY_FILE="$HOME/${BODY_FILE#\$\{HOME\}/}" ;;
+esac
 [ -f "$BODY_FILE" ] || fail "body file '$BODY_FILE' does not exist yet."
 
 miss=""

@@ -0,0 +1,52 @@
+# harvest-dedupe-processed
+
+Goal: `hooks/harvest.js` must not re-queue an ★ Insight candidate that a previous
+knowledge-flush already retired to `~/.dev-loop/queue/.processed.jsonl`. Today the
+dedupe set is seeded only from the session's own queue file, so once a flush
+empties that file the next Stop event re-parses the unchanged transcript and
+re-appends the same hash.
+
+Acceptance criteria:
+1. A hash present in `.processed.jsonl` is not appended to the session queue file,
+   even when that queue file is empty or absent.
+2. Existing behavior is preserved: intra-file dedupe, normal harvesting of new
+   candidates, the `~/.dev-loop/repo` recursion guard, silent exit when there is
+   no transcript, and no error ever surfacing to the user.
+3. `bats tests/` passes on the full suite.
+
+Stack: Node.js (CommonJS, no dependencies — `fs`/`path`/`os`/`crypto` only) for the
+hook; bats for tests; CI runs `bats tests/` on ubuntu-latest and macos-latest
+(`.github/workflows/test.yml`).
+
+Baseline: branch `fix/harvest-dedupe-processed` at HEAD `91409d0`, working tree clean.
+
+## Decisions
+
+| # | Decision | Choice | Wiki basis |
+|---|----------|--------|------------|
+| D1 | Where the dedupe set comes from | Seed `seen` from **both** the session queue file and `~/.dev-loop/queue/.processed.jsonl` | backend-common-jobs-idempotent-handlers — "Increment or append that must remain" row: a **processed-messages store keyed by message id**; a hit means already processed → skip. `.processed.jsonl` is that store; `hash` is the id |
+| D2 | Why a re-read is required at all | Treat every Stop as a full re-execution of the same input; the handler must tolerate it | backend-common-jobs-idempotent-handlers directive 1 — "Every handler must tolerate full re-execution from any point it can crash at" |
+| D3 | Processed-store path | `path.join(queueDir, '.processed.jsonl')` — same directory as the session queue files, resolved from `os.homedir()` exactly as `queueDir` already is (`harvest.js:150`) | `[no-wiki]` — fixed by the existing knowledge-flush contract (skills/knowledge-flush/SKILL.md step 5 writes there) |
+| D4 | Hook's access mode to the processed store | **Read-only.** The hook never creates, writes, or prunes `.processed.jsonl`; only knowledge-flush moves rows there | `[no-wiki]` — single-writer keeps the flush the sole owner of retirement |
+| D5 | Corrupt / missing store behavior | Each source is guarded by `fs.existsSync`, and every line goes through the existing `safeJson` (returns `{}` on parse failure) so a corrupt line is skipped, not fatal | backend-common-jobs-idempotent-handlers directive 1 (tolerate re-execution) + the file's existing `safeJson` + top-level `try/catch` convention (`harvest.js:34-40`, `189-193`) |
+| D6 | Test isolation | `HOME="$BATS_TEST_TMPDIR/home"` per test (Node's `os.homedir()` reads `$HOME` on POSIX); transcript and payload files under `BATS_TEST_TMPDIR`; nothing written into the repo or the real `~/.dev-loop` | testing-data-test-data-and-isolation — "Filesystem / temp files" row (fresh per-test temp directory) and "Global config / environment variables" row (set in setup) |
+| D7 | Case set | One normal, one error, and one boundary case per behavior; the bug gets a regression test | testing-quality-minimum-case-set directive 1 and directive 5 |
+| D8 | Red before green | The regression test is written first and **observed failing** on unmodified `harvest.js`; that output is the evidence the test guards the bug | testing-quality-minimum-case-set directive 5 — "first write a regression test that reproduces the bug and fails on the current code" |
+| D9 | What each test asserts | The observable outcome: the **number of lines** in the session queue file and **which hashes** it contains — never "the hook exited 0" alone, since the hook exits 0 unconditionally | testing-quality-minimum-case-set directive 2 + testing-quality-tests-that-cannot-fail (an assertion that cannot detect the defect is not a test) |
+| D10 | Missing `node` in the test environment | Fail the test loudly in `setup()` rather than `skip` — a permanent skip is a test that cannot fail | testing-quality-tests-that-cannot-fail |
+| D11 | Pruning `.processed.jsonl` | **Out of scope**, recorded as follow-up | backend-common-jobs-idempotent-handlers edge case "Dedupe table grows unbounded → prune rows older than the redelivery horizon" — real, but it is user data and the user deferred it |
+
+## Task order
+
+| Task | Depends on | Parallel-ok |
+|------|-----------|-------------|
+| 01-dedupe-against-processed-store | — | — |
+
+One task: the fix and its regression test are one concern (2 files, 4 wiki pages),
+and splitting them would leave a task whose Verify is "the suite is red".
+
+## Follow-up (not in this branch)
+
+- Prune duplicate rows already in `~/.dev-loop/queue/.processed.jsonl` (103 rows /
+  57 unique hashes as measured 2026-08-05). User data — separate, consented pass.
+- Consider a retention horizon for `.processed.jsonl` per D11's wiki edge case.
@@ -0,0 +1,128 @@
+# Task 01: Dedupe harvested insights against the processed store
+
+## Objective
+
+`hooks/harvest.js` seeds its dedupe set from both the session queue file and
+`~/.dev-loop/queue/.processed.jsonl`, so a hash a previous knowledge-flush retired
+is never re-appended — even when the session queue file is empty or absent.
+`tests/harvest.bats` covers this and the behaviors it must not break.
+
+## Wiki pages (read these first, only these)
+
+- wiki/backend/common/jobs/idempotent-handlers.md — use for: why the handler must
+  tolerate full re-execution (directive 1) and that the dedupe key is looked up in
+  a **processed-messages store** before the append (directive 2, the
+  "Increment or append that must remain" row). This is the row that decides D1.
+- wiki/testing/quality/minimum-case-set.md — use for: which cases the test file
+  must hold (directive 1: normal + error + boundary per behavior), what each test
+  asserts (directive 2: observable outcome), and the red-first rule for a bug fix
+  (directive 5).
+- wiki/testing/quality/tests-that-cannot-fail.md — use for: judging that each new
+  assertion can actually detect the defect; and D10 (never make a permanent skip).
+- wiki/testing/data/test-data-and-isolation.md — use for: the "Filesystem / temp
+  files" row (fresh per-test temp directory) and the "Global config / environment
+  variables" row — how to point `HOME` at a per-test directory so the real
+  `~/.dev-loop/queue` is never touched.
+
+## Inputs
+
+- `hooks/harvest.js` (existing, 193 lines). The site to change is the `seen`
+  construction inside `main()` — the block that begins `const seen = new Set();`
+  and ends just before `const repo = path.basename(cwd);`. Anchor by that symbol,
+  not by line number.
+- `hooks/harvest.js` helpers you must reuse, not re-implement: `safeJson(s)`
+  (returns `{}` on parse failure) and the existing `queueDir` /`queueFile`
+  computation.
+- `tests/loop-gate.bats` — the style to match: `setup()` resolves the hook via
+  `${BATS_TEST_DIRNAME}/../hooks/...`, work happens under `BATS_TEST_TMPDIR`, the
+  payload is piped in with `printf`.
+- Decisions that bind you: D1 (seed from both), D3 (`.processed.jsonl` in
+  `queueDir`), D4 (read-only), D5 (existsSync + safeJson), D6 (HOME isolation),
+  D7 (case set), D8 (red first), D9 (assert line count + hashes), D10 (no skip).
+
+## Steps
+
+1. Create `tests/harvest.bats`. In `setup()`:
+   - `HARVEST="${BATS_TEST_DIRNAME}/../hooks/harvest.js"`
+   - `export HOME="${BATS_TEST_TMPDIR}/home"`, `QDIR="$HOME/.dev-loop/queue"`,
+     `mkdir -p "$QDIR"`
+   - `WORK="${BATS_TEST_TMPDIR}/work"`, `mkdir -p "$WORK"` — this is the payload's
+     `cwd` (it must NOT be under `$HOME/.dev-loop/repo`, which the hook skips).
+   - Assert `node` is available and fail the test if not (D10):
+     `command -v node >/dev/null || { echo "node is required for these tests"; return 1; }`
+2. Add a helper that writes a transcript containing one ★ Insight block and runs
+   the hook, so each test differs only in the queue/processed state:
+
+   ```bash
+   _mk_transcript() { # <file> <trigger-text>
+     local body
+     body="★ Insight ─────\ntrigger: $2\ndirective: seed the dedupe set from the processed store too\nwhy: the transcript is re-parsed on every Stop\nevidence: measured 46 duplicate rows\ndomain: testing\n─────"
+     printf '{"message":{"role":"assistant","content":"%s"}}\n' "$body" > "$1"
+   }
+
+   _run_harvest() { # <transcript>
+     printf '{"cwd":"%s","session_id":"s1","transcript_path":"%s"}' "$WORK" "$1" | node "$HARVEST"
+   }
+   ```
+
+   The delimiter run must be U+2500 box-drawing characters (─), matching
+   `BLOCK_RE` in `harvest.js`. Write the `\n` escapes so they land as real
+   newlines inside the JSON string value.
+3. Write these tests (each asserts the observable outcome per D9 — the queue
+   file's line count and, where it matters, the hash it holds):
+
+   | Case | Setup | Assert |
+   |------|-------|--------|
+   | normal — a new candidate is harvested | empty `$QDIR`, no `.processed.jsonl` | `s1.jsonl` has exactly 1 line, and its `hash` field is non-empty |
+   | **regression** — a processed hash is not re-queued | run the hook once, capture `HASH` from `s1.jsonl`, then move that line into `.processed.jsonl` and truncate `s1.jsonl` (this is exactly what a flush does), then run the hook again | `s1.jsonl` has **0 lines** |
+   | preserved — intra-session dedupe still works | run the hook twice with the same transcript and no flush in between | `s1.jsonl` has exactly 1 line |
+   | error — corrupt line in the processed store | put `not json` plus the real processed row in `.processed.jsonl`, truncate `s1.jsonl`, run | hook exits 0 **and** `s1.jsonl` has 0 lines (the corrupt line is skipped, the valid one still dedupes) |
+   | error — no `.processed.jsonl` at all | remove it, empty queue, run | `s1.jsonl` has exactly 1 line (absence must not throw) |
+   | boundary — empty `.processed.jsonl` | `: > "$QDIR/.processed.jsonl"`, empty queue, run | `s1.jsonl` has exactly 1 line |
+   | boundary — transcript holds no ★ block | transcript with a plain assistant line | no `s1.jsonl` is created (or it has 0 lines) |
+   | preserved — recursion guard | payload `cwd` set to `$HOME/.dev-loop/repo` | no `s1.jsonl` is created |
+
+4. Run `bats tests/harvest.bats` on the **unmodified** `hooks/harvest.js` and
+   record the output. The regression row must FAIL and the others must pass
+   (D8). If the regression row passes here, the test does not reproduce the bug —
+   fix the test before touching the hook.
+5. Edit `hooks/harvest.js`: replace the `seen` construction with a loop over both
+   sources, keeping `safeJson` and the `existsSync` guard:
+
+   ```js
+   const processedFile = path.join(queueDir, '.processed.jsonl');
+   const seen = new Set();
+   for (const src of [queueFile, processedFile]) {
+     if (!fs.existsSync(src)) continue;
+     for (const line of fs.readFileSync(src, 'utf8').split('\n')) {
+       const o = safeJson(line);
+       if (o.hash) seen.add(o.hash);
+     }
+   }
+   ```
+
+   Do not change the append target (`queueFile`), the row shape, the recursion
+   guard, or the top-level `try/catch`. Do not write to `processedFile` (D4).
+6. Re-run `bats tests/harvest.bats` — all rows green — then `bats tests/` for the
+   whole suite.
+
+## Deliverables
+
+- `tests/harvest.bats` (new)
+- `hooks/harvest.js` (modified — only the `seen` construction inside `main()`)
+
+## Verify
+
+- `bats tests/harvest.bats` → all tests pass, and the run before step 5 showed the
+  regression test failing (paste both outputs).
+- `bats tests/` → the full suite passes with no new failures against the `91409d0`
+  baseline.
+- `git diff --stat` → exactly the two Deliverable files.
+
+## Out of scope
+
+- Pruning or rewriting `~/.dev-loop/queue/.processed.jsonl` (user data; follow-up
+  in `plan.md`).
+- `hooks/auto-flush.sh`, `hooks/harvest-insights.sh`, any other hook.
+- `skills/knowledge-flush/SKILL.md` and any wiki page.
+- The installed plugin copy under `~/.claude/plugins/cache/` — read-only, never edit.
@@ -0,0 +1,36 @@
+# wiki-audit — Final report (2026-08-05)
+
+질문: "이 wiki는 LLM이 구현 계획을 세울 때 쓰는 프로덕션급 지식 베이스로서,
+시멘틱하게 케이스를 확실히 찾을 수 있고, 내용이 충분·명확하며, harvest→flush
+파이프라인이 재료를 잘 걸러 중복 없는 PR을 만드는가?"
+
+## Axis verdicts
+
+| 축 | 판정 | 근거 |
+|----|------|------|
+| 1. 구조 (기계적 건전성) | **PASS — 결함 0** | 141페이지·13인덱스 전수 스윕: frontmatter/id/related-link/인덱스 양방향/120줄/모호 한정어/소싱/신선도 전부 0건 (findings/structure.md). lint 규칙 2번의 문구-관행 불일치만 이슈화 (#36) |
+| 2. 시멘틱 라우팅 | **PASS with 2 findings** | 15 프로브 중 in-charter 11/13 UNIQUE, out-of-charter 2건 clean MISS. AMBIGUOUS 2건(doc-gate 이중 소유, flaky 이중 소유) → #37 |
+| 3. 카테고리 충분성 | **PASS, additive gaps** | 60개 카테고리 일관·명확. 16개 개발 관점 체크리스트에서 진짜 갭 G1–G6(아키텍처, CORS, API 버저닝, 피처 플래그, 백필, 실시간) + 하위 G7–G10 → #38. 재구조화 불요 |
+| 4. harvest 필터링 | **PASS after fix** | "수집은 관대, 승격은 엄격" 설계 건전. trigger+directive 필수, 30자 하한, 템플릿 에코 차단, 해시 dedup. **processed-store dedup 누락 버그 수정**(red→green 회귀 테스트) |
+| 5. flush 게이트/dedup | **PASS after 3 fixes** | INGEST_REPORT 3섹션 하드 게이트(13 테스트로 고정). 수정: 따옴표 경로 파싱, $HOME 확장, SKILL 리터럴 경로 지침, drop 후보 retire 명시(무한 오토플러시 루프 차단). 잔여 강화 → #39 |
+
+## Fixes applied on this branch
+
+- hooks/harvest.js — dedupe set seeded from `.processed.jsonl` (read-only)
+- hooks/pre-flush-pr-gate.sh — quoted `--body-file` 인식 + `~`/`$HOME`/`${HOME}` 확장
+- skills/knowledge-flush/SKILL.md — 게이트-호환 리터럴 경로 지침(자체 wiki 페이지 인용), step 5 "모든 처리 후보 retire"
+- tests/harvest.bats (8), tests/pre-flush-pr-gate.bats (13) — 전부 green; 독립
+  test-quality-auditor VERDICT: PASS; 전체 스위트 `bats tests/` exit 0 (315 tests)
+
+## Issues filed
+
+- #36 wiki-lint check 2 vs corpus (규칙 정련)
+- #37 라우팅 이중 소유 2건 + 마이너 문구
+- #38 카테고리 갭 G1–G10 (시드 페이지 목록 포함)
+- #39 파이프라인 잔여 강화 (게이트 우회 창, 기계 검증 가능한 dedup 증빙, 수집 상한)
+
+## Overall
+
+프로덕션급 판정: **가깝다 — 구조·내용 규율은 이미 프로덕션 수준이고, 이번에
+수정한 파이프라인 결함 3건이 실사용 신뢰성의 실제 병목이었다.** 남은 것은
+전부 가산적(카테고리 확장, 트리거 문구 정련, 게이트 강화)이며 이슈로 추적된다.