Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
| 2026-08-17 | claude/rag-d2-handover-sync | e0db503cb93ec0bc9843c91cc7b84a0e8627ea6e | docs/rag-improvement HANDOVER/COORDINATION sync after S1b canary pair + S1d packet + S4/T4 rows (docs-only) | docs-only; S1b canary root cause recorded; S1d packet added | verify:pr-local docs scope |
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"version": 2,
"id": "e8a5480e-0b0e-4ab1-b7bd-6090699dd7c1",
"createdOn": "2026-08-17",
"action": "add",
"payload": {
"pri": "P2",
"type": "task",
"summary": "S1d: a hedged, cited, low-confidence fast answer collapses to a citation-free provider_source_gap in the finalizer with no extractive recovery — turned the S1b canary red on summary-discharge-guidance",
"detail": "First post-S1b canary run 32038751592 failed one case (summary-discharge-guidance): strong_routine_retrieval; final_quality_gate:provider_source_gap, grounded false, 0 citations, citation_failure_rate 0.0227. Re-run 32039841070 and 3/3 cache-bypassed live probes took the other branch: generation_fallback:generation_quality_failed; source_backed_extractive_fallback, grounded, 4 citations. Live index unchanged (0 documents/chunks/jobs changed between runs); retrieval 36/36 identical; S1b's diff is dosing-class only. Mechanism: in src/lib/rag/rag.ts (~3580-3625) every fast-route failure shape throws into shouldRecoverFastFailureExtractively (fast + strong_routine_retrieval + results -> source-backed extractive answer), but a hedged CITED confidence:low fast answer whose lead misses providerSourceGapLeadPattern and whose body matches the broader gapLikeAnswer regex in finalizeRagAnswerQualityCore (src/lib/rag/rag-extractive-answer.ts ~3822-3845) survives the loop and is converted to a citation-free evidence_gap with NO extractive recovery. Model phrasing alone chooses the branch. Fix: give the finalizer gap conversion the same extractive recovery for fast + routine + results (or detect the broad gap in-loop); keep the citation-free terminal for genuinely empty retrieval; offline fixtures for both. Own PR, RAG impact behaviour change, canary pair, Clinical Governance Preflight. Lands BEFORE S2 (A2/A3 lengthen answers, more hedged prose). Packet: docs/rag-improvement/HANDOVER.md S1d.",
"source": "RAG programme coordinator, S1b canary root-cause 2026-08-17",
"issueUlid": "01M085NVEB2K48464KM44VGVCE"
}
}
39 changes: 21 additions & 18 deletions docs/rag-improvement/COORDINATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,13 +67,13 @@ their status row, and stop. They never merge, never watch CI, never dispatch can
**Wave plan** (Track A is strictly consecutive — shared files and evidence dependencies;
Track B is the parallel lane):

| Wave | Dispatch (one fresh chat per task) | Gate to the next wave |
| ---- | ------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| 0 | **D1** (this docs PR) + **S4** + **S1b** + **#212 T4** in parallel; **C1** post-S1 canary (done) | D1 merged (unblocks R0); S1b merged + canary green (unblocks S1c) |
| 1 | **R0** `issues:reconcile` (serialized) + **G1** (any time, disjoint) + **S1c** (after S1b canary) | S1c merged + canary green |
| 2 | **S2** (+S2b if split) + **S5** + **S6** (S5/S6 after S4 merges) | S2 merged + canary pair + `eval:answer-quality` + Gate E |
| 3 | **S3** + #212 closure | S3 merged |
| 4 | S7+ only after explicit owner decisions (Gate B verdict, experiment appetite) | — |
| Wave | Dispatch (one fresh chat per task) | Gate to the next wave |
| ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| 0 | **D1** (this docs PR) + **S4** + **S1b** + **#212 T4** in parallel; **C1** post-S1 canary (done) | D1 merged (unblocks R0); S1b merged + canary green (unblocks S1c) |
| 1 | **S1c** + **S1d** + **G1** + **S5** + **S6** in parallel (disjoint files); S1c and S1d merged one at a time, each with its own canary pair (R0 reconcile landed as #2045) | S1c and S1d merged + canaries green |
| 2 | **S2** (+S2b if split) + **S5** + **S6** (S5/S6 after S4 merges) | S2 merged + canary pair + `eval:answer-quality` + Gate E |
| 3 | **S3** + #212 closure | S3 merged |
| 4 | S7+ only after explicit owner decisions (Gate B verdict, experiment appetite) | — |

Waves 1–2 of the original plan (S1 + S4, then S2 + S5 + S6) were re-cut on 2026-08-17 after S1
landed: the owner chose **R1 before S2** (S1's residual R1 explains most remaining lithium
Expand Down Expand Up @@ -174,17 +174,20 @@ Always a fresh owner ask, every single time:

## 7. Current state and next actions (update on change)

- **Done:** PR #1895 (guide), PR #1899 (A1 phase 1), PR #1908 (worker handover) — merged
2026-08-13. **PR #2022 (S1)** merged 2026-08-17 as squash `2bd146eed`, landing verified by
content; canary pair baseline run 31964560921 (`8f8d111ab`) → post run 32025082010
(`2bd146eed`): recall 1.0/1.0, zero per-case rr regressions, answer gate 45/45. Sibling
`#212` T3 (PR #2023) merged as `440a34f71`.
- **Owner decisions 2026-08-17:** R1 before S2; governance Option B.
- **Next (Wave 0):** dispatch S4, S1b and #212 T4 as three fresh chats; merge this docs PR;
then R0 reconcile. G1 may start any time. Prompts: HANDOVER §7 (S1b, S1c, G1, S4) and the
#212 handover artifact (T4).
- **Waiting on owner:** merges as PRs open; S1b's canary approval when it merges; later S2's
canary + `eval:answer-quality` + Gate E.
- **Done (2026-08-17):** #1895, #1899, #1908 (2026-08-13); **#2022 S1** (squash `2bd146eed`);
**#2024 D1** (`78fe906b8`); **#2035 S1b** (merge `92f7618c0`); **#2036 S4** (`f5b093291`);
**#2037 #212 T4** (`1726537b7`); **#2045 R0 reconcile** (28 requests; #212 closed). Canary
pairs: S1 baseline 32025082010 → S1b post 32039841070, recall 1.0/1.0, zero per-case rr
regressions, answer gate 45/45. The first S1b post run (32038751592) was red on one non-golden
case; root-caused to the finalizer gap-recovery hole → packet S1d, not S1b.
- **Owner decisions:** R1 before S2; governance Option B; S1d lands before S2.
- **Wave 1 (dispatch now, parallel, disjoint files):** S1c (`rag-claim-support.ts`), S1d
(`rag-extractive-answer.ts` finalizer / `rag.ts` fast-failure block), G1 (`types.ts`,
`answer-stream-contract.ts`, `rag-row-contracts.ts`), S5 (harness/telemetry), S6 (`eval/docling/`).
Prompts: HANDOVER §7. Merge order for the RAG-behaviour ones: S1c and S1d each get their own
canary pair; merge one, run its canary, then the next.
- **Wave 2:** S2 (+S2b) after S1c and S1d canaries are green. **Wave 3:** S3; S7+ owner decisions.
- **Waiting on owner:** merges as PRs open; canary approvals for S1c, S1d, S2.
- **Live board (artifact, owner-private):** RAG Master Plan v2 —
`https://claude.ai/code/artifact/d5dba709-0df3-40e3-8a45-15997231533d`.

Expand Down
Loading
Loading