feat(db): Phase 3 staging proof + forward-codify three hybrid RPC bodies the chain never received (#316)

docs/audit/live-drift-forensics-2026-08.md

@@ -833,7 +833,8 @@ Phase 3, and none of it was touched here.
 
 _2026-08-18 (repo-side session; owner decisions D1 codify-as-live and D2 canary exemption in force;
 no production access; two read-only `SELECT`s against staging `ikoiolksxqxfxgiyqpnu` for the (c)
-triage only, authorised in-session, `list_projects` verified the ref first). This supersedes the
+triage, then — once the §2.5 re-measure merged — the authorised staging apply in 3.5; `list_projects`
+verified the ref first and the project id was passed literally on every call). This supersedes the
 playbook's Phase 3 prompt, which assumed repo-ahead canaries the §1.2 dossier ruled out. **No
 function body changed.** `#292` open-PR check before starting: no open PR touched `schema.sql`,
 `supabase/migrations/**`, `drift-manifest.json` or `scripts/check-drift.ts`._
@@ -971,17 +972,105 @@ finding fires but does not name the column; per-column expansion needed the raw
   failures `tests/session-start-hook.test.ts` / `tests/worker-observability.test.ts` are the only
   expected red set on this host).
 
-### 3.5 Staging proof — BLOCKED (not run)
+### 3.5 Staging proof — RUN for the three migrations; a fourth authored from what it found
 
-The staging apply of `20260818110000`/`111000`/`112000` was gated on the Phase 2 re-measure
-(`main` ≥ 195 migrations, `20260818090000` applied to staging). At `main` `4551b6e4d` this file's §2
-carries **no "Re-measure" subsection** and no open PR adds one, so per instruction the staging step
-was **not** started: no `execute_sql` DDL, no `schema_migrations` row, no `check:drift` run against
-staging in this session. When unblocked, the method is Phase 2's: `execute_sql` running each file
-verbatim in one implicit transaction plus an explicit md5-matched history row carrying the repo
-version and name (never `apply_migration`), ref verified before every call; expected result: zero
-`match_*` function mismatches, zero never-created objects, zero table column-set mismatches, and
-exactly one named residual — `document_chunks_content_trgm_idx` (`c3db2960…` vs `8499c3d3…`, 3.3 d).
+_The gate opened when the Phase 2 re-measure (§2.5, PR #2104) merged onto `main` (`f19cf8f60`).
+Owner-authorised staging window (original Phase 3 authorisation), target `Clinical KB Staging`
+`ikoiolksxqxfxgiyqpnu` via the Supabase MCP connector; `list_projects` verified the ref and the
+project id was passed literally on every call; production `sjrfecxgysukkwxsowpy` was never a
+target. Pre-flight before the first write: `current_user postgres`, `total_rows 195`,
+`latest_version 20260818090000`, `no_statements 0`, `documents 0`, `document_chunks 0`, none of the
+three versions present._
+
+**Apply, by the §2.2 method** (each file's content verbatim through `execute_sql`, then an explicit
+`schema_migrations` row carrying the repository version and name in the same call; `apply_migration`
+not used). Faithfulness proof read back from staging — all three md5-identical to the repository files:
+
+```
+version 20260818110000 · name codify_live_rpc_work_mem                  · stmt_count 1 · bytes 3987 · md5 dd5c8c9ea07c17f76a5f219814f8f19d = repo
+version 20260818111000 · name codify_schema_only_indexes_and_triggers   · stmt_count 1 · bytes 3470 · md5 9d02d14e14d7ea07bf257e2dc99adaa6 = repo
+version 20260818112000 · name reconcile_chain_stale_table_columns       · stmt_count 1 · bytes 2631 · md5 ea5f9c6931f85b81613082b4b6fd6a6e = repo
+```
+
+**Drift comparison.** The local env has no staging service-role key, so instead of `check:drift`'s
+network path the comparison was reproduced offline with the same rules: staging returned, per
+object, `md5(<compared fields as jsonb>::text)` for exactly `check-drift.ts`'s `categoryKeys` /
+`comparedFields`, and the manifest side was rendered in PostgreSQL jsonb text form and hashed
+locally (harness in the session scratchpad; 590 of 594 objects hash-equal, which validates the
+renderer). Result against the regenerated manifest (`generated 2026-08-18T08:30:22Z from
+schema.sql 87ac9fc4849e…`; staging `snapshot_version 2`, `migration_history_probe ok`, 0 history
+rows):
+
+```
+Compared 6 extensions, 38 tables, 1 views, 93 functions, 210 indexes, 48 policies, 170 constraints, 26 triggers, 2 storage_buckets against staging.
+UNEXPECTED DRIFT (4):
+  ! [functions] mismatch public.match_document_embedding_fields_hybrid(extensions.vector,text,integer,double precision,uuid[],uuid)
+  ! [functions] mismatch public.match_document_index_units_hybrid(extensions.vector,text,integer,double precision,uuid[],uuid)
+  ! [functions] mismatch public.match_document_memory_cards_hybrid_v2(extensions.vector,text,integer,double precision,uuid[],uuid)
+  ! [indexes] mismatch document_chunks_content_trgm_idx
+```
+
+Against §2.5's 19: the 8 never-created objects are **gone**, the 3 table column-set mismatches are
+**gone**, and 4 of the 7 `work_mem` function mismatches are **gone** (`chunks_hybrid`,
+`chunks_text`, `lookup_chunks_text`, `memory_cards_hybrid` now hash-equal). The trgm index is the
+one residual 3.3 (d) predicted. **Three functions still mismatch, and that is a finding, not
+`work_mem`:** their ACLs and `proconfig` (values and order) now equal the manifest exactly, and
+`memory_cards_hybrid_v2`'s staging hash `5e792e26…` is unchanged from §2.5 even though its
+`work_mem` was already 64MB — so the difference is in the body. `pg_get_functiondef` on staging
+shows the decisive hunk for all three:
+
+```diff
+-      and public.retrieval_owner_matches(owner_filter, d.owner_id)      -- schema.sql = production
++      and (owner_filter is null or d.owner_id = owner_filter)           -- staging (chain-built)
+```
+
+Provenance: `20260712000000_forward_codify_retrieval_owner_matches.sql` recorded that all eight
+primary retrieval RPCs on live already gate ownership through the fail-closed, sentinel-aware
+`retrieval_owner_matches`, and left the byte-perfect body codification to an owner step; later
+migrations (`20260712171500`, `20260714110000`, `20260724120000`) codified five, but
+`match_document_embedding_fields_hybrid`, `match_document_index_units_hybrid` and
+`match_document_memory_cards_hybrid_v2` were never re-created from live — their newest committed
+body is still `20260701140631` with the legacy predicate. §2.3/§2.5 could not see this because the
+`work_mem` diff sat on top of it. **Classification: chain-stale body (production and `schema.sql`
+agree — 3.1 proved manifest hash = live hash for all three); not a production tenancy hole; a
+reproducibility hole for any migrations-only environment.** Per the plan's live-ahead remedy,
+`20260818113000_forward_codify_hybrid_owner_matches_bodies.sql` re-creates the three from
+`schema.sql` verbatim (every `SET` clause restated so proconfig is preserved; ACLs untouched by
+`CREATE OR REPLACE`; identical text on production ⇒ no-op). Its first staging attempt was declined
+by the tool-permission classifier; **applied in a second owner-authorised window the same day**
+(same method, same target, ref verified): the three hashes now equal the manifest and live —
+`bb975485ee3a5776bce4abdc2e3a3cbd`, `d0e277a2f3067f49463b85ac84b33276`,
+`ab87a18bea57612db83428c24c425825` — and the history row read back `md5 d35c199b19915505ca86663d0be2bf4d
+= repo`.
+
+**Deviation found and resolved.** Before PR #2106 was squash-merged, `20260818111000` and
+`20260818112000` gained a `set local lock_timeout = '5s'; set local statement_timeout = '30s';`
+preamble on the PR branch, so the text that reached `main` (`22585b9e…`, `ec154770…`) differed from
+the text staging had recorded and executed (`9d02d14e…`, `ea5f9c69…`; `20260818110000` unchanged,
+`dd5c8c9e…`). The executed DDL is identical (transaction-local timeouts only), so staging's object
+state already equalled what `main` produces; in the second window the two rows' `statements` were
+refreshed to the merged text so the faithfulness proof holds again. Read-back after the window:
+
+```
+20260818110000 codify_live_rpc_work_mem                    stmt_count 1 bytes  3987 md5 dd5c8c9ea07c17f76a5f219814f8f19d = repo
+20260818111000 codify_schema_only_indexes_and_triggers     stmt_count 1 bytes  3875 md5 22585b9eb81becb6c62d94e597b9e172 = repo
+20260818112000 reconcile_chain_stale_table_columns         stmt_count 1 bytes  3062 md5 ec154770ddaee44bcb64d3d2fc4f838a = repo
+20260818113000 forward_codify_hybrid_owner_matches_bodies  stmt_count 1 bytes 12101 md5 d35c199b19915505ca86663d0be2bf4d = repo
+```
+
+**Final staging comparison (same offline reproduction, 594 objects received):**
+
+```
+Compared 6 extensions, 38 tables, 1 views, 93 functions, 210 indexes, 48 policies, 170 constraints, 26 triggers, 2 storage_buckets against staging (snapshot_version 2, probe ok, migration_history rows 0).
+UNEXPECTED DRIFT (1):
+  ! [indexes] mismatch document_chunks_content_trgm_idx
+```
+
+**Zero function mismatches, zero never-created objects, zero table mismatches** — the Phase 3
+target — with the single named residual being the trgm index of 3.3 (d) (staging holds the
+`20260606000000` bare-`content` form `c3db2960…`; canonical `8499c3d3…`), owned by Phase 4.4.
+Staging after the two windows: 199 rows in `schema_migrations`, `no_statements 0`, `documents` /
+`document_chunks` still `0`, no vault secret seeded.
 
 ### 3.6 Production window (NOT authorised in this task — for the coordinator)
 
@@ -991,14 +1080,16 @@ statements) — **not** `supabase migration repair --status applied` or any othe
 path. None of the three Phase 3 migrations ships a validation guard, so per AGENTS.md "Supabase
 project safety" none of them is eligible for history repair; only real execution is authorised here.
 
-| Migration                                                | Effect on production                                       |
-| -------------------------------------------------------- | ---------------------------------------------------------- |
-| `20260818090000_schema_drift_snapshot_history_probe`     | **real change** — `schema_drift_snapshot()` v2 (Phase 6.1) |
-| `20260818110000_codify_live_rpc_work_mem`                | no-op — the ten `work_mem` values already match (3.1)      |
-| `20260818111000_codify_schema_only_indexes_and_triggers` | no-op — all eight objects already exist (3.2)              |
-| `20260818112000_reconcile_chain_stale_table_columns`     | no-op — column and both defaults already as declared (3.3) |
+| Migration                                                   | Effect on production                                          |
+| ----------------------------------------------------------- | ------------------------------------------------------------- |
+| `20260818090000_schema_drift_snapshot_history_probe`        | **real change** — `schema_drift_snapshot()` v2 (Phase 6.1)    |
+| `20260818110000_codify_live_rpc_work_mem`                   | no-op — the ten `work_mem` values already match (3.1)         |
+| `20260818111000_codify_schema_only_indexes_and_triggers`    | no-op — all eight objects already exist (3.2)                 |
+| `20260818112000_reconcile_chain_stale_table_columns`        | no-op — column and both defaults already as declared (3.3)    |
+| `20260818113000_forward_codify_hybrid_owner_matches_bodies` | no-op — identical to the live bodies (3.5; hash proof in 3.1) |
 
-One window covers all four via `db push`; none needs a canary (D2) and none builds an index.
+One window covers all five via `db push`; none needs a canary (D2), none builds an index, none
+changes a body that production does not already run.
 
 ## Phase 4 — Index restoration
 

docs/branch-review-records/374fbb8ab1553ad0ad69733f3456873bf7e30871fdafbd985fd22feaea12279b.record.md

@@ -0,0 +1 @@
+| 2026-08-18 | claude/db-phase3-staging-proof-bodies | 57385d00559ad6d9ec072b01cf97342cb9a2d1cf | db remediation Phase 3 follow-up: staging proof (110000/111000/112000 applied to ikoiolksxqxfxgiyqpnu, md5-verified) + 20260818113000 forward-codify three hybrid RPC bodies verbatim from schema.sql; forensics 3.5; #316 combined update (PR #2111, follows merged #2106) | Reviewed and handed off; staging drift residual after apply = trgm index + three chain-stale bodies, the latter fixed by 20260818113000 (staging apply pending owner permission); no canonical body changed; production window list in PR body | vitest 6 schema/drift files 109 passed; check:migration-role passed; check:outstanding-issues passed; docs:check-links passed; verify:pr-local not re-run for the one-migration follow-up (green on #2106) |

docs/branch-review-records/9d9815f1d88e54d6803eca19580348e2e3d9e209af3725527ade39fc5ad9badd.record.md

@@ -0,0 +1 @@
+| 2026-08-18 | claude/db-phase3-staging-proof-bodies | b5d228ad5ada7bbb82624ab0573894974f3aa232 | db remediation Phase 3 follow-up, final: 20260818113000 applied to staging, 111000/112000 history text refreshed, staging drift = 1 residual (trgm idx); #316 final update (PR #2111) | Reviewed and handed off; staging proof complete — zero function mismatches, zero never-created objects, zero table mismatches, single residual document_chunks_content_trgm_idx (Phase 4.4); production window list unchanged | staging def_hash for the three functions equal manifest/live; four history rows md5 = repo; check:outstanding-issues passed; docs:check-links passed; earlier gates on this branch unchanged |

docs/outstanding-issues-inbox/10e480da-b2e3-4e2d-bfc8-15456bf962c1.json

@@ -0,0 +1,12 @@
+{
+  "version": 2,
+  "id": "10e480da-b2e3-4e2d-bfc8-15456bf962c1",
+  "createdOn": "2026-08-18",
+  "action": "update",
+  "payload": {
+    "id": "#316",
+    "detail": "PHASE 3 (reframed) BUILT AND STAGING-PROVEN 2026-08-18 (PR #2106) — no production access, no canonical (schema.sql/production) function body changed, no hosted value changed; owner decisions D1 codify-as-live and D2 canary exemption applied. (1) SET work_mem codified on all ten match_* RPCs: schema.sql carries the clause on every definition (both blocks for chunks_hybrid and table_facts_text) and migration 20260818110000_codify_live_rpc_work_mem runs ALTER FUNCTION ... SET work_mem per function, versioned after every recreate incl. 20260724120000. Values: 128MB chunks_hybrid, embedding_fields_hybrid, index_units_hybrid, index_units_hybrid_v2; 64MB chunks_text, chunks_text_v2, lookup_chunks_text, memory_cards_hybrid, memory_cards_hybrid_v2, table_facts_text. PROOF: regenerated drift-manifest def_hash for all ten equals the live production def_hash in issue #1963 (run 32051068106) byte-for-byte, e.g. chunks_hybrid 5902c39286335c07714e498ea31513a0, table_facts_text 0ef9a5dfbde03fe6d48d9223e245aa69 — the next production live-drift run reports zero match_* mismatches once the migration is marked applied. (2) Eight never-created objects codified verbatim by 20260818111000_codify_schema_only_indexes_and_triggers (five document_embedding_fields indexes, documents_status_idx, documents_updated_at and ingestion_jobs_updated_at triggers); disjoint from #102; all six indexes stay on search-health-unmonitored-indexes.json (5 accepted-unmonitored, search_tsv_chunk_gin_idx monitor-candidate), required_indexes untouched — Phase 4.4 decides by migration. (3) Triage: document_chunks CHAIN-stale (token_estimate integer not null default 0 in schema.sql/live/app, zero migrations); rag_visual_eval_cases/runs CHAIN-stale (id default bound to extensions.gen_random_uuid because 20260705230000 sets search_path with pg_catalog last; schema.sql and production bind pg_catalog) — both fixed idempotently by 20260818112000_reconcile_chain_stale_table_columns; document_chunks_content_trgm_idx: production's restored definition (coalesce(content,'') form, md5 8499c3d3...) IS canonical = schema.sql = 20260705180000; staging carries the 20260606000000 bare-content form (c3db2960...) — staging-only residual for Phase 4.4 guard + staging rebuild, no escalation. (4) STAGING PROOF RUN after re-measure PR #2104 merged (owner-authorised window, ref ikoiolksxqxfxgiyqpnu verified per call, production never targeted): 110000/111000/112000 applied by the Phase 2 method with md5-identical history rows (dd5c8c9e.., 9d02d14e.., ea5f9c69..); staging 198 rows, no_statements 0, corpus 0. Offline reproduction of check:drift (same categoryKeys/comparedFields, per-object md5; 590/594 hash-equal): the 8 objects, the 3 column-set mismatches and 4 of the 7 work_mem mismatches are GONE. Residual 4: the trgm index (expected) and THREE FUNCTIONS whose ACL+proconfig equal the manifest but whose BODIES differ — embedding_fields_hybrid, index_units_hybrid, memory_cards_hybrid_v2 carry the legacy predicate (owner_filter is null or d.owner_id = owner_filter) on staging where schema.sql/production carry public.retrieval_owner_matches(owner_filter, d.owner_id). Provenance: 20260712000000_forward_codify_retrieval_owner_matches left the bodies to an owner step; five RPCs were later codified, these three never (newest chain body 20260701140631). NOT a production tenancy hole (manifest hash = live hash for all three); a reproducibility hole for migrations-only environments. New migration 20260818113000_forward_codify_hybrid_owner_matches_bodies re-creates the three verbatim from schema.sql (all SET clauses restated; no-op on production); its staging apply is PENDING owner permission (tool-permission classifier declined the hosted CREATE OR REPLACE of protected RPCs); expected to leave the trgm index as the single staging residual. Tooling note: check-drift.ts:192 clips columns diffs to 240 chars so a wide-table column drift never names the column (own P3 queued). PRODUCTION WINDOW NEEDED (not authorised in this task): 20260818090000 (real change, probe v2) + 20260818110000/111000/112000/113000 (all no-ops, state already matches) — one window, no canary, no index build. Evidence: forensics §Phase 3.",
+    "source": "session 2026-08-18 Phase 3 repo-side codification + staging proof (PR #2106)",
+    "baseRowFingerprint": "e03db7a376a6074d0ae25b648139caf82129f019627b20096058dd731a83a8db"
+  }
+}