Design docs: the drift PR #58 missed, and a Grid re-walk now that realignment shipped

documents/BACKLOG.md

@@ -1,7 +1,8 @@
 # DUNavigator Backlog
 
-Last reviewed: 2026-08-05 (§3.3 block tree built; §1.1 archaic status corrected; four
-pre-existing bugs found during that work recorded under Cross-cutting). Rewritten against the Rust
+Last reviewed: 2026-08-24 (§3.3 corrected — the block tree merged, and the private-Y batch has had a
+GUI trigger since PR #47; §1.2 re-verified genuinely open; the `project_report` item closed).
+Earlier passes: 2026-08-23 (design-header audit, PR #58) and 2026-08-05. Rewritten against the Rust
 tree 2026-07-26.
 
 > **This file was rewritten.** The previous version was the **Scala-era** inventory (last reviewed
@@ -48,6 +49,10 @@ Code exists or the design is settled; these are the near-term threads.
 - **Scope:** a GUI trigger for panel genotyping (`genotype_panel_for_subject` is CLI-only today —
   `navigator genotype-panel`), and folding panel batch mode into the project-wide analyze /
   deep-analyze streaming flow with progress.
+- **Re-verified open 2026-08-24:** `genotype_panel_for_subject` still has exactly two callers,
+  itself and `cli.rs:1945`. The **workspace chore table** (`navigator-app/src/maintenance.rs`, PR
+  #47) is now the obvious home — it already answers the identical question for the private-Y batch,
+  and a fourth chore is one row.
 
 ### 1.3 Chromosome painter v2 — tail
 - **Memory:** `chromosome-painter-v2.md`
@@ -135,6 +140,14 @@ here under its old number so that existing references still resolve.
   instances reserve a lease, fetch, realign to CHM13, run the analysis stack, submit signed results,
   and earn capped compute credit. Cross-repo (Navigator worker + AppView coordinator + shared wire
   records). **Depended on 2.1, which has now shipped — this is unblocked for the first time.**
+- **Re-scoped 2026-08-24** — the design's §11 is a reconnaissance refresh against all three repos.
+  Net: cheaper than drafted. The aligner turned out to be **pure Rust**, so there is no OS split and
+  no Windows spike (D1 and §7.3 corrected in place); `App::analyze_biosample` from PR #47 is already
+  the headless per-unit driver the design asked someone to build; and the AppView already resolves
+  ENA at run level and curates it (`EnaClient::run_files` + `du-jobs/crawl_project.rs`). The
+  critical path is the **AppView half** — `grid.work_unit`, the `SKIP LOCKED` claim, the validator.
+  **One decision to settle before P1 code:** `mt_terminal` sits in the agreement digest, but
+  `analyze_biosample` declines to assign mtDNA on CHM13 by design (§11.3).
 
 ### 2.3 Academic / public-dataset (ENA) import
 - **Design:** [`design/academic-ena-import.md`](design/academic-ena-import.md)
@@ -171,14 +184,18 @@ here under its old number so that existing references still resolve.
 - **Status:** **Mostly built.** Per-subject: `ui/descent.rs` draws the root→terminal path
   (YFull-YReport style, Simple and Advanced densities) and `ui/branch.rs` gives a per-marker branch
   report with TSV export. Cohort: the project **block tree** below.
-- **Built, cohort-scoped** — [`design/project-block-tree.md`](design/project-block-tree.md), branch
-  `feat/project-block-tree` (13 commits, unpushed). A project Y **block tree**: induced subtree over
-  the members' terminals, equivalent-SNP blocks, and **candidate branches** inferred from private
-  variants two or more members share — the thing a published tree cannot show. A `ProjectTab::Tree`
-  canvas draws it; clicking a candidate opens the per-carrier read evidence behind it.
+- **Built, cohort-scoped** — [`design/project-block-tree.md`](design/project-block-tree.md), merged
+  to `main` as `4cb9eca` (PR #45, 2026-08-06) and released in `v0.1.0-alpha.16`. A project Y **block
+  tree**: induced subtree over the members' terminals, equivalent-SNP blocks, and **candidate
+  branches** inferred from private variants two or more members share — the thing a published tree
+  cannot show. A `ProjectTab::Tree` canvas draws it; clicking a candidate opens the per-carrier read
+  evidence behind it.
 - **Scope remaining:** a zoomable/searchable *whole-tree* view (this is cohort-scoped by design), and
-  the open items in that doc's §11 — chiefly **no GUI trigger for the private-Y batch** (CLI only,
-  and candidates cannot fire without it) and 162 `:`-suffixed terminals that fall to `unplaced`.
+  the open items in that doc's §11 — chiefly the **untested canvas interaction layer** (all 13 tests
+  drive the pure `layout()` function, which has no input handling; the click bug that made candidate
+  review unreachable shipped because of exactly that gap). The private-Y batch **does** have a GUI
+  trigger now — the chore table from PR #47 — and the 162 `:`-suffixed terminals are closed as
+  won't-fix (§11.5: stale placements against a superseded tree generation, not a naming convention).
 - **Note:** phase 3 grew well past its original scope because private-Y existed for exactly one
   subject workspace-wide. That pulled in a `private-y --project` batch, a **VCF-backed private-Y
   engine** for the majority of members who have no alignment, and an artefact-filter stack. Four
@@ -224,9 +241,10 @@ here under its old number so that existing references still resolve.
     could have answered (50–90 s → 1.3–3.3 s);
   - **`localize` leaked its alignment copies** — cleanup lived in one caller while three created
     them; reached 687 files / **145 GB** and filled the volume mid-run. Now RAII with a refcount.
-- **`project_report` reads every artifact payload** — it still uses the unfiltered
-  `artifact::list_for_alignments`, which pulled gigabytes of `tree-genotype` JSON to read a few small
-  kinds. `artifact::list_for_alignments_of_kind` exists; convert the caller.
+- **`project_report` read every artifact payload — done** (`c0994d0`, 2026-08-23). It used the
+  unfiltered `artifact::list_for_alignments`, which pulled gigabytes of `tree-genotype` JSON to read
+  a few small kinds; it now reads only the five kinds it uses (`PROJECT_REPORT_KINDS`) through
+  `artifact::list_for_alignments_of_kind`.
 - **mtDNA FASTA export** — the Scala app had it; the Rust `export.rs` covers coverage / read-metrics
   / ancestry / mtDNA-variants / IBD-segments / branch / descent / callable-BED / subject-brief
   (TSV + HTML), but **not** FASTA. Carry it over if still wanted. No PDF export exists either.

documents/HaplogroupReconciliation_Implementation.md

@@ -1,5 +1,13 @@
 # Haplogroup Reconciliation Implementation
 
+> **STALE — Scala-era document (header added 2026-08-24).** The types and file paths below were
+> deleted at the Rust cutover (`0dee32c`, 2026-06-19). The **problem it states still holds** — a chip
+> result must not overwrite a WGS result, and calls must be tracked per tree provider — but the
+> mechanism is now `navigator-domain::consensus` (`reconcile` / `ConsensusStatus`) plus
+> `reconciliation::CallProvenance` for caller precedence. Read
+> [`design/MultiSource_Reconciliation.md`](design/MultiSource_Reconciliation.md) and
+> [`design/external-caller-precedence.md`](design/external-caller-precedence.md) instead.
+
 ## Overview
 
 This document describes the multi-run haplogroup reconciliation system implemented to prevent lower-quality results (e.g., chip data) from overwriting higher-quality results (e.g., WGS data), and to properly track results across different tree providers.

documents/YProfile_and_RegionAnnotations.md

@@ -1,5 +1,14 @@
 # Y Chromosome Profile & Region-Aware Haplogroup Reporting
 
+> **Partly stale — Scala-era document (header added 2026-08-24).** Every `.scala` path below refers
+> to code deleted at the Rust cutover (`0dee32c`, 2026-06-19). **Both capabilities it describes were
+> carried into Rust and shipped**: the unified Y profile is `App::build_y_profile` /
+> `cached_y_profile` over the DNA-type-agnostic consensus engine (`navigator-domain::consensus`),
+> surfaced as the Y-profile concordance card and its source-audit modal; region-aware reporting is
+> the `YRegionClass` quality-modifier ladder
+> (`navigator-analysis/src/mask.rs:110`). Read the **concepts** here; do not trust the class names,
+> file paths, or schema. Current designs: [`design/MultiSource_Reconciliation.md`](design/MultiSource_Reconciliation.md).
+
 ## Executive Summary
 
 Decoding-Us Navigator now provides two powerful capabilities for Y chromosome analysis:

documents/design/AncestryAnalysis.md

@@ -1,5 +1,12 @@
 # Ancestry Analysis Design
 
+> **Header added 2026-08-24.** The design body is **Scala-era** and its file paths no longer exist,
+> but the approach it locks in — PCA projection plus a Gaussian mixture, rather than ADMIXTURE — is
+> the one that shipped. The Rust state is recorded in *Implementation status (Rust rewrite)* further
+> down this file, and the panels have since been rebuilt at 200k depth with continental-European
+> populations. Deep (qpAdm) ancestry is a separate design:
+> [`ancient-ancestry-rebuild.md`](ancient-ancestry-rebuild.md).
+
 ## Overview
 
 Population percentage estimation using autosomal DNA (atDNA) with reference panels from 1000 Genomes and HGDP/SGDP. Provides ADMIXTURE-style ancestry breakdowns at sub-continental granularity.

documents/design/DecodingUsTreeProvider.md

@@ -1,5 +1,15 @@
 # DecodingUs haplogroup tree provider — plan
 
+**Status: IMPLEMENTED, and DecodingUs is the default provider** (header added 2026-08-24; this doc
+had no status line of any kind). Both halves of the plan ship: the AppView serves
+`/api/v1/y-tree/full`, and Navigator parses it with `haplo::parse_decodingus_json` behind the
+`YTreeProvider` enum (`navigator-app/src/lib.rs:2636`). All three "Decisions to confirm" resolved —
+**Q1** the separate `/full` endpoint; **Q2** DecodingUs as the default with FTDNA as the fallback
+(`resolve_y_provider`); **Q3** env *and* a persisted setting *and* a Settings-modal dropdown
+(`NAVIGATOR_Y_TREE_PROVIDER` wins, then `AppSettings::y_tree_provider`), which is more than the plan
+asked for. The native-`hs1` no-liftover path is the point of the whole exercise and it works. Y-DNA
+only, as scoped — mtDNA stays on FTDNA.
+
 Last updated: 2026-06-10. Repos: **DUNavigator** (`rust-rewrite`) + **decodingus** AppView
 (`rust-rewrite-foundation`). Both are ours. Source of truth for the DecodingUs tree is the
 **Rust AppView**, not the live `decoding-us.com` site.

documents/design/RustRewrite_Plan.md

@@ -5,8 +5,9 @@ original plan, kept as the architectural rationale.) The legacy ScalaFX app was
 (commit `0dee32c`, 2026-06-19) and lives in git history only; the Rust workspace is `main` and ships
 signed installers. Per-phase status is **not** tracked here — the authoritative sources are
 `documents/design/HANDOFF.md` (orientation), `documents/design/scala-rust-gap-analysis.md` (what the
-Scala app had that Rust did not), and agent memory. §4e (self-referential callable loci) is still
-flagged "not yet built" in place.
+Scala app had that Rust did not), and agent memory. §4e (self-referential callable loci) **was built
+on 2026-06-03** (`04728ba`); its sub-heading and its "Missing" list said otherwise for almost three
+months, while the measured result sat in the same section all along.
 **Date:** 2026-06-01 (plan)
 **Decisions locked:** egui/eframe GUI · purpose-built pure-Rust haploid caller (no GATK/JVM) · shared crates extracted, Navigator in its own Cargo workspace
 
@@ -159,7 +160,7 @@ If the caller's validation lags the rest of the rewrite, the JVM GATK can run as
 proceeds unblocked — explicitly temporary, removed once §4c passes. (This is *not* the
 shipped architecture; the end state is JVM-free.)
 
-### 4e. Self-referential callable loci (design note — not yet built)
+### 4e. Self-referential callable loci (BUILT — `04728ba`, 2026-06-03)
 
 **Idea.** Derive the callable-region mask from *the sample's own alignment* (a per-sample
 callable-loci BED), rather than gating against a fixed external mask. The CallableLoci BED
@@ -190,7 +191,8 @@ mappability track**.
   pass (~13 min observed) to the reliable subset.
 
 **Ingredients in place:** per-position `CallableState` classification (`coverage` walker);
-read/fragment-length estimates (`read_metrics` walker). **Missing:** coalesce CALLABLE
+read/fragment-length estimates (`read_metrics` walker). **Missing *when this was written*** (every
+item but the stored artifact now ships — see *Built + validated* below): coalesce CALLABLE
 runs → BED, the run-length gate, store as a per-alignment artifact, and route it into
 `private_y_variants` / de-novo region restriction / force-call site selection in place of
 external masks. `ExcessiveCoverage` (collapsed-repeat pileups) stays non-callable; the
@@ -207,6 +209,10 @@ BED; UI exposes self-referential / external-BED / none. The haploid **caller** d
 now tech-adaptive too (long reads → `min_depth 2`), mirroring the mask — the two no longer
 disagree.
 
+**The one piece still not done** (re-verified 2026-08-24): the BED is computed on demand and can be
+*exported* (`export::callable_bed`), but it is not persisted as a per-alignment artifact, so each
+consumer recomputes it.
+
 Real-data validation (same individual, GRCh38):
 - *Callable chrY:* WGS229 short-read ~13× → 13.9 Mb (3336 runs); GFX0457637 HiFi ~1.6× →
   1.1 Mb (70 runs ≈ one ~11 kb read each).