Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
ce58656
docs(ste): the checker and the project dictionary for Simplified Tech…
JamesKane Aug 18, 2026
22fc129
docs(ste): the mechanical vocabulary substitutions, across every crate
JamesKane Aug 18, 2026
95db9eb
docs(ste): navigator-resource and sig_cache, converted in full
JamesKane Aug 18, 2026
ab34476
docs(ste): navigator-app, the first thirteen files
JamesKane Aug 18, 2026
15ae0f9
docs(ste): navigator-app, ten more files
JamesKane Aug 18, 2026
19cc8c4
docs(ste): navigator-app, the scientific rationale files
JamesKane Aug 18, 2026
b88b34b
docs(ste): navigator-app, the pipeline and import files
JamesKane Aug 19, 2026
b407ed1
docs(ste): analysis.rs to zero, and lib.rs started
JamesKane Aug 19, 2026
8e97248
docs(ste): the conversion recipe, and the tool it depends on
JamesKane Aug 19, 2026
6ef7910
docs(ste): lib.rs to zero, and haplogroup.rs halfway
JamesKane Aug 19, 2026
867f01b
docs(ste): navigator-app to zero
JamesKane Aug 19, 2026
20787e7
docs(ste): navigator-sync to zero
JamesKane Aug 19, 2026
130b1a6
docs(ste): navigator-refgenome to zero
JamesKane Aug 20, 2026
77a2fcf
docs: put eleven stranded doc comments back on the items they describe
JamesKane Aug 20, 2026
2daecb0
docs(ste): navigator-store to zero
JamesKane Aug 20, 2026
a02b632
docs(ste): navigator-panelbuild to zero
JamesKane Aug 20, 2026
f84ea58
docs(ste): ancestry.rs and haplo.rs to zero
JamesKane Aug 20, 2026
8fdf714
docs: put locus_carried's doc back on locus_carried
JamesKane Aug 20, 2026
6e72f88
docs(ste): archaic.rs, caller.rs and archaic_match.rs to zero
JamesKane Aug 20, 2026
0358609
docs(ste): lai.rs and unified.rs to zero
JamesKane Aug 20, 2026
70c98e0
docs(ste): preflight, coverage, archaic_segments and reader to zero
JamesKane Aug 20, 2026
5579643
docs(ste): reassembly.rs and sv/walker.rs to zero
JamesKane Aug 20, 2026
160cf6b
docs(ste): postprocess/tests.rs and gvcf.rs to zero
JamesKane Aug 20, 2026
9376e28
docs(ste): ibd_panel.rs and revert/mod.rs to zero
JamesKane Aug 20, 2026
168b66f
docs(ste): the whole revert/ module to zero
JamesKane Aug 20, 2026
2b61804
docs(ste): the whole postprocess/ module, and index.rs, to zero
JamesKane Aug 20, 2026
ff3b9c4
docs(ste): mask.rs, roh.rs and scan.rs to zero
JamesKane Aug 20, 2026
eb18075
docs(ste): readview.rs, sex.rs and testtype.rs to zero
JamesKane Aug 20, 2026
09131aa
docs(ste): strcaller, strmarker, cancel, phasing to zero
JamesKane Aug 20, 2026
c6fb90a
docs(ste): reassembly_probe, sidecar.rs and mastervar.rs to zero
JamesKane Aug 20, 2026
a208363
docs(ste): heteroplasmy, realign, gzio, error, parity_real, probe, st…
JamesKane Aug 20, 2026
cbf1f2f
docs(ste): nine more files to zero
JamesKane Aug 21, 2026
0153981
docs(ste): the remaining 37 files — navigator-analysis to zero
JamesKane Aug 21, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions crates/navigator-align/src/batch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ impl BatchSize {

/// Why [`BatchSize::for_this_machine`] chose what it did, for a log line or a UI tooltip.
///
/// A realignment is a multi-hour job whose memory profile the user cannot see; when it is
/// A realignment is a multi-hour job whose memory profile the user can not see; when it is
/// sized automatically, the sizing has to be inspectable rather than a mystery.
pub fn explain() -> String {
if let Some(bases) = env_override() {
Expand Down Expand Up @@ -269,7 +269,7 @@ mod tests {

/// Detection has to work on whatever machine this runs on — that is the entire point of taking
/// the dependency. The assertions are about plausibility rather than a specific number, since
/// the test cannot know the host.
/// the test can not know the host.
#[test]
fn the_machine_reports_its_own_memory() {
let memory = detect_memory().expect("every desktop target sysinfo supports reports memory");
Expand Down
4 changes: 2 additions & 2 deletions crates/navigator-align/src/index.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,9 @@ pub type ProgressFn<'a> = &'a mut dyn FnMut(usize, u64);
/// Deliberately the same answer `navigator-refgenome::cache::base_dir` gives, reached the same way
/// — through `navigator_domain::paths::decodingus_dir`, the one definition of the cache root — so
/// `minimap2_index/` lands beside `references/` and `liftover/` rather than in a second location
/// that only this crate knows about. This crate is a leaf and cannot depend on `navigator-refgenome`
/// that only this crate knows about. This crate is a leaf and can not depend on `navigator-refgenome`
/// (that would invert the layering), which is why the resolution is repeated rather than imported;
/// the shared *definition* is what stops the two drifting.
/// the shared *definition* is what keeps the two the same.
pub fn cache_root() -> PathBuf {
if let Some(dir) = std::env::var_os("NAVIGATOR_REFGENOME_DIR") {
return PathBuf::from(dir);
Expand Down
4 changes: 2 additions & 2 deletions crates/navigator-align/src/map.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
//! ## Why not the upstream file-level entry points
//!
//! `minimap2-pure-rs` ships `map_file_sam_split` and friends, which look like exactly this. They
//! cannot be used: they write to **stdout** (unusable from a desktop app) and they take
//! can not be used: they write to **stdout** (unusable from a desktop app) and they take
//! `parts: &[MmIdx]`, holding every part resident — giving up the entire memory bound this design
//! exists to buy. What is reused is the per-part record format and the merge; the loop is ours.
//!
Expand Down Expand Up @@ -113,7 +113,7 @@ pub struct MapStats {

/// Cancellation, as a callback rather than a shared token type.
///
/// This crate is a leaf — it deliberately does not depend on `navigator-analysis`, so it cannot
/// This crate is a leaf — it deliberately does not depend on `navigator-analysis`, so it can not
/// take that crate's `CancelToken` without inverting the layering. A closure lets the caller wire
/// whatever cancellation it already has, and costs this crate no dependency.
pub type CancelFn<'a> = &'a dyn Fn() -> bool;
Expand Down
2 changes: 1 addition & 1 deletion crates/navigator-align/src/map/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -590,7 +590,7 @@ fn the_output_format_can_be_read_off_the_path() {
assert_eq!(F::from_path(Path::new("x")), F::Bam, "BAM is the default");
}

/// CRAM cannot be written without the reference it is compressed against, and saying so up front
/// CRAM can not be written without the reference it is compressed against, and saying so up front
/// beats failing partway through a multi-hour job.
#[test]
fn cram_without_a_reference_is_refused_before_any_work() {
Expand Down
2 changes: 1 addition & 1 deletion crates/navigator-align/src/output.rs
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ impl AlignmentWriter {

/// Parse one SAM line from the mapper and hand it to `edit` before writing.
///
/// `edit` is where paired fields get set. It sees a typed record, so it cannot write a value
/// `edit` is where paired fields get set. It sees a typed record, so it can not write a value
/// into the wrong column — which was the entire failure mode this module removes.
pub fn write_line_with(
&mut self,
Expand Down
4 changes: 2 additions & 2 deletions crates/navigator-align/src/pe.rs
Original file line number Diff line number Diff line change
Expand Up @@ -707,7 +707,7 @@ fn primary(result: &MapResult) -> Option<&AlignReg> {
/// Fill in the paired half of a record: flags, `RNEXT`, `PNEXT`, `TLEN`.
///
/// The single-end writer produced everything else. This used to patch the formatted SAM text by
/// column position; it now mutates a typed [`RecordBuf`], so a mate position cannot end up in the
/// column position; it now mutates a typed [`RecordBuf`], so a mate position can not end up in the
/// template-length field however the formatter's layout changes.
///
/// `own` is this record's region (`None` for an unmapped read) and `mate` is the mate's primary.
Expand Down Expand Up @@ -822,7 +822,7 @@ impl PairReader {
/// read is the same length.
///
/// A file genuinely ending before the other is still an error rather than a truncation: R1/R2
/// that have drifted out of step would pair every later read with the wrong mate, which is far
/// that have lost their order would pair every later read with the wrong mate, which is far
/// worse than refusing to run.
fn next_batch(&mut self, chunk: i64) -> Result<Option<Vec<(BseqRecord, BseqRecord)>>, AlignError> {
let mut batch = Vec::new();
Expand Down
2 changes: 1 addition & 1 deletion crates/navigator-align/src/pe/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -372,7 +372,7 @@ fn a_split_index_pairs_reads_exactly_as_a_whole_index_does() {
}
}

/// R1/R2 that have drifted out of step would pair every later read with the wrong mate — a
/// R1/R2 that have lost their order would pair every later read with the wrong mate — a
/// corruption that produces confident, wrong alignments. It must refuse rather than truncate.
#[test]
fn mismatched_read_counts_are_refused() {
Expand Down
9 changes: 5 additions & 4 deletions crates/navigator-analysis/examples/archaic_callable_dump.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
//! Dump the Tier B callability mask as BED, so what the segment caller can and cannot see is
//! checkable against an external callset rather than assumed.
//! Write out the Tier B callability mask as a BED file. Somebody can then check what the segment
//! caller can see, and what it can not, against an external callset. Nobody has to assume it.
//!
//! Windows below `min_frac` of `window_bp` callable are excluded by the caller itself, so the same
//! threshold is applied here — the output is the territory a segment could actually be called in.
//! The caller itself removes a window whose callable fraction is below `min_frac` of `window_bp`.
//! This tool applies the same threshold. So its output is the territory where a segment could get
//! a call.
//!
//! ```sh
//! cargo run --release -p navigator-analysis --example archaic_callable_dump -- \
Expand Down
13 changes: 8 additions & 5 deletions crates/navigator-analysis/examples/archaic_classify_dump.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
//! Dump the archaic **diagnostic** sites (position, derived base, lineage class) as TSV.
//!
//! Written to test a different observable for the Tier B HMM. The current model counts *all*
//! private variants per window, and that signal is weak: measured on a real European, archaic
//! tracts carry only 2.89x the background density while the background itself varies 5.3x between
//! its 10th and 90th percentile. Restricting the observable to sites where the derived allele is
//! actually known to be archaic should be far more specific.
//! Somebody wrote this to test a different observable for the Tier B HMM.
//!
//! The current model counts *all* of the private variants in each window, and that signal is weak.
//! A measurement on a real European showed it. An archaic tract carries only 2.89x the background
//! density. And the background itself changes by 5.3x, between its 10th and 90th percentile.
//!
//! Take an observable that holds only the sites where the derived allele is archaic, and that
//! nobody doubts. That observable must be far more specific.
//!
//! ```sh
//! cargo run --release -p navigator-analysis --example archaic_classify_dump -- \
Expand Down
25 changes: 15 additions & 10 deletions crates/navigator-analysis/examples/archaic_match_probe.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
//! Run the reference-based archaic tract caller ([`archaic_match`]) on real cached calls.
//!
//! The unit tests prove the model behaves on synthetic runs; this is what shows whether it finds
//! REAL tracts. Emits segments as JSON for `scripts/archaic-validation/compare_locations.py`, which
//! scores them against an external callset and — critically — against the random-placement null the
//! density caller failed.
//! The unit tests show that the model behaves on a synthetic run. This tool shows whether it finds
//! a REAL tract.
//!
//! It writes the segments as JSON, for `scripts/archaic-validation/compare_locations.py`. That
//! script scores them against an external callset, and, most important, against the null from
//! random placement that the density caller failed.
//!
//! ```sh
//! cargo run --release -p navigator-analysis --example archaic_match_probe -- \
Expand Down Expand Up @@ -46,8 +48,8 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut observations: BTreeMap<String, Vec<SiteObs>> = BTreeMap::new();
let mut lengths: Vec<(String, i32)> = Vec::new();
for (contig, pos_map) in &by_contig {
// The reference base decides which diagnostic sites are informative at all, so it is read
// rather than assumed (see `observations_for_contig`).
// The reference base decides which diagnostic sites carry information at all. So the code
// reads it, and it does not assume it. See `observations_for_contig`.
let seq = read_contig_sequence(&reference, contig)?;
let obs = observations_for_contig(
contig,
Expand Down Expand Up @@ -83,10 +85,13 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
GeneticMap::from_bytes(&std::fs::read(&a[3])?).map_err(|e| e.to_string())?
};

// `ARCHAIC_RATIOS=2.0,2.5,3.04` sweeps the emission ratio in one process. It cannot be swept
// post-hoc like the three thresholds — it changes the emissions, so the HMM must be re-decoded —
// but the expensive part (reading the reference, walking the diagnostic sites) is per sample,
// not per ratio, so doing it here costs one pass instead of one per value.
// `ARCHAIC_RATIOS=2.0,2.5,3.04` sweeps the emission ratio inside one process. Nobody can sweep
// that ratio after the run, as they can the three thresholds, because it changes the emissions
// and the HMM must decode again.
//
// But the costly part happens once for each sample, and not once for each ratio. That part is
// the read of the reference, and the walk over the diagnostic sites. So a sweep here costs one
// pass, and not one pass at every value.
if let Ok(spec) = std::env::var("ARCHAIC_RATIOS") {
let mut out = serde_json::Map::new();
for tok in spec.split(',').filter(|t| !t.trim().is_empty()) {
Expand Down
25 changes: 15 additions & 10 deletions crates/navigator-analysis/examples/archaic_outgroup_density.rs
Original file line number Diff line number Diff line change
@@ -1,15 +1,20 @@
//! Per-window counts of African-outgroup segregating sites — a candidate local mutation-rate proxy.
//! The count of sites that vary in the African outgroup, in each window. It is a candidate proxy
//! for the local mutation rate.
//!
//! The Tier B emission model assumes one background rate genome-wide. Measured, the background
//! private-variant density varies 5.3x between its 10th and 90th percentile and is 14.6x
//! overdispersed relative to the Poisson it is modelled with, which is larger than the 2.89x
//! enrichment inside real archaic tracts — so the model calls its own upper tail archaic. hmmix
//! avoids this with a mutation-rate map; we have no such asset.
//! The Tier B emission model takes one background rate over the whole genome. A measurement shows
//! otherwise. The background density of private variants changes by 5.3x between its 10th and 90th
//! percentile. It is also 14.6x more spread out than the Poisson distribution that the model gives
//! it.
//!
//! The density of sites segregating in Africans is already in `archaic_outgroup_af_<build>.bin` and
//! is a direct measure of how variable a region is, for reasons that have nothing to do with
//! archaic introgression (mutation rate, reference quality, mappability). This dumps it so that
//! proxy can be tested as a normalizer before an asset is built for the purpose.
//! That spread is larger than the 2.89x enrichment inside a real archaic tract. So the model calls
//! its own upper tail archaic. The hmmix tool avoids this with a map of the mutation rate, and
//! this project has no such asset.
//!
//! `archaic_outgroup_af_<build>.bin` already holds the density of the sites that vary in Africans.
//! That density measures directly how much a region varies. Its reasons have nothing to do with
//! archaic introgression. They are the mutation rate, the quality of the reference, and how well
//! reads map there. This tool writes it out, so that somebody can test it as a normalizer before anybody
//! builds an asset for the purpose.
//!
//! ```sh
//! cargo run --release -p navigator-analysis --example archaic_outgroup_density -- \
Expand Down
33 changes: 19 additions & 14 deletions crates/navigator-analysis/examples/archaic_panel_dump.rs
Original file line number Diff line number Diff line change
@@ -1,16 +1,19 @@
//! Dump the Tier A marker panel with its **per-archaic-genome** calls, as TSV.
//! Write out the Tier A marker panel as a TSV, with the calls of **each archaic genome**.
//!
//! This is the independent evidence for arbitrating Tier B calls. The segment caller
//! ([`navigator_analysis::archaic_match`]) reads only `ArchaicClassify` — a derived base and a
//! lineage class per site — and never sees which archaic genome carries what. So the per-genome
//! pattern is information the caller cannot have fitted to, which is what makes it usable as a
//! referee.
//! This is the independent evidence that decides a Tier B call. The segment caller
//! ([`navigator_analysis::archaic_match`]) reads `ArchaicClassify` alone, which holds a derived
//! base and a lineage class at each site. It never sees which archaic genome carries what. The
//! pattern over the genomes is information that nobody could have fitted the caller to, and that is
//! what makes it a referee.
//!
//! Why a referee is needed: precision has been measured against hmmix's callset, but a call absent
//! from hmmix is not necessarily wrong — hmmix's own tracts are enriched only 1.84x for their own
//! archaic SNPs, so that callset is incomplete by an unknown amount. Scoring a segment against the
//! archaic genomes directly asks whether it looks like an inherited archaic haplotype, without
//! asking another caller's opinion.
//! Here is why a referee is necessary. A measurement gave the precision against the callset of
//! hmmix. But a call that hmmix does not hold is not wrong by that fact alone. The own tracts of
//! hmmix show an enrichment of only 1.84x for their own archaic SNPs. So that callset is
//! incomplete by an amount that nobody knows.
//!
//! A score of a segment against the archaic genomes asks a direct question. Does this segment look
//! like an archaic haplotype that came down from an ancestor? It asks no other caller for an
//! opinion.
//!
//! ```sh
//! cargo run --release -p navigator-analysis --example archaic_panel_dump -- \
Expand All @@ -29,9 +32,11 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
let panel = ArchaicMarkerPanel::from_bytes(&std::fs::read(&path)?).map_err(|e| e.to_string())?;
eprintln!("panel: {} sites, build {}", panel.sites.len(), panel.build);

// One column per archaic genome: D = carries the derived allele, A = positively called
// homozygous-ancestral, . = no call. The A/. distinction is load-bearing — treating a no-call as
// ancestral is the error that produced ~19 % Denisovan for a European in an earlier pass.
// One column for each archaic genome. D means that the genome carries the derived allele. A
// means that the caller positively called it homozygous-ancestral. A `.` means no call.
//
// The difference between A and `.` carries weight. To read a no-call as ancestral is the error
// that gave about 19% Denisovan for a European, in an earlier pass.
println!("contig\tposition\tderived\tclass\t{}", ARCHAIC_GENOMES.join("\t"));
let mut n = 0usize;
for s in &panel.sites {
Expand Down
22 changes: 12 additions & 10 deletions crates/navigator-analysis/examples/archaic_private_dump.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
//! Dump the **private** variant positions the Tier B HMM actually sees — the subject's derived
//! variants after the African-outgroup strip — so the input to the model can be checked against an
//! external truth set independently of the model.
//! Write out the **private** variant positions that the Tier B HMM sees. Those are the derived
//! variants of the subject, after the code removes the ones that the African outgroup also carries.
//! Somebody can then check the input of the model against an external truth set, and that check
//! does not depend on the model.
//!
//! The segment caller is a density model over exactly these positions. If they are not enriched
//! inside known archaic tracts, no amount of HMM tuning can help, and the fault is upstream in the
//! variant calls or the outgroup strip rather than in the model. That question is unanswerable from
//! the caller's own output, which is why this exists.
//! The segment caller is a density model over exactly these positions. If they show no enrichment
//! inside a known archaic tract, then no change to the HMM can help. The fault would lie earlier,
//! in the variant calls or in the removal of the outgroup sites, and not in the model. The own
//! output of the caller can not answer that question, and that is why this tool exists.
//!
//! ```sh
//! cargo run --release -p navigator-analysis --example archaic_private_dump -- \
Expand All @@ -27,14 +28,15 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
let calls: Vec<SiteGenotype> = serde_json::from_str(&std::fs::read_to_string(&calls_path)?)?;
let og = ArchaicOutgroup::from_bytes(&std::fs::read(&og_path)?).map_err(|e| e.to_string())?;

// Group by contig, mirroring what the caller does before it strips.
// Put them into groups by contig, as the caller does before it removes the outgroup sites.
let mut by_contig: std::collections::BTreeMap<String, Vec<&SiteGenotype>> = Default::default();
for c in &calls {
by_contig.entry(c.contig.clone()).or_default().push(c);
}

// Quality columns come out too: whether the background's excess variance is real biology or
// this caller's own error rate varying by region is not answerable without them.
// The quality columns come out too. Without them, nobody can answer one question. Is the
// excess variance of the background real biology, or is it the own error rate of this caller,
// which changes from region to region?
println!("contig\tposition\tdosage\tgq\tdepth");
for (contig, mut sites) in by_contig {
sites.sort_by_key(|s| s.position);
Expand Down
12 changes: 8 additions & 4 deletions crates/navigator-analysis/examples/archaic_segments_probe.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
//! Development harness: run the Tier B segment caller on cached diploid calls.
//! A harness for development. It runs the Tier B segment caller on cached diploid calls.
//!
//! The threshold sweep this also carried has been removed — its output is recorded in the design
//! doc (M3 calibration) and the winning values are the `ArchaicConfig` defaults.
//! archaic_segments_probe <calls.json> <outgroup.bin> <classify.bin> <callable.bin>
//! This tool once also swept the thresholds, and that sweep is gone. Its output sits in the design
//! document, under the M3 calibration, and the values that won are the defaults of
//! `ArchaicConfig`.
//!
//! ```text
//! archaic_segments_probe <calls.json> <outgroup.bin> <classify.bin> <callable.bin>
//! ```
use navigator_analysis::archaic::{ArchaicCallable, ArchaicClassify, ArchaicOutgroup};
use navigator_analysis::archaic_segments::{call_archaic_segments, ArchaicConfig};
use navigator_analysis::caller::SiteGenotype;
Expand Down
Loading
Loading