examples: add the IMPROVE neoepitope immunogenicity cohort

[aRealGem]

Requested by Precious · project thread

Before: predval shipped two example cohorts — a clinical risk score (GUSTO-I) and an imaging classifier roster (PCam). Neither exercised the harness at the shape that budget-constrained selection problems have: 2.7% prevalence, 70 independent units, and 250 correlated observations inside each one. The neoantigen immunogenicity literature reports ranking metrics only, so there was no worked example of what the harness has to say about that kind of model.

After: a third example cohort reads the out-of-fold predictions released with Borch et al. 2024 (github.com/SRHgroup/IMPROVE_paper) — 17,520 candidate neoepitopes from 70 patients, 467 experimentally immunogenic, seven models — and evaluates them with the patient as the unit of independence. Library code is untouched; this is an example directory, a .gitignore entry for its regenerable parquets, and nothing else.

The cross-validation behind those predictions was partitioned by patient. build_fixture.py verifies that rather than assuming it, which is what makes the file admissible as a predictions-only input instead of an apparent-fit artefact. Two subgroups are prespecified in cohort.yaml: the release's three accrual cohorts, as the closest available proxy for transfer to another indication, and 9-mers versus other lengths, where a fault would be expected a priori.

What the run produces is in examples/improve/README.md. In short: six of the seven models carry a large intercept-only miscalibration that predval's rung1 repairs with a cross-fitted, interval-backed gain; the seventh comes back not miscalibrated, with rung3 flagged as reliably harmful, so the harness declines to recalibrate it. That is a second independent instance of the refuse-to-manufacture-a-gain behaviour the PCam fixture demonstrates, on a model class the harness was not designed around.

How: build_fixture.py assembles cohort.parquet and predictions.parquet from the released archive, including the NNAlign comparator — whose header line crams five field names into one tab-delimited cell, so the identifiers are read off the index levels rather than the header. run_evaluation.py is the standard end-to-end evaluation with a prevalence-anchoring block added ahead of discrimination. budget_yield.py is a sidecar, explicitly outside predval's contract, that scores each patient's own top-k candidates for k ∈ {3, 5, 10, 20} and reports observed yield against the yield the published probabilities promise over that identical set — so the gap is miscalibration rather than ranking error. It reuses predval's own cross-fitting helper rather than reimplementing it.

Checks: 232 tests pass (2 skipped), ruff check and ruff format --check clean on the new directory.

🤖 Generated with Claude Code

https://claude.ai/code/session_0138yQfFhvnVRHd1XhTEz54w

---

Generated by Claude Code