Gene Expression Models: Cross-Study ER Prediction

A reproducible external-validation study of expression-based ER-status prediction across two public breast-tumor cohorts. The central question is whether a frozen model's probabilities and decisions transfer, rather than whether it retains a high AUROC.

Read the findings · Analysis contract · Scientific decisions · Public data and attribution · Validation evidence

Experiment

Develop on GSE7390 / TRANSBIG, hold out GSE3494 / Uppsala as an entire external study. Use 198 development and 247 labelled external specimens on the same 22,283-probe GPL96 platform. Four target records with uncertain ER labels are explicitly excluded. Labels come from the original GEO metadata and series-level clinical tables, joined by exact specimen/patient IDs.

Four fixed models compare source-prevalence, literature-motivated ESR1-only prediction, deposited-scale multivariable prediction and specimen-local rank-normalized prediction. Training-only feature selection and scaling are repeated inside each source CV fold. The external cohort is never used to tune, select, normalize or calibrate the ER predictors. All specified comparisons are reported, including failure.

This is retrospective prediction of depositor-coded ER status, not a clinical assay, survival analysis, novel biomarker discovery or a demonstration of analytical equivalence. Reference-label procedures and cohort selection differ. Those limits are part of the findings.

Reproduce with Docker

docker compose build
docker compose run --rm lab python -m pytest -q
docker compose run --rm lab
docker compose run --rm lab python -m transferlab.verify artifacts/run

The first analysis run downloads about 66 MiB of public files and validates pinned SHA-256 values. Named volumes retain raw data and analysis results; subsequent runs verify and reuse the cache. No credentials are needed for GEO. The image runs as UID 10001 with one BLAS thread. Dependencies are hash-locked; the Python base is pinned by manifest digest.

Export the generated results to the host without changing directory permissions:

docker compose run --rm -T lab tar -C /lab/artifacts -czf - run > transferability-results.tar.gz

The archive includes the findings, PNG figures, specimen-level predictions, model artifact, selected probe coefficients, subgroup counts, data audit and source-fit receipt. To remove the project containers and networks, use docker compose down; add --volumes only if you intend to remove the cached source and generated results.

Local Python

Python 3.12+ is supported; the validated container uses Python 3.13.

python3 -m venv .venv
.venv/bin/python -m pip install --require-hashes -r requirements.lock
PYTHONPATH=src .venv/bin/python -m pytest -q
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 PYTHONPATH=src .venv/bin/python -m transferlab run --output artifacts/run
PYTHONPATH=src .venv/bin/python -m transferlab.verify artifacts/run

The fixed plan is in docs/analysis-plan.json, with an initial Git protocol commit preceding model fitting. It is internal provenance, not independent preregistration. No minimum model accuracy is required for the project to report results.

What to inspect

Engineering or scientific concern	Implementation and evidence
Reproducible source acquisition	data/sources.json, strict hash verification and ID-based joins in data.py
Leakage prevention	Specimen-local rank transform; pipeline-contained variance filtering, supervised feature selection and scaling
External evaluation	Whole-study holdout, fixed threshold, AUROC, balanced accuracy, sensitivity/specificity, probability scores and conditional confidence intervals
Scientific controls	ESR1-only baseline, prevalence baseline, full-pipeline source-label permutations and near-duplicate profile screening
Verification	Independent rank-sum AUROC and direct confusion/Brier checks from saved predictions; tampered-evidence tests
Traceability	Source/label hashes, feature and specimen hashes, fold IDs, fitted-artifact hash, code/dependency hashes and exclusions
Container behavior	Digest-pinned multi-stage Docker build, non-root runtime, cached named volumes and network-free unit tests

flowchart LR
  GEO[Checksum-bound GEO files] --> Parse[Strict expression and clinical-ID joins]
  Parse --> Source[Development study]
  Parse --> External[External study]
  Source --> CV[Source-only fold pipelines]
  Source --> Frozen[Final frozen source models]
  Frozen --> Receipt[Fit receipt and artifact hash]
  Frozen --> Predict[External predictions]
  External --> Predict
  Predict --> Evaluate[Prespecified metrics and uncertainty]
  CV --> Evaluate
  Evaluate --> Verify[Independent saved-output checks]
  Verify --> Report[Findings, plots and limitations]

Loading

Project layout

• src/transferlab/data.py: acquisition, parsing, phenotype alignment and exclusions.
• model.py: feature contracts, single-specimen transforms and source fitting.
• audit.py: data summaries and sample-profile similarity.
• metrics.py: outcome measures and bootstrap comparisons.
• analysis.py: ordered execution, frozen-source receipt and negative controls.
• report.py: figures and findings generated from calculated results.
• verify.py: numerical reconciliation of saved outputs without loading model pickles.
• tests/: strict-input, leakage-canary, invariance, metric and evidence-tampering checks.
• reports/reference/: committed outputs from the validated Docker execution. Raw downloads are not committed.

Interpretation and reuse

One external cohort cannot establish robustness across all laboratories, populations, platforms or reference assays. Neither high AUROC nor confidence-based abstention establishes clinical utility. The findings distinguish numerical results from mechanistic explanations that remain untested.

Code is MIT-licensed. Public source data retain their original provenance and terms; the code license does not relicense GEO records. No employer, client or restricted-access data are included.