Skip to content

Commit dcc5778

Browse files
hiddenlifhiddenlif
committed
Publish reviewed scientific portfolio snapshot
0 parents  commit dcc5778

48 files changed

Lines changed: 5600 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.dockerignore‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
.git
2+
.venv
3+
.pytest_cache
4+
**/__pycache__
5+
data/raw
6+
reports
7+
artifacts
8+
*.zip

‎.github/workflows/validate.yml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
name: Reproduce and validate
2+
on:
3+
push:
4+
branches: [main]
5+
pull_request:
6+
permissions:
7+
contents: read
8+
jobs:
9+
unit-and-integration:
10+
runs-on: ubuntu-latest
11+
timeout-minutes: 30
12+
steps:
13+
- uses: actions/checkout@v4
14+
- name: Build locked, non-root analysis image
15+
run: docker build -t transferlab:ci .
16+
- name: Unit and adversarial-input checks
17+
run: docker run --rm --network none transferlab:ci python -m pytest -q
18+
- name: Reproduce public-data analysis
19+
run: |
20+
mkdir -p artifacts
21+
docker run --name transferlab-run transferlab:ci
22+
docker cp transferlab-run:/lab/artifacts/run artifacts/run
23+
docker rm transferlab-run
24+
- name: Verify generated evidence
25+
run: docker run --rm --network none -v "$PWD/artifacts/run:/lab/check:ro" transferlab:ci python -m transferlab.verify check
26+
- name: Retain findings and numerical evidence
27+
uses: actions/upload-artifact@v4
28+
with:
29+
name: transferability-evidence
30+
path: artifacts/run
31+
retention-days: 30

‎.gitignore‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
.venv/
2+
__pycache__/
3+
.pytest_cache/
4+
*.egg-info/
5+
data/raw/*
6+
!data/raw/.gitkeep
7+
artifacts/
8+
reports/run-local/
9+
reports/run-docker/
10+
reports/repeat/
11+
.env
12+
.DS_Store

‎CITATION.cff‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
cff-version: 1.2.0
2+
message: "Cite the original GEO studies as well as this software when reusing the analysis."
3+
title: "Assay Transferability Lab"
4+
type: software
5+
authors:
6+
- family-names: Shear
7+
given-names: Kalif
8+
repository-code: "https://github.com/hiddenlif/assay-transferability-lab"
9+
license: MIT

‎Dockerfile‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
FROM python:3.13-slim@sha256:70729b46c69b4f1e97c4822c1af3df53a1476cf5ddc6c087c0c10bc3a5678c2f AS dependencies
2+
WORKDIR /build
3+
COPY requirements.lock .
4+
RUN python -m pip install --no-cache-dir --require-hashes --prefix=/install -r requirements.lock
5+
6+
FROM python:3.13-slim@sha256:70729b46c69b4f1e97c4822c1af3df53a1476cf5ddc6c087c0c10bc3a5678c2f
7+
COPY --from=dependencies /install /usr/local
8+
RUN useradd --uid 10001 --create-home scientist && mkdir -p /lab/data/raw /lab/artifacts /tmp/matplotlib && chown -R scientist:scientist /lab /tmp/matplotlib
9+
WORKDIR /lab
10+
ENV PYTHONPATH=/lab/src OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 MPLCONFIGDIR=/tmp/matplotlib PYTHONUNBUFFERED=1
11+
COPY --chown=scientist:scientist src src
12+
COPY --chown=scientist:scientist tests tests
13+
COPY --chown=scientist:scientist docs docs
14+
COPY --chown=scientist:scientist data/sources.json data/sources.json
15+
COPY --chown=scientist:scientist pyproject.toml requirements.lock ./
16+
USER scientist
17+
CMD ["python", "-m", "transferlab", "run", "--output", "artifacts/run"]

‎LICENSE‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
MIT License
2+
3+
Copyright (c) Kalif Shear
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.
22+
23+
This license covers project code, not the original public GEO data.

‎Makefile‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
.PHONY: build test run
2+
build:
3+
docker build -t assay-transferability-lab:validated .
4+
test: build
5+
docker run --rm --network none assay-transferability-lab:validated python -m pytest -q
6+
run: build
7+
docker compose run --rm lab

‎README.md‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# Assay Transferability Lab
2+
3+
A reproducible external-validation study of expression-based ER-status prediction across two public breast-tumor cohorts. The central question is whether a frozen model's **probabilities and decisions** transfer, rather than whether it retains a high AUROC.
4+
5+
[Read the findings](reports/reference/FINDINGS.md) · [Analysis contract](docs/DESIGN.md) · [Scientific decisions](docs/DECISIONS.md) · [Public data and attribution](data/README.md) · [Validation evidence](docs/VALIDATION.md)
6+
7+
## Experiment
8+
9+
Develop on **GSE7390 / TRANSBIG**, hold out **GSE3494 / Uppsala** as an entire external study. Use 198 development and 247 labelled external specimens on the same 22,283-probe GPL96 platform. Four target records with uncertain ER labels are explicitly excluded. Labels come from the original GEO metadata and series-level clinical tables, joined by exact specimen/patient IDs.
10+
11+
Four fixed models compare source-prevalence, literature-motivated ESR1-only prediction, deposited-scale multivariable prediction and specimen-local rank-normalized prediction. Training-only feature selection and scaling are repeated inside each source CV fold. The external cohort is never used to tune, select, normalize or calibrate the ER predictors. All specified comparisons are reported, including failure.
12+
13+
This is retrospective prediction of depositor-coded ER status, not a clinical assay, survival analysis, novel biomarker discovery or a demonstration of analytical equivalence. Reference-label procedures and cohort selection differ. Those limits are part of the findings.
14+
15+
## Reproduce with Docker
16+
17+
```sh
18+
docker compose build
19+
docker compose run --rm lab python -m pytest -q
20+
docker compose run --rm lab
21+
docker compose run --rm lab python -m transferlab.verify artifacts/run
22+
```
23+
24+
The first analysis run downloads about 66 MiB of public files and validates pinned SHA-256 values. Named volumes retain raw data and analysis results; subsequent runs verify and reuse the cache. No credentials are needed for GEO. The image runs as UID 10001 with one BLAS thread. Dependencies are hash-locked; the Python base is pinned by manifest digest.
25+
26+
Export the generated results to the host without changing directory permissions:
27+
28+
```sh
29+
docker compose run --rm -T lab tar -C /lab/artifacts -czf - run > transferability-results.tar.gz
30+
```
31+
32+
The archive includes the findings, PNG figures, specimen-level predictions, model artifact, selected probe coefficients, subgroup counts, data audit and source-fit receipt. To remove the project containers and networks, use `docker compose down`; add `--volumes` only if you intend to remove the cached source and generated results.
33+
34+
## Local Python
35+
36+
Python 3.12+ is supported; the validated container uses Python 3.13.
37+
38+
```sh
39+
python3 -m venv .venv
40+
.venv/bin/python -m pip install --require-hashes -r requirements.lock
41+
PYTHONPATH=src .venv/bin/python -m pytest -q
42+
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 PYTHONPATH=src .venv/bin/python -m transferlab run --output artifacts/run
43+
PYTHONPATH=src .venv/bin/python -m transferlab.verify artifacts/run
44+
```
45+
46+
The fixed plan is in `docs/analysis-plan.json`, with an initial Git protocol commit preceding model fitting. It is internal provenance, not independent preregistration. No minimum model accuracy is required for the project to report results.
47+
48+
## What to inspect
49+
50+
| Engineering or scientific concern | Implementation and evidence |
51+
|---|---|
52+
| Reproducible source acquisition | `data/sources.json`, strict hash verification and ID-based joins in `data.py` |
53+
| Leakage prevention | Specimen-local rank transform; pipeline-contained variance filtering, supervised feature selection and scaling |
54+
| External evaluation | Whole-study holdout, fixed threshold, AUROC, balanced accuracy, sensitivity/specificity, probability scores and conditional confidence intervals |
55+
| Scientific controls | ESR1-only baseline, prevalence baseline, full-pipeline source-label permutations and near-duplicate profile screening |
56+
| Verification | Independent rank-sum AUROC and direct confusion/Brier checks from saved predictions; tampered-evidence tests |
57+
| Traceability | Source/label hashes, feature and specimen hashes, fold IDs, fitted-artifact hash, code/dependency hashes and exclusions |
58+
| Container behavior | Digest-pinned multi-stage Docker build, non-root runtime, cached named volumes and network-free unit tests |
59+
60+
```mermaid
61+
flowchart LR
62+
GEO[Checksum-bound GEO files] --> Parse[Strict expression and clinical-ID joins]
63+
Parse --> Source[Development study]
64+
Parse --> External[External study]
65+
Source --> CV[Source-only fold pipelines]
66+
Source --> Frozen[Final frozen source models]
67+
Frozen --> Receipt[Fit receipt and artifact hash]
68+
Frozen --> Predict[External predictions]
69+
External --> Predict
70+
Predict --> Evaluate[Prespecified metrics and uncertainty]
71+
CV --> Evaluate
72+
Evaluate --> Verify[Independent saved-output checks]
73+
Verify --> Report[Findings, plots and limitations]
74+
```
75+
76+
## Project layout
77+
78+
- `src/transferlab/data.py`: acquisition, parsing, phenotype alignment and exclusions.
79+
- `model.py`: feature contracts, single-specimen transforms and source fitting.
80+
- `audit.py`: data summaries and sample-profile similarity.
81+
- `metrics.py`: outcome measures and bootstrap comparisons.
82+
- `analysis.py`: ordered execution, frozen-source receipt and negative controls.
83+
- `report.py`: figures and findings generated from calculated results.
84+
- `verify.py`: numerical reconciliation of saved outputs without loading model pickles.
85+
- `tests/`: strict-input, leakage-canary, invariance, metric and evidence-tampering checks.
86+
- `reports/reference/`: committed outputs from the validated Docker execution. Raw downloads are not committed.
87+
88+
## Interpretation and reuse
89+
90+
One external cohort cannot establish robustness across all laboratories, populations, platforms or reference assays. Neither high AUROC nor confidence-based abstention establishes clinical utility. The findings distinguish numerical results from mechanistic explanations that remain untested.
91+
92+
Code is MIT-licensed. Public source data retain their original provenance and terms; the code license does not relicense GEO records. No employer, client or restricted-access data are included.

‎compose.yaml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
services:
2+
lab:
3+
image: assay-transferability-lab:validated
4+
build: .
5+
volumes:
6+
- geo-cache:/lab/data/raw
7+
- analysis-results:/lab/artifacts
8+
init: true
9+
security_opt:
10+
- no-new-privileges:true
11+
cap_drop:
12+
- ALL
13+
volumes:
14+
geo-cache:
15+
analysis-results:

‎data/README.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# Public data
2+
3+
The downloader retrieves original public GEO files over HTTPS and checks the exact SHA-256 values in `sources.json`. Raw files are ignored by Git. No client, employer, restricted-access or synthetic data is used.
4+
5+
- GSE7390, GPL96 series matrix, TRANSBIG: <https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE7390>. Desmedt et al., PMID [17545524](https://pubmed.ncbi.nlm.nih.gov/17545524/).
6+
- GSE3494, GPL96 series matrix and series-level SOFT clinical tables, Uppsala: <https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE3494>. Miller et al., PMID [16141321](https://pubmed.ncbi.nlm.nih.gov/16141321/).
7+
8+
GEO describes its public availability and reuse policy in its [disclaimer](https://www.ncbi.nlm.nih.gov/geo/info/disclaimer.html). NCBI places no restrictions itself but does not guarantee third-party rights are absent. This project preserves source attribution, does not claim ownership of the source data and does not apply its code license to GEO records. Check original records and applicable requirements before other reuse.
9+
10+
Only public sample accessions, the required clinical label and limited age/hospital metadata are parsed into analytical tables. Contact metadata present in full GEO downloads is not copied into reports. Missing labels and mapping discrepancies are logged explicitly. Matrix and clinical-table checksums bind the dataset and label versions used in a run.

0 commit comments

Comments
 (0)