The three scripts that produced the 300-repo controlled study (STUDY-empirical-validation.md). Committed as run on 2026-07-07 — cosmetic quirks preserved deliberately, because these exact scripts generated the exact dossier/assets/analysis.json on record. Fix-forward in a copy, never edit these in place.
Why this folder exists: the scripts originally lived in a session scratchpad, which is ephemeral — it was wiped once mid-session and the detector had to be re-implemented inline to score loopback (rep 1). A study's numbers are only as durable as the code that made them; per repo hard-constraint #2 (every claim cites a file), the generating code is now a file.
| Script | What it does | Output |
|---|---|---|
pull_sample.sh |
Outcome-blind stratified pull: 5 star buckets × 60 repos, AI/LLM devtools created 2025, sorted by creation date within bucket (selection blind to outcome inside each stratum) | sample.jsonl |
enrich_and_measure.py |
Fetches each repo's README + author follower count; mechanically detects the copy features on the first screen (45 lines) | measured.jsonl |
analyze.py |
The three analytical cuts: per-feature lift (high vs low bucket), the author-reach confounder, and the non-famous-author slice | analysis.json + stdout tables |
Data locations: outputs land in ./study-data/ — row-level data stays out of the repo (real repo names + author follower counts; privacy constraint). Only the aggregate analysis.json is committed, under dossier/assets/.
Requirements: gh (authenticated), Python 3 stdlib only — no numpy/pandas/sklearn.
Known limits (unchanged from the study): mechanical detection is noisier than hand-coding (the badge regex undercounts); the Search API's ranker shapes which repos are searchable; the date-sharded draw only approximates outcome-blind. See the study's Limits section before re-using any number.
To re-run for a different category/cohort: change Q_BASE in pull_sample.sh (and the created: window), then run the three scripts in order. If comparing against the committed analysis.json, remember it is AI/LLM-devtools-2025 — a different category needs its own baseline, not this one.