Scrapes soccer news, social, and match data from public sources into a timestamped DuckDB store, and summarizes the latest of it into a concise, themed digest using a hosted LLM (NVIDIA Llama-3.3-70B).
uv sync
cp .env.example .env # add NVIDIA_API_KEY (free: build.nvidia.com)
uv run python -m src.summarize --scrape # scrape fresh news + print a digest--scrape pulls fresh headlines first; drop it to digest what's already stored.
--out digest.md writes to a file; --limit N sets how many recent items to include.
Sources (pluggable — any can be disabled in config/default.yaml):
| Source | Auth | Provides |
|---|---|---|
| Guardian + BBC football RSS | none | news headlines |
| football-data.co.uk | none | match results + closing odds |
| Reddit r/soccer (PRAW) | optional | social posts |
Scraping needs no key; only the summary step calls the LLM (NVIDIA_API_KEY). Data
sources are read-only via their official feeds/APIs, throttled politely.
Forecasting layer (research extension — optional)
The same scraped store also feeds a calibration-first match-outcome forecaster (RAG + LightGBM/isotonic + a leak-free walk-forward backtest + MLOps). Its honest result: forecasts are well-calibrated but sit below the efficient market — it does not beat the closing line, and the paper-trading layer is simulation only, never live. Details in the sections below.
Everything below is the optional match-outcome forecaster built on the same scraped store. It is a research exercise in calibration + MLOps, not the primary tool. Skip it if you only care about scrape-and-summarize.
Every feature for a match may use only observations with ts < kickoff. Enforced
at each layer that builds features, with a leakage test per layer:
- Form features fold a match's result into history only after its own row is
built (
src/prediction/features.py→tests/test_prediction.py). - Retrieval filters the vector store on
ts_ms < kickoffand re-asserts it on the result bundle (src/retrieval/retrieve.py→tests/test_retrieval.py). - Backtest trains only on matches strictly before each prediction block
(
eval/backtest.py→tests/test_backtest.py, which also asserts no live API call).
uv sync # venv + core deps (duckdb, pyyaml, pytest, ...)
cp .env.example .env # then add your NVIDIA_API_KEY (free: build.nvidia.com)
# behind a TLS-intercepting proxy? add --system-certs to uv commands.env is gitignored — keys never get committed. Phases 1, 4, 5, 6 run without any
key; Phases 2–3 need NVIDIA_API_KEY (hosted LLM + embeddings).
uv run pytest -q # full suite incl. the per-layer leakage tests
# --- Primary use: scrape + summarize ---
uv run python -m src.summarize --scrape # scrape fresh news, print a digest
uv run python -m src.summarize --limit 40 --out digest.md # digest stored news to a file
# Phase 1 — ingestion (idempotent; safe to re-run):
uv run python -m src.ingestion.run # all enabled sources
uv run python -m src.ingestion.run --source news_rss # just one source
# football-data + news RSS need no auth. Reddit skips unless you copy
# .env.example -> .env and set REDDIT_CLIENT_ID / REDDIT_CLIENT_SECRET.
# Phase 2 — signal extraction (hosted NVIDIA LLM; needs NVIDIA_API_KEY in .env):
uv run python -m src.extraction.extract --limit 20 # small run; scale up after sign-off
# Phase 3 — embed signals into Qdrant (local mode, no Docker); needs NVIDIA_API_KEY:
uv run python -m src.retrieval.index # index relevant signals as vectors
# Phase 4 — train + calibrate the outcome model (CPU, no API; reads EPL matches):
uv run python -m src.prediction.train # prints the calibration report
# Phase 5 — leak-free walk-forward backtest (no API; persists predictions):
uv run python -m eval.backtest # Brier/log-loss/reliability over history
# Phase 6 — paper trading over the backtest (SIMULATED, no execution):
uv run python -m src.decision.paper_trade # fractional-Kelly ledger + P&L
# Phase 7 — MLOps: calibration eval gate, tracking, drift, dashboard
uv run python -m eval.gate # PASS/FAIL vs eval/baseline_metrics.json
uv run python -m eval.gate --simulate-regression # DEMO: worse model -> GATE FAIL, exit 1
uv run python -m eval.gate --update-baseline # accept current metrics as baseline
uv sync --group mlops # adds mlflow + streamlit (optional, heavy)
uv run streamlit run dashboards/app.py # live calibration + P&L dashboard
docker compose up -d qdrant # optional Qdrant *server* (local mode used by default)What to look for: pytest is green, including the three per-layer leakage
tests above (an observation exactly at or after kickoff is excluded from features;
the backtest never trains on a same-or-future match and makes no live API call).
src/common/ # UTC time, DuckDB store, config, network clients
src/ingestion/ # (A) collectors Phase 1
src/extraction/ # (B) hosted-LLM extractor Phase 2
src/retrieval/ # (C) RAG Phase 3
src/prediction/ # (D) model + calibration Phase 4
eval/ # leak-free backtest + gate Phase 5/7 (centerpiece)
src/decision/ # (E) paper trading Phase 6
src/mlops/ # (F) tracking/drift/dashboard Phase 7
config/default.yaml
tests/ # per-layer leakage + unit tests
- Phase 0 — scaffold + point-in-time schema/config + per-layer leakage tests
- Phase 1 — pluggable ingestion (football-data.co.uk, news RSS, Reddit) → DuckDB, idempotent
- Phase 2 — signal extraction live (NVIDIA llama-3.3-70b → strict JSON; 47 signals, 100% valid, 68% pre-filter drop)
- Phase 3 — RAG retrieval (NVIDIA nv-embedqa-e5-v5 + Qdrant local; point-in-time, recency/credibility re-rank, leakage-guarded)
- Phase 4 — prediction + calibration: LightGBM + isotonic, out-of-time split; calibrated ECE 0.047 (raw 0.149), honestly below the market baseline
- Phase 5 — leak-free walk-forward backtest: 1,140 predictions, calibration stable over time (ECE ~0.04–0.05/season), honestly below market; no-lookahead + no-API tests
- Phase 6 — paper trading (fractional-Kelly, simulated ledger): −10% ROI / near-total ruin on 1,093 bets — the model loses to the vig, exactly as a calibrated-but-not-market-beating forecaster should
- Phase 7 — MLOps wrap: calibration eval gate (PASS on baseline / FAIL+exit-1 on a worse model) + GitHub Actions CI, MLflow tracking+registry, PSI feature drift, hosted-API quota monitor, Streamlit dashboard, DVC pipeline
- Phase 8 — adding a fine tuned ai to make it more efficent
** 7 phases complete.** 38 tests pass (incl. the per-layer leakage tests + the gate's block-a-worse-model test). The result is honest: the forecaster is calibrated (ECE ~0.04, stable across seasons) and sits below an efficient market which proves calibration, not beating the line, exactly as the brief framed success. must complete phase 8
- Real: leak-free point-in-time architecture + per-layer leakage tests; live NVIDIA extraction & embeddings; walk-forward backtest; isotonic calibration; CI eval gate.
- Honest limits: RAG signals are live-World-Cup only (no contemporaneous text for historical EPL, so they're zero on the backtest); drift/quota are lightweight (PSI, cache-count) with Evidently notable as a swap-in; single league.
Research / educational project. No real-money trading — the decision layer only logs simulated positions and settles them against actual results. Data sources are used read-only via their official APIs/feeds, throttled politely.