proper-scoring-rules

Scoring rules like the Brier Score (Mean Squared Error, Quadratic Score) and Log Loss (Cross-Entropy, Negative Log-Likelihood, Logarithmic Score) can favor incorrect predictions. To address this limitation, the Probabilistic Brier Score (PBS) and Probabilistic Logarithmic Loss (PLL) have been introduced for probabilistic classifiers.

Proper scoring rules, reduces LLM overconfidence in multiple-choice QA.

A public forecast ledger: predictions filed in git before the fact, scored against reality by a proper rule, forever. First round already on the books.

Reproducible Polymarket threshold-ladder benchmark: 67k forecasts, PAV/calibration baselines, chronological holdout, paired bootstrap, CI-verified results.

Do published probabilistic forecasts hold up when you score them? Pre-registered verification of NWS precipitation, AirNow air quality, and CAISO day-ahead power prices — one scoring toolkit, three domains.

Pre-registered calibration audit of ten mortality-forecasting families and seven uncertainty mechanisms across the COVID-19 structural break.

Interacting-particle sampler for Proper-Scoring-Rule (PrO) posteriors. NumPy-first, Poetry-managed, with log-score and MMD² drift examples.

Near-Optimal Online Recalibration for Log Loss DOI: https://doi.org/10.5281/zenodo.21698617