From 342eb75d0f06d1b6e0a8a58e8583af3eaad192f3 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Tue, 18 Aug 2026 10:56:38 +1000 Subject: [PATCH] =?UTF-8?q?Land=20wave=20C2=20+=20Track=20D=20=E2=80=94=20?= =?UTF-8?q?the=20four=20remaining=20static=20datasets?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Wave C2 (advanced.myst): fred_data.csv, acs_data_summary.csv, and dataBHS.csv — the format-converted successor to lectures/dataBHS.mat, which the published site 404s while the published notebook calls loadmat on it. Track D rides along: test_pwt.csv from lecture-python-programming, whose rename is on the same phase-2 review (QuantEcon/data-lectures#87). Two builders recovered against annotations recording no build script anywhere: fred_data.py reproduces the committed bytes from live FRED (three runs, 2026-08-18 — yields and the recession dummy are stable history, unlike the BBH national-accounts snapshot), and dataBHS.py converts sources/dataBHS.mat (un-refetchable; searched with positive controls, trail in sources/README.md) with the lecture's hardcoded moments asserted on every run. acs_data_summary.csv and test_pwt.csv are honest unrecovered: the ACS summary has no occupation key, year or recorded filters, and test_pwt.csv matches NO downloadable PWT vintage (7.0/6.3/6.2/6.1 all fetched and eliminated, hashes in the manifest). The dataBHS.mat annotation stays until the repoint lands — the lecture still reads the .mat and the basenames differ, so dual_recorded cannot fire; the flip PR deletes it. The other three annotations are deleted here. builders/README.md coverage table re-derived from parsed manifests (28 constructed, 18 with builders, 10 unrecovered), adding the two bbh rows omitted by QuantEcon/data-lectures#92. Strict audit exits 0 against current clones with all four records landed, and the negative control fires: flipping fred_data.csv to repointed yields exit 1 with the expected migration_inconsistencies. Co-Authored-By: Claude Fable 5 --- CATALOG.md | 6 +- builders/README.md | 21 +- builders/dataBHS.py | 111 ++++ builders/fred_data.py | 125 +++++ lectures/acs_data_summary.csv | 352 ++++++++++++ lectures/acs_data_summary.csv.yml | 155 ++++++ lectures/dataBHS.csv | 237 ++++++++ lectures/dataBHS.csv.yml | 196 +++++++ lectures/fred_data.csv | 862 ++++++++++++++++++++++++++++++ lectures/fred_data.csv.yml | 191 +++++++ lectures/test_pwt.csv | 9 + lectures/test_pwt.csv.yml | 204 +++++++ migration.yml | 72 ++- requirements.txt | 4 +- scripts/audit_annotations.yml | 16 +- sources/README.md | 50 ++ sources/dataBHS.mat | 3 + 17 files changed, 2588 insertions(+), 26 deletions(-) create mode 100644 builders/dataBHS.py create mode 100644 builders/fred_data.py create mode 100644 lectures/acs_data_summary.csv create mode 100644 lectures/acs_data_summary.csv.yml create mode 100644 lectures/dataBHS.csv create mode 100644 lectures/dataBHS.csv.yml create mode 100644 lectures/fred_data.csv create mode 100644 lectures/fred_data.csv.yml create mode 100644 lectures/test_pwt.csv create mode 100644 lectures/test_pwt.csv.yml create mode 100644 sources/dataBHS.mat diff --git a/CATALOG.md b/CATALOG.md index 7fa4da3..35699b6 100644 --- a/CATALOG.md +++ b/CATALOG.md @@ -6,13 +6,14 @@ The dataset registry, **auto-generated** from the sidecar manifests (`lectures/*.yml`). Do not edit by hand — run `python scripts/build_catalog.py`. A dataset appears here once it has a manifest, which may be before its consuming lectures are repointed — an empty **Used by** column means the file is here and documented but no lecture reads it from this repo yet. Files still to migrate are tracked in [PLAN.md](PLAN.md). -**36 datasets** · 36 read by lectures today · 113.1 MB total · 30 permitted / 6 restricted redistribution +**40 datasets** · 40 read by lectures today · 113.1 MB total · 34 permitted / 6 restricted redistribution | Dataset | Class | Source | Licence | Redist. | Integrity | Builder | Size | Used by | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | [**NEWQDATA.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/NEWQDATA.csv)
Cogley-Sargent (2005) "Drifts and Volatilities" — quarterly US inflation, unemployment and T-bill rate | constructed | [Cogley and Sargent (2005), "Drifts and Volatilities" — the authors' NEWQDATA.MAT](https://doi.org/10.1016/j.red.2004.10.009) | | ✅ permitted | ✅ verified | ✅ committed | 13.4 KB | [lecture-python.myst · phillips_drifts_volatilities.md](https://github.com/QuantEcon/lecture-python.myst/blob/main/lectures/phillips_drifts_volatilities.md)
[lecture-python.zh-cn · phillips_drifts_volatilities.md](https://github.com/QuantEcon/lecture-python.zh-cn/blob/main/lectures/phillips_drifts_volatilities.md) | | [**SCF_plus_mini.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/SCF_plus_mini.csv)
SCF+ mini — net wealth, income and survey weights, 1950-2016 | constructed | [SCF+ (Kuhn, Schularick and Steins) — an extension of the Survey of Consumer Finances](https://www.journals.uchicago.edu/doi/10.1086/708815) | | ✅ permitted | ⚠️ unverifiable | committed-frozen | 31.3 MB | [lecture-python-intro · inequality.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/inequality.md)
[lecture-wasm · inequality.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/inequality.md)
[lecture-intro.zh-cn · inequality.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/inequality.md)
[test-actions-lecture-intro · inequality.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/inequality.md) | | [**SCF_plus_mini_no_weights.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/SCF_plus_mini_no_weights.csv)
SCF+ mini, weight-expanded — net wealth and income, 1950-2016 | constructed | [SCF+ (Kuhn, Schularick and Steins) — an extension of the Survey of Consumer Finances](https://www.journals.uchicago.edu/doi/10.1086/708815) | | ✅ permitted | ⚠️ unverifiable | committed-frozen | 72.4 MB | [lecture-python-intro · mle.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/mle.md)
[lecture-wasm · mle.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/mle.md)
[lecture-intro.zh-cn · mle.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/mle.md)
[test-actions-lecture-intro · mle.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/mle.md) | +| [**acs_data_summary.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/acs_data_summary.csv)
American Community Survey — earnings count, mean and dispersion by occupation cell | constructed | [American Community Survey (US Census Bureau) — occupation-level summary, assembly unrecorded](https://www.census.gov/programs-surveys/acs) | | ✅ permitted | ⚠️ unverifiable | ⚠️ unrecovered | 14.0 KB | [lecture-python-advanced.myst · match_transport.md](https://github.com/QuantEcon/lecture-python-advanced.myst/blob/main/lectures/match_transport.md)
⚠️ Reads its own copy at `lectures/_static/lecture_specific/match_transport/`, not this file — the repoint PR in lecture-python-advanced.myst follows this one, and migration.yml records the dataset as `landed` until it merges | | [**ames_house_prices.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/ames_house_prices.csv)
Ames, Iowa — residential house sales, 2006-2010 | constructed | [Ames Housing data (De Cock 2011), Journal of Statistics Education](http://jse.amstat.org/v19n3/decock.pdf) | | ✅ permitted | ✅ verified | ✅ committed | 75.2 KB | [lecture-python-intro · observed_distributions.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/observed_distributions.md)
[lecture-python-intro · fitting_distributions.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/fitting_distributions.md)
[lecture-intro.zh-cn · fitting_distributions.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/fitting_distributions.md)
[lecture-intro.zh-cn · observed_distributions.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/observed_distributions.md) | | [**assignat.xlsx**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/assignat.xlsx)
French Revolution — assignat issues, budgets and seigniorage (Sargent-Velde) | verbatim | [Sargent and Velde, "Macroeconomic Features of the French Revolution" — supporting spreadsheets](https://www.journals.uchicago.edu/doi/10.1086/261992) | | ✅ permitted | ⚠️ unverifiable | n/a (verbatim) | 204.6 KB | [lecture-python-intro · french_rev.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/french_rev.md)
[lecture-wasm · french_rev.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/french_rev.md)
[lecture-intro.zh-cn · french_rev.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/french_rev.md)
[test-actions-lecture-intro · french_rev.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/french_rev.md) | | [**bbh_macro_quarterly.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/bbh_macro_quarterly.csv)
Bhandari-Borovička-Ho replication — quarterly US macro series for the belief-wedge VAR, 1955Q1-2019Q4 | constructed | [Replication package for "Survey data and subjective beliefs in business cycle models" (Bhandari, Borovička and Ho), file `data input/FRED/data_FRED.xlsx`](https://doi.org/10.5281/zenodo.10194324) | CC-BY-4.0 | ✅ permitted | ✅ verified | ✅ committed | 31.5 KB | [lecture-python-advanced.myst · subjective_beliefs_business_cycles.md](https://github.com/QuantEcon/lecture-python-advanced.myst/blob/main/lectures/subjective_beliefs_business_cycles.md) | @@ -22,6 +23,7 @@ The dataset registry, **auto-generated** from the sidecar manifests (`lectures/* | [**cities_brazil.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/cities_brazil.csv)
World Population Review — Brazilian city populations, 2023 | verbatim | [World Population Review — cities in Brazil](https://worldpopulationreview.com/countries/cities/brazil) | | ⚠️ restricted | ⚠️ unverifiable | n/a (verbatim) | 17.5 KB | [lecture-python-intro · heavy_tails.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/heavy_tails.md)
[lecture-wasm · heavy_tails.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/heavy_tails.md)
[lecture-intro.zh-cn · heavy_tails.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/heavy_tails.md)
[test-actions-lecture-intro · heavy_tails.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/heavy_tails.md) | | [**cities_us.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/cities_us.csv)
World Population Review — US city populations, 2023 | verbatim | [World Population Review — US cities](https://worldpopulationreview.com/us-cities) | | ⚠️ restricted | ⚠️ unverifiable | n/a (verbatim) | 47.0 KB | [lecture-python-intro · heavy_tails.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/heavy_tails.md)
[lecture-wasm · heavy_tails.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/heavy_tails.md)
[lecture-intro.zh-cn · heavy_tails.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/heavy_tails.md)
[test-actions-lecture-intro · heavy_tails.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/heavy_tails.md) | | [**countries.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/countries.csv)
WorldData.info country reference table | verbatim | [WorldData.info — country data downloads](https://www.worlddata.info/downloads/) | Proprietary — © WorldData.info, all rights reserved | ⚠️ restricted | ⚠️ unverifiable | n/a (verbatim) | 48.4 KB | [lecture-python-programming · pandas_panel.md](https://github.com/QuantEcon/lecture-python-programming/blob/main/lectures/pandas_panel.md)
[lecture-python.myst · pandas_panel.md](https://github.com/QuantEcon/lecture-python.myst/blob/main/lectures/pandas_panel.md)
[lecture-python.zh-cn · pandas_panel.md](https://github.com/QuantEcon/lecture-python.zh-cn/blob/main/lectures/pandas_panel.md) | +| [**dataBHS.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/dataBHS.csv)
Barillas-Hansen-Sargent "Doubts or variability?" — quarterly US log consumption and real returns, 1948Q1-2006Q4 | constructed | [Barillas, Hansen and Sargent (2009), "Doubts or variability?" — the authors' dataBHS.mat](https://doi.org/10.1016/j.jet.2008.11.014) | | ✅ permitted | ✅ verified | ✅ committed | 9.9 KB | [lecture-python-advanced.myst · five_preferences.md](https://github.com/QuantEcon/lecture-python-advanced.myst/blob/main/lectures/five_preferences.md)
⚠️ Reads its own copy — loadmat('dataBHS.mat') on a bare local filename at lectures/dataBHS.mat, resolved against the notebook working directory — not this file
[lecture-tools-techniques · five_preferences.md](https://github.com/QuantEcon/lecture-tools-techniques/blob/main/lectures/five_preferences.md)
⚠️ Reads its own byte-identical dataBHS.mat copy (five_preferences.md:1882), not this file — same dataset, same lecture, second published series, with the same downloadable-notebook 404 | | [**dette.xlsx**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/dette.xlsx)
French Revolution — public debt, military spending and revenues (Sargent-Velde) | verbatim | [Sargent and Velde, "Macroeconomic Features of the French Revolution" — supporting spreadsheets](https://www.journals.uchicago.edu/doi/10.1086/261992) | | ✅ permitted | ⚠️ unverifiable | n/a (verbatim) | 617.2 KB | [lecture-python-intro · french_rev.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/french_rev.md)
[lecture-wasm · french_rev.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/french_rev.md)
[lecture-intro.zh-cn · french_rev.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/french_rev.md)
[test-actions-lecture-intro · french_rev.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/french_rev.md) | | [**employ.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/employ.csv)
Eurostat employment in Europe — by age and sex, 2007–2016 | constructed | [Eurostat — Employment database](https://ec.europa.eu/eurostat/data/database) | Eurostat reuse (Commission Decision 2011/833/EU) | ✅ permitted | ⚠️ unverifiable | ⚠️ unrecovered | 1.6 MB | [lecture-python-programming · pandas_panel.md](https://github.com/QuantEcon/lecture-python-programming/blob/main/lectures/pandas_panel.md)
[lecture-python.myst · pandas_panel.md](https://github.com/QuantEcon/lecture-python.myst/blob/main/lectures/pandas_panel.md)
[lecture-python.zh-cn · pandas_panel.md](https://github.com/QuantEcon/lecture-python.zh-cn/blob/main/lectures/pandas_panel.md) | | [**epl_match_goals.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/epl_match_goals.csv)
English Premier League — full-time scores, 2015-16 to 2024-25 | constructed | [openfootball / football.json](https://github.com/openfootball/football.json) | Public domain | ✅ permitted | ✅ verified | ✅ committed | 203.2 KB | [lecture-python-intro · fitting_distributions.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/fitting_distributions.md)
[lecture-intro.zh-cn · fitting_distributions.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/fitting_distributions.md) | @@ -29,6 +31,7 @@ The dataset registry, **auto-generated** from the sidecar manifests (`lectures/* | [**forbes-billionaires.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/forbes-billionaires.csv)
Forbes Billionaires — individual net worth | constructed | [Forbes Billionaires](https://www.forbes.com/billionaires/) | | ⚠️ restricted | ⚠️ unverifiable | committed-frozen | 775.7 KB | [lecture-python-intro · heavy_tails.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/heavy_tails.md)
[lecture-wasm · heavy_tails.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/heavy_tails.md)
[lecture-intro.zh-cn · heavy_tails.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/heavy_tails.md)
[test-actions-lecture-intro · heavy_tails.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/heavy_tails.md) | | [**forbes-global2000.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/forbes-global2000.csv)
Forbes Global 2000 — firm size measures | constructed | [Forbes Global 2000](https://www.forbes.com/lists/global2000/) | | ⚠️ restricted | ⚠️ unverifiable | committed-frozen | 115.6 KB | [lecture-python-intro · heavy_tails.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/heavy_tails.md)
[lecture-wasm · heavy_tails.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/heavy_tails.md)
[lecture-intro.zh-cn · heavy_tails.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/heavy_tails.md)
[test-actions-lecture-intro · heavy_tails.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/heavy_tails.md) | | [**fp.dta**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/fp.dta)
Treisman (2016) Russia's Billionaires — country-year panel of billionaire counts and covariates | verbatim | [Replication package for Treisman (2016), "Russia's Billionaires" (AER Papers & Proceedings)](https://www.aeaweb.org/articles?id=10.1257/aer.p20161068) | | ✅ permitted | ✅ verified | n/a (verbatim) | 1000.1 KB | [lecture-python.myst · mle.md](https://github.com/QuantEcon/lecture-python.myst/blob/main/lectures/mle.md)
[lecture-python.zh-cn · mle.md](https://github.com/QuantEcon/lecture-python.zh-cn/blob/main/lectures/mle.md)
[lecture-stats · mle.md](https://github.com/QuantEcon/lecture-stats/blob/main/lectures/mle.md) | +| [**fred_data.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/fred_data.csv)
US Treasury yields and NBER recessions — GS1/GS5/GS10, DFII5/DFII10 and USREC, monthly 1953-04 to 2024-12 | constructed | [FRED (Federal Reserve Bank of St. Louis) — series GS1, GS5, GS10, DFII5, DFII10, USREC](https://fred.stlouisfed.org) | | ✅ permitted | ✅ verified | ✅ committed | 27.3 KB | [lecture-python-advanced.myst · risk_aversion_or_mistaken_beliefs.md](https://github.com/QuantEcon/lecture-python-advanced.myst/blob/main/lectures/risk_aversion_or_mistaken_beliefs.md)
⚠️ Reads its own copy over an own-repo raw URL (`lecture-python-advanced.myst/.../risk_aversion_or_mistaken_beliefs/`), not this file — the repoint PR in lecture-python-advanced.myst follows this one, and migration.yml records the dataset as `landed` until it merges | | [**hansen_jagannathan_1991_data.json**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/hansen_jagannathan_1991_data.json)
Hansen-Jagannathan (1991) replication — US asset returns 1891-1986, with annual consumption (three-table bundle) | constructed | [Robert J. Shiller's public data workbooks (chapt26.xlsx, ie_data.xls) and FRED](http://www.econ.yale.edu/~shiller/data.htm) | | ✅ permitted | ✅ verified | ⚠️ unrecovered | 62.0 KB | [lecture-python-advanced.myst · hansen_jagannathan_1991.md](https://github.com/QuantEcon/lecture-python-advanced.myst/blob/main/lectures/hansen_jagannathan_1991.md) | | [**hansen_singleton_1982_data.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/hansen_singleton_1982_data.csv)
Hansen-Singleton (1982) replication — monthly US gross real market return and consumption growth, 1959-1978 | constructed | [FRED (BEA and BLS monthly series) and the Ken French data library (F-F_Research_Data_Factors)](https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html) | | ✅ permitted | ✅ verified | ✅ committed | 11.4 KB | [lecture-python.myst · hansen_singleton_1982.md](https://github.com/QuantEcon/lecture-python.myst/blob/main/lectures/hansen_singleton_1982.md)
[lecture-python.zh-cn · hansen_singleton_1982.md](https://github.com/QuantEcon/lecture-python.zh-cn/blob/main/lectures/hansen_singleton_1982.md) | | [**hansen_singleton_1983_data.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/hansen_singleton_1983_data.csv)
Hansen-Singleton (1983) replication — monthly US returns, consumption and inflation, 1959-1978 | constructed | [FRED (BEA and BLS monthly series) and the Ken French data library (F-F_Research_Data_Factors)](https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html) | | ✅ permitted | ✅ verified | ✅ committed | 25.5 KB | [lecture-python.myst · hansen_singleton_1983.md](https://github.com/QuantEcon/lecture-python.myst/blob/main/lectures/hansen_singleton_1983.md)
[lecture-python.zh-cn · hansen_singleton_1983.md](https://github.com/QuantEcon/lecture-python.zh-cn/blob/main/lectures/hansen_singleton_1983.md) | @@ -44,6 +47,7 @@ The dataset registry, **auto-generated** from the sidecar manifests (`lectures/* | [**mpd2020.xlsx**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/mpd2020.xlsx)
Maddison Project Database 2020 — GDP per capita and population, 1 CE to 2018 | constructed | [Maddison Project Database, version 2020](https://www.rug.nl/ggdc/historicaldevelopment/maddison/releases/maddison-project-database-2020) | CC BY 4.0 | ✅ permitted | ⇄ diverged | ⚠️ unrecovered | 1.7 MB | [lecture-python-intro · long_run_growth.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/long_run_growth.md)
[lecture-wasm · long_run_growth.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/long_run_growth.md)
[lecture-intro.zh-cn · long_run_growth.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/long_run_growth.md)
[test-actions-lecture-intro · long_run_growth.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/long_run_growth.md) | | [**nom_balances.npy**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/nom_balances.npy)
French Revolution — monthly nominal assignat balances, 1789-1796 | constructed | unrecorded | | ✅ permitted | ⚠️ unverifiable | ⚠️ unrecovered | 1.4 KB | [lecture-python-intro · french_rev.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/french_rev.md)
[lecture-wasm · french_rev.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/french_rev.md)
[lecture-intro.zh-cn · french_rev.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/french_rev.md)
[test-actions-lecture-intro · french_rev.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/french_rev.md)
⚠️ Reads a local `datasets/` copy, not this file | | [**realwage.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/realwage.csv)
OECD real minimum wages — 32 countries, 2006–2016 | constructed | [OECD — Real minimum wages (RMW)](https://stats.oecd.org/Index.aspx?DataSetCode=RMW) | CC BY 4.0 | ✅ permitted | ⚠️ unverifiable | ⚠️ unrecovered | 118.7 KB | [lecture-python-programming · pandas_panel.md](https://github.com/QuantEcon/lecture-python-programming/blob/main/lectures/pandas_panel.md)
[lecture-python.myst · pandas_panel.md](https://github.com/QuantEcon/lecture-python.myst/blob/main/lectures/pandas_panel.md)
[lecture-python.zh-cn · pandas_panel.md](https://github.com/QuantEcon/lecture-python.zh-cn/blob/main/lectures/pandas_panel.md) | +| [**test_pwt.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/test_pwt.csv)
Penn World Table teaching extract — eight countries, year 2000 | constructed | [Penn World Table (Heston, Summers and Aten; PWT 6.x/7.x era) — exact vintage unestablished](https://www.rug.nl/ggdc/productivity/pwt/pwt-releases/pwt-7.0) | | ✅ permitted | ⚠️ unverifiable | ⚠️ unrecovered | 793 B | [lecture-python-programming · pandas.md](https://github.com/QuantEcon/lecture-python-programming/blob/main/lectures/pandas.md)
⚠️ Reads the repo's own raw URL (one unsplit literal, pandas.md:172), not this file — the repoint PR follows this one, and migration.yml records the dataset as `landed` until it merges
[lecture-python-programming · polars.md](https://github.com/QuantEcon/lecture-python-programming/blob/main/lectures/polars.md)
⚠️ Reads the repo's own raw URL three times (polars.md:161-163, 347-349, 425-427), each split across three adjacent string literals — collapse at the repoint, do not patch the stem (NEWQDATA precedent)
[lecture-python-programming.zh-cn · pandas.md](https://github.com/QuantEcon/lecture-python-programming.zh-cn/blob/main/lectures/pandas.md)
⚠️ Synced translation; reads lecture-python-programming's raw URL, not this file
[lecture-python-programming.zh-cn · polars.md](https://github.com/QuantEcon/lecture-python-programming.zh-cn/blob/main/lectures/polars.md)
⚠️ Synced translation; same URL, same sync dependency as pandas.md above
[lecture-python-programming.fr · pandas.md](https://github.com/QuantEcon/lecture-python-programming.fr/blob/main/lectures/pandas.md)
⚠️ Synced translation; reads lecture-python-programming's raw URL, not this file
[lecture-python-programming.fr · polars.md](https://github.com/QuantEcon/lecture-python-programming.fr/blob/main/lectures/polars.md)
⚠️ Synced translation; same URL, same sync dependency
[lecture-python-programming.fa · pandas.md](https://github.com/QuantEcon/lecture-python-programming.fa/blob/main/lectures/pandas.md)
⚠️ Synced translation; reads lecture-python-programming's raw URL, not this file
[lecture-python-programming.fa · polars.md](https://github.com/QuantEcon/lecture-python-programming.fa/blob/main/lectures/polars.md)
⚠️ Synced translation; same URL, same sync dependency | | [**us_adult_heights.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/us_adult_heights.csv)
United States — adult standing height by sex, NHANES 2015-2018 | constructed | [National Health and Nutrition Examination Survey (NHANES), US Centers for Disease Control and Prevention, National Center for Health Statistics](https://www.cdc.gov/nchs/nhanes/index.htm) | US Government work — public domain | ✅ permitted | ✅ verified | ✅ committed | 123.1 KB | [lecture-python-intro · prob_dist.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/prob_dist.md)
[lecture-python-intro · observed_distributions.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/observed_distributions.md)
[lecture-python-intro · fitting_distributions.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/fitting_distributions.md)
[lecture-intro.zh-cn · fitting_distributions.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/fitting_distributions.md)
[lecture-intro.zh-cn · observed_distributions.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/observed_distributions.md)
[lecture-intro.zh-cn · prob_dist.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/prob_dist.md)
⚠️ Reads a local `_static/lecture_specific/prob_dist/` copy, not this file, under a comment that still says to switch once the datasets repo exists | | [**usa-gini-nwealth-tincome-lincome.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/usa-gini-nwealth-tincome-lincome.csv)
US Gini coefficients — net wealth, total income and labour income, 1950-2016 | constructed | [Derived from SCF_plus_mini.csv (this repo), an extract of the SCF+ panel](https://github.com/QuantEcon/data-lectures/blob/main/lectures/SCF_plus_mini.csv) | | ✅ permitted | ⚠️ unverifiable | committed-frozen | 1.2 KB | [lecture-python-intro · inequality.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/inequality.md)
[lecture-wasm · inequality.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/inequality.md)
[lecture-intro.zh-cn · inequality.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/inequality.md)
[test-actions-lecture-intro · inequality.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/inequality.md) | diff --git a/builders/README.md b/builders/README.md index 6656647..9876e6d 100644 --- a/builders/README.md +++ b/builders/README.md @@ -39,7 +39,11 @@ re-fetched** — see `AGENTS.md`. | `japan_earthquakes.py` | `japan_earthquakes.csv` | committed | | `japan_population_by_age.py` | `japan_population_by_age.csv` | committed | | `us_adult_heights.py` | `us_adult_heights.csv` | committed | -| `NEWQDATA.py` | `NEWQDATA.csv` | committed — the **only** builder here that reads a committed input (`sources/NEWQDATA.MAT`) instead of fetching. Its upstream is published nowhere; see `sources/README.md`. Reproduces its output byte for byte | +| `NEWQDATA.py` | `NEWQDATA.csv` | committed — reads a committed input (`sources/NEWQDATA.MAT`) instead of fetching. Its upstream is published nowhere; see `sources/README.md`. Reproduces its output byte for byte | +| `dataBHS.py` | `dataBHS.csv` | committed — the second `sources/` reader (`sources/dataBHS.mat`, un-refetchable; see `sources/README.md`). A value-preserving MATLAB-to-CSV conversion; validates the consuming lecture's hardcoded moments on every run | +| `bbh_macro_quarterly.py` | `bbh_macro_quarterly.csv` | committed — range-reads one workbook out of the 198.8 MB Zenodo replication package. Reproduces its output byte for byte (2026-08-17) | +| `bbh_michigan_monthly.py` | `bbh_michigan_monthly.csv` | committed — same Zenodo package, different workbook. Reproduces its output byte for byte (2026-08-17) | +| `fred_data.py` | `fred_data.csv` | committed — fetches six FRED series live over a pinned 1953-04..2024-12 window (yields and the recession dummy are stable history, unlike the BBH national-accounts snapshot). Reproduces its output byte for byte (2026-08-18) | | `hansen_singleton_1982_data.py` | `hansen_singleton_1982_data.csv` | committed — fetches FRED and the Ken French factors live. Reproduces its output byte for byte (2026-08-13) | | `hansen_singleton_1983_data.py` | `hansen_singleton_1983_data.csv` | committed — the same construction plus a T-bill leg, so its output is a strict superset of the 1982 file's. Reproduces its output byte for byte (2026-08-13) | | `business_cycle.py` | `business_cycle_data.csv`, `business_cycle_info.md`, `business_cycle_metadata.md` | run by hand, no validate stage yet (PLAN Phase 5); its three outputs are the repo's only unmanifested files | @@ -67,13 +71,14 @@ here instead of patched — fixing it would mean this file is no longer the thin that produced those bytes. The fix belongs in `lecture-python-intro`, which still serves that notebook to readers. -**This listing is the coverage report.** The repo has 21 `constructed` datasets. -Fourteen ship a builder (9 `committed`, 5 `committed-frozen`), carried by **12** -distinct builder files — fewer than the datasets because `generating_mini.md` -and `webscrape_forbes.ipynb` each produce two. The remaining **7** have none: -they carry `builder_status: unrecovered` in their manifests, which is the Phase -9 recovery backlog, kept visible rather than hidden by reclassifying the file as -`verbatim`. The table above lists a **13th** builder, `business_cycle.py`, which +**This listing is the coverage report.** The repo has 28 `constructed` datasets +(re-derived from the parsed manifests, 2026-08-18). Eighteen ship a builder (13 +`committed`, 5 `committed-frozen`), carried by **16** distinct builder files — +fewer than the datasets because `generating_mini.md` and +`webscrape_forbes.ipynb` each produce two. The remaining **10** have none: they +carry `builder_status: unrecovered` in their manifests, which is the Phase 9 +recovery backlog, kept visible rather than hidden by reclassifying the file as +`verbatim`. The table above lists a **17th** builder, `business_cycle.py`, which no manifest references — its three outputs are the repo's only unmanifested files. diff --git a/builders/dataBHS.py b/builders/dataBHS.py new file mode 100644 index 0000000..09a9f93 --- /dev/null +++ b/builders/dataBHS.py @@ -0,0 +1,111 @@ +""" +Builder for lectures/dataBHS.csv. + +Converts the Barillas-Hansen-Sargent "Doubts or variability?" (JET, 2009) +MATLAB data file into the CSV the `five_preferences` lecture reads. Three +quarterly US series over 1948Q1-2006Q4: log real per-capita consumption and +two gross real asset returns. The file carries no date column; the sample is +stated in the paper and in the consuming lecture's prose ("1948.I-2006.IV"). + +This is a value-preserving container conversion and nothing else -- .mat to +.csv, no filtering, no rescaling, no reordering. The published CSV parses back +bit-exactly under pandas' correctly-rounded reader, and -- measured, not +assumed -- the consuming lecture's histogram of consumption growth has +identical counts and bin edges under pandas' DEFAULT parser, so the lecture +needs no float_precision flag. + +READS ITS INPUT FROM sources/, WHICH IS THE EXCEPTION, NOT THE RULE. +AGENTS.md permits it only when the input cannot be re-fetched, and this one +cannot: neither author hosts the replication files (tomsargent.com's source +page 404s, larspeterhansen.org lists no code or data for the paper), the +Journal of Economic Theory article carries no data supplement, and a +GitHub-wide code search finds only QuantEcon's own inherited copies of this +blob. Searched with positive controls 2026-08-18 -- see sources/README.md. + +Stages: fetch -> pre-process -> validate -> write. +""" + +import io +import os + +import pandas as pd +from scipy.io import loadmat + +CURRENT_FILE_DIR = os.path.dirname(os.path.abspath(__file__)) +REPO_ROOT = os.path.dirname(CURRENT_FILE_DIR) +PUBLISHED_DIR = os.path.join(REPO_ROOT, 'lectures') +SOURCES_DIR = os.path.join(REPO_ROOT, 'sources') + +SOURCE_FILE = 'dataBHS.mat' + +# The .mat holds three bare (236, 1) float64 arrays under these names, in this +# order. The output column order is the input order -- see validate(). +COLUMNS = ['c', 'rb', 'rs'] + +# 1948Q1 to 2006Q4 inclusive, quarterly, no gaps -- 59 years x 4. +N_QUARTERS = 236 + +# The consuming lecture hardcodes the mean and standard deviation of quarterly +# log consumption growth (five_preferences.md, "Set parameter values"). They +# are moments of THIS vintage, so they double as its fingerprint: a substituted +# or truncated input fails here rather than silently mis-plotting the lecture's +# approximating and worst-case densities against its histogram. +GROWTH_MEAN = 0.004952 +GROWTH_STD = 0.005050 + +OUT_FILE = 'dataBHS.csv' + + +def fetch(): + return loadmat(os.path.join(SOURCES_DIR, SOURCE_FILE)) + + +def pre_process(raw): + # Each array is (236, 1); ravel to 1-D so the frame is 236 rows, not 236 + # columns of one element. + return pd.DataFrame({name: raw[name].ravel() for name in COLUMNS}) + + +def validate(df): + """Refuse to write anything that is not the shape we expect.""" + assert list(df.columns) == COLUMNS + assert len(df) == N_QUARTERS, f'expected {N_QUARTERS} quarters, got {len(df)}' + assert not df.isnull().values.any() + assert (df.dtypes == 'float64').all() + + # c is LOG per-capita consumption; rb and rs are GROSS real returns. A + # vintage stored in levels, percentages or net returns would pass the + # structural checks above and quietly rescale everything downstream. + assert df['c'].between(-5.0, -3.0).all(), 'c is not log consumption' + assert df['rb'].between(0.9, 1.1).all(), 'rb is not a gross return' + assert df['rs'].between(0.6, 1.4).all(), 'rs is not a gross return' + + # The lecture's hardcoded moments of quarterly log consumption growth, + # reproduced to their printed precision. + growth = df['c'].to_numpy()[1:] - df['c'].to_numpy()[:-1] + assert round(growth.mean(), 6) == GROWTH_MEAN, growth.mean() + assert round(growth.std(), 6) == GROWTH_STD, growth.std() + + # The conversion contract: the CSV must parse back bit-exactly under the + # correctly-rounded reader. (pandas' default parser is fast, not correctly + # rounded -- PLAN-QELD-PACKAGE.md section 4.3 measured 18 of 708 values off + # by <= 2.1e-16 relative under 'high'. The lecture's histogram is identical + # either way, which is what lets the lecture keep a plain read_csv.) + buffer = io.StringIO() + df.to_csv(buffer, index=False) + buffer.seek(0) + back = pd.read_csv(buffer, float_precision='round_trip') + for name in COLUMNS: + assert (back[name].to_numpy() == df[name].to_numpy()).all(), \ + f'{name} does not round-trip bit-exactly' + + +def run(): + df = pre_process(fetch()) + validate(df) + df.to_csv(os.path.join(PUBLISHED_DIR, OUT_FILE), index=False) + print(f'wrote {OUT_FILE}: {len(df)} quarters x {len(df.columns)} series') + + +if __name__ == '__main__': + run() diff --git a/builders/fred_data.py b/builders/fred_data.py new file mode 100644 index 0000000..0f91e20 --- /dev/null +++ b/builders/fred_data.py @@ -0,0 +1,125 @@ +#!/usr/bin/env python3 +""" +Builder for lectures/fred_data.csv. + +Fetches the six FRED series the `risk_aversion_or_mistaken_beliefs` lecture +plots -- three nominal Treasury constant-maturity yields (GS1, GS5, GS10), two +real (TIPS) yields (DFII5, DFII10) and the NBER recession indicator (USREC) -- +monthly, over the fixed window 1953-04-01 to 2024-12-01, and writes them as +one date-indexed CSV. + +Unlike the BBH files this IS a live-FRED read, deliberately: none of these +series is revised the way the national accounts are. The nominal and real +yields are historical H.15 market rates and USREC is a dummy built from +NBER's published turning points, so the live values are stable -- measured +2026-08-18, a fresh fetch reproduced the committed file byte for byte. The +window end is pinned; this file is a frozen extract, not a tracking snapshot. + +Two fetch details that are easy to get wrong: + +- FRED publishes DFII5/DFII10 daily. The lecture's file carries their MONTHLY + AVERAGES, which fredgraph serves with `fq=Monthly&fam=avg`. GS1/GS5/GS10 and + USREC are monthly at source and need no aggregation. +- fredgraph.csv now titles its date column `observation_date` (it used to be + `DATE`). The committed file predates the rename, so the index is renamed on + read; a builder that trusted the served header would change the byte layout. + +Stages: fetch -> pre-process -> validate -> write. + +Requires pandas. +""" +import io +import os +import urllib.request + +import pandas as pd + +CURRENT_FILE_DIR = os.path.dirname(os.path.abspath(__file__)) +REPO_ROOT = os.path.dirname(CURRENT_FILE_DIR) +PUBLISHED_DIR = os.path.join(REPO_ROOT, 'lectures') + +OUT_FILE = 'fred_data.csv' + +FRED_CSV = 'https://fred.stlouisfed.org/graph/fredgraph.csv' +START, END = '1953-04-01', '2024-12-01' + +# Monthly at source. +MONTHLY = ['GS1', 'GS5', 'GS10', 'USREC'] +# Daily at source; fetched as monthly averages. +DAILY_AVERAGED = ['DFII5', 'DFII10'] +# Column order of the published file. +COLUMNS = ['GS1', 'GS5', 'GS10', 'DFII5', 'DFII10', 'USREC'] + +N_MONTHS = 861 # 1953-04 .. 2024-12 inclusive, no gaps +# FRED publishes the TIPS yields from 2003-01, so the first 597 months of the +# window are empty in both DFII columns and every other column is complete. +KNOWN_NULLS = {'DFII5': 597, 'DFII10': 597} +TIPS_START = pd.Timestamp('2003-01-01') + + +def _fetch_series(code): + url = f'{FRED_CSV}?id={code}&cosd={START}&coed={END}' + if code in DAILY_AVERAGED: + url += '&fq=Monthly&fam=avg' + request = urllib.request.Request(url, headers={'User-Agent': 'qeld-builder'}) + with urllib.request.urlopen(request) as response: + payload = response.read() + frame = pd.read_csv(io.BytesIO(payload), index_col=0, parse_dates=True, + na_values='.') + frame.columns = [code] + return frame + + +def fetch(): + return pd.concat([_fetch_series(code) for code in COLUMNS], axis=1) + + +def pre_process(fred): + fred = fred.loc[START:END] + fred.index.name = 'DATE' + fred['USREC'] = fred['USREC'].astype('int64') + return fred[COLUMNS] + + +def validate(frame): + """Refuse to write anything that is not the shape we expect.""" + assert list(frame.columns) == COLUMNS, list(frame.columns) + assert frame.index.name == 'DATE' + + # 1953-04 .. 2024-12 on an unbroken monthly grid of first-of-month stamps. + assert len(frame) == N_MONTHS, f'expected {N_MONTHS}, got {len(frame)}' + assert frame.index[0] == pd.Timestamp(START) + assert frame.index[-1] == pd.Timestamp(END) + assert frame.index.is_monotonic_increasing + assert (frame.index.day == 1).all() + months = frame.index.year * 12 + frame.index.month + assert (pd.Series(months).diff().dropna() == 1).all(), 'gap in the grid' + + # Exactly the declared holes, and nowhere else: the TIPS series before + # 2003-01, full stop. + nulls = frame.isnull().sum() + assert dict(nulls[nulls > 0]) == KNOWN_NULLS, dict(nulls[nulls > 0]) + for code in DAILY_AVERAGED: + assert frame.loc[frame.index < TIPS_START, code].isnull().all() + assert frame.loc[frame.index >= TIPS_START, code].notnull().all() + + # Units: percent per annum for every yield, 0/1 for the recession dummy. + # A fetch that silently switched to decimals or to an index would pass the + # grid checks above and rescale the lecture's figure. + for code in ['GS1', 'GS5', 'GS10']: + assert frame[code].between(0.0, 20.0).all(), f'{code} out of band' + for code in DAILY_AVERAGED: + assert frame[code].dropna().between(-3.0, 5.0).all(), f'{code} out of band' + assert set(frame['USREC'].unique()) <= {0, 1} + + +def run(): + frame = pre_process(fetch()) + validate(frame) + frame.to_csv(os.path.join(PUBLISHED_DIR, OUT_FILE)) + print(f'wrote {OUT_FILE}: {frame.shape[0]} months x {frame.shape[1]} series ' + f'({frame.index[0].date()} .. {frame.index[-1].date()})') + + +if __name__ == '__main__': + run() diff --git a/lectures/acs_data_summary.csv b/lectures/acs_data_summary.csv new file mode 100644 index 0000000..29a0e21 --- /dev/null +++ b/lectures/acs_data_summary.csv @@ -0,0 +1,352 @@ +count,mean_Earnings,std_Earnings +1331,14900.405709992487,16432.77000114295 +113,14945.044247787611,16555.977239779477 +536,16235.858208955224,14390.705658707586 +3788,16620.493928194297,15627.600938733565 +1337,16868.38444278235,20674.271930182003 +1455,17200.240549828177,15666.75943016007 +9900,17485.504646464648,16898.448536979464 +413,17855.375302663437,24934.411743555444 +317,18443.72239747634,18910.04500054648 +12370,18945.816572352465,21375.376002331996 +12031,19723.61815310448,19450.33475965967 +8501,19992.385601693917,26313.652148468223 +8800,20444.996704545454,16164.839628000927 +773,20644.954721862872,13843.979122054649 +7403,20739.541267053897,21147.46175638877 +583,21070.686106346482,13184.831707584104 +275,21266.98181818182,14862.656038103265 +1359,21793.024282560706,18438.68669968874 +2191,21903.929712460063,19776.452810936185 +58,22020.344827586207,11427.21619881441 +2745,22091.814207650274,18025.068526790623 +905,22172.475138121546,26617.647130774334 +654,23896.785932721712,26438.239579467132 +1373,24324.493809176984,21485.325471629778 +4016,24728.254482071712,19186.253404900126 +59,24757.627118644068,13522.576312760239 +1808,24881.766592920354,21619.594437915246 +5611,25004.48957405097,23614.704606262276 +968,25148.421487603307,38377.07442008185 +844,25229.348341232228,29279.511776925534 +276,25384.565217391304,16203.599964522593 +78,25448.71794871795,11606.690229141901 +445,25581.460674157304,27127.679937797322 +7586,25584.14909042974,23230.713830277662 +6467,26015.113653935365,20486.78966825072 +362,26178.701657458565,36192.85405241237 +455,26305.69230769231,36790.92885699715 +2155,26422.19489559165,22525.939195468625 +261,26515.708812260535,18824.594468394233 +193,26571.709844559584,33818.247365493444 +15991,26618.94715777625,25397.867018966135 +8857,26899.661736479622,25346.770850944646 +86,26936.04651162791,18352.143088771747 +189,27107.989417989418,21099.854849251024 +249,27184.176706827307,39977.39939420132 +295,27234.20338983051,21839.87282143491 +235,27331.063829787236,16073.20635525605 +611,27346.85761047463,16419.357945885145 +327,27385.443425076453,17875.40932739134 +16,27528.125,22813.8113778328 +2640,27693.99659090909,26681.361802938936 +166,27747.349397590362,22331.91891953525 +515,27771.844660194176,25404.165049131756 +2467,27878.57965139846,26307.11527752712 +481,28258.89812889813,16958.543502956967 +3803,28274.057323165922,21546.587241382982 +267,28470.674157303372,23340.82883106868 +67,28489.10447761194,46698.42203327656 +364,28600.274725274725,44244.289584147 +266,28997.48120300752,15988.964261640675 +90,29713.333333333332,20165.735763328958 +51,29776.470588235294,11574.29199969128 +81,29792.716049382718,22010.41747743246 +801,29881.373283395755,20583.989667013906 +1021,30278.530852105778,23218.795813219058 +177,30332.20338983051,17935.78926601116 +486,30465.637860082305,28259.00371625996 +3405,30653.36563876652,23401.127379700003 +422,30680.52132701422,27465.065363194262 +227,30769.471365638765,42474.80173020462 +246,30885.731707317074,55546.60613517166 +13320,31045.933183183184,26946.80530452291 +1627,31137.301782421633,24389.434720529644 +2115,31139.739952718675,24737.420133310752 +690,31313.75507246377,39546.37376933845 +234,31333.076923076922,19440.086409031836 +3528,31353.257369614512,30201.88617852633 +55,31432.727272727272,17733.19814636707 +2556,31740.395148669795,23858.236519580147 +226,31883.893805309734,20194.11051365129 +4184,31913.109464627152,20005.96503471116 +3216,32061.293843283584,37659.195701316865 +115,32449.565217391304,33948.000416305025 +124,32482.016129032258,30264.572488880414 +1571,32499.86632718014,22220.855809623023 +459,32765.054466230937,29315.317406751692 +1229,32881.47111472742,30461.849977536294 +214,32882.056074766355,29665.01036379667 +70,32884.28571428572,16767.59846185654 +216,33215.416666666664,40694.92326274207 +2473,33393.8819247877,23015.060530423583 +8822,33526.67671729766,27492.126718365995 +4217,33726.49276737017,26718.033992494773 +307,33792.44299674267,26712.79555993049 +3990,34137.160401002504,22439.7319837709 +229,34149.34497816594,40238.782484030686 +7184,34210.84632516704,26237.520916352805 +80,34211.25,18678.499070633203 +257,34308.21011673152,23132.80010145342 +371,34429.757412398925,37069.96194480972 +941,34452.96493092455,25627.40342289543 +5733,34578.5853828711,32184.155611732942 +127,34662.04724409449,17078.78696012971 +444,34668.80630630631,37250.29975389088 +927,34846.03020496224,26688.921318718658 +898,34897.29510022272,36904.565568355814 +912,35151.304824561405,37031.80265454963 +11425,35212.49426695843,32120.41867256671 +3685,35402.0461329715,23899.45809249939 +935,35720.95935828877,31137.61957437525 +25472,35847.70846419598,27026.4837865098 +125,35863.36,32142.15057301027 +1092,36288.919413919415,19371.86049014382 +69,36692.76811594203,29093.918151669208 +74,36800.0,26658.59752345963 +6188,36829.22915319974,27642.010924837155 +16323,36919.59504992955,36092.85778402987 +814,37040.78746928747,20638.92809201809 +260,37185.846153846156,30210.24643442601 +5042,37199.77072590242,37321.19404953267 +517,37514.85493230174,26726.8595597498 +17530,37572.28807758129,46212.2496496395 +9662,37583.05837300766,29650.724538505343 +375,37587.706666666665,27981.512490403795 +698,37674.140401146135,24760.513994609373 +9364,38058.768154634774,29711.909070934867 +605,38081.90082644628,23280.336767112032 +3214,38167.74113254512,23812.013413820972 +97,38174.742268041235,17783.972951601645 +94,38236.17021276596,42287.02450030028 +194,38319.58762886598,19153.0575714908 +8294,38398.73306004341,30491.30807084761 +558,38404.74910394265,26292.91925062153 +1159,38474.03019844694,42600.93738514701 +1208,38751.82698675497,54174.68548176281 +746,38856.635388739945,40640.23796181628 +1084,39038.4778597786,28601.32669679299 +933,39267.71918542337,32088.15983517525 +643,39439.28460342146,20534.148204346457 +529,39501.32325141777,36024.72238957143 +1391,39717.54852624011,31001.739732344027 +363,39813.691460055095,23568.937152323517 +381,39837.40157480315,29296.054715859427 +348,40133.96551724138,21622.060389069033 +229,40141.18340611354,34957.815699927174 +1374,40187.91848617176,25394.113583574104 +265,40230.83018867925,49287.03812706216 +423,40320.3073286052,35871.12588392239 +843,40352.9418742586,34984.772515512734 +207,40529.371980676326,18488.14469501662 +668,40725.37425149701,28373.754221909596 +431,40830.533642691415,31645.303379706 +280,40862.0,41453.58181687112 +1334,41027.22488755622,21706.293340332068 +111,41052.252252252256,21810.659807646378 +4703,41081.73931533064,30155.6280550901 +655,41335.87786259542,27797.56459945549 +2204,41376.19328493648,27692.15659180858 +175,41430.857142857145,26229.435663736567 +6045,41482.893300248135,36341.09627465994 +600,41527.473333333335,46211.735299325366 +2916,41577.743484224964,31769.736154906233 +691,41609.75397973951,29195.504519399656 +221,41659.72850678733,23336.66755672774 +964,41809.242738589215,25307.404897071097 +257,42811.284046692606,35015.27999474527 +175,42911.08571428571,23269.90880323775 +192,43063.541666666664,21051.575259319146 +117,43194.8717948718,20689.856110872162 +636,43281.588050314465,20160.23022366003 +1489,43330.597716588316,40353.07331609059 +277,43351.73285198556,24633.325751226825 +64,43659.375,18341.753207212383 +197,43745.93908629441,27462.099741286213 +888,43828.04054054054,28074.22863018439 +1069,44094.79981290926,29729.09800287853 +242,44177.396694214876,26173.4245868871 +91,44213.18681318681,26434.641450571416 +272,44218.82352941176,29516.530737921956 +167,44351.796407185626,58675.72744099032 +4611,44567.13294296248,30221.6451884297 +2132,44736.44512195122,31898.371997271137 +198,44871.919191919194,46820.006985967295 +1324,44981.56344410876,26774.104176795416 +5914,45026.84984781873,32824.83888195073 +66,45305.15151515151,28342.115725944735 +97,45441.237113402065,21805.325957670862 +745,45480.040268456374,24964.540446454812 +24474,45618.37149628177,37652.8508249016 +4008,45730.57634730539,25712.302187776186 +184,45741.30434782609,31541.84367018702 +1483,45752.27242076871,33880.20921041752 +292,45850.68493150685,35447.424630448375 +205,46430.73170731707,32978.16392752343 +1987,46552.1791645697,24796.0699878023 +480,46663.333333333336,19953.078511888845 +790,46709.91139240506,26102.47071751405 +724,47023.4544198895,33101.32035161209 +2333,47139.80325760823,27632.913326289738 +2760,47171.37463768116,29721.461675688308 +132,47271.969696969696,18757.46493391808 +420,47279.333333333336,28742.175401487362 +1344,47448.56398809524,34781.02022125067 +2409,47598.87920298879,33112.226785536164 +1182,47735.897631133674,53417.473248027534 +2753,48275.37595350527,25392.883035987077 +355,48282.81690140845,48664.66383493207 +920,48287.891304347824,24054.77273442469 +30711,48485.85624043502,25974.994170520782 +1510,48623.9821192053,58496.75850772877 +912,48924.59429824561,45467.12428353692 +1431,49157.00908455625,67020.1922750646 +182,49536.26373626374,26353.35749305159 +2785,49738.92315978456,22858.619677648265 +3355,49961.701937406855,27507.04969097614 +2910,50379.9587628866,32544.59237225434 +85,50483.529411764706,48670.047616722186 +352,50664.28977272727,39921.32811931963 +431,50763.96751740139,36132.32513053202 +727,50929.40852819807,26898.772377839156 +2581,50996.691204959316,34830.513679188836 +1558,51065.64184852375,46185.647507106645 +3991,51556.95088950138,44608.90529421183 +6397,52829.89057370643,26272.447766654015 +1298,52961.702619414486,30427.788117067503 +3310,53273.045317220545,27325.337718782423 +255,53495.686274509804,40788.49735834336 +1766,54103.9637599094,36140.820368584136 +1161,54156.770025839796,35941.12992327487 +876,55171.57534246575,48064.29058816983 +487,55279.05544147844,39941.18068275288 +126,55473.01587301587,74930.57737687431 +1692,55520.135933806145,37275.07901366944 +6575,55656.67634980989,52923.80018420368 +505,55812.158415841586,30225.258521198244 +87,55926.4367816092,31894.668431653958 +1195,55943.054393305436,26573.791302972855 +60,56070.0,27083.169456262134 +2927,56210.256235052955,37252.18885203767 +104,56230.730769230766,22789.881803682973 +452,56331.50442477876,54095.804251743946 +176,57216.47727272727,36990.113135879896 +445,57314.831460674155,45440.10219663298 +2363,57923.77486246297,39485.808205901696 +131,58089.31297709924,25045.909390890094 +2924,58194.476744186046,30824.85153938926 +318,58424.55974842767,89388.3005190242 +271,58573.87453874539,44868.13525051241 +162,58727.16049382716,30604.836675139595 +1418,58852.99788434415,53490.17855659326 +71,58945.77464788732,48006.99260560076 +569,59005.92267135325,32993.73118382685 +756,59329.40476190476,48712.277259477036 +322,59479.19254658385,34228.098283699226 +199,59486.38190954774,32873.24013763075 +2285,59627.90371991247,44712.67296198911 +1005,59827.164179104475,45586.46427960656 +62,60098.3870967742,53146.91821158291 +185,60263.24324324324,55709.45347798263 +1346,60270.31203566122,47423.885049741664 +254,60493.700787401576,37174.341416732925 +596,61021.54362416107,43967.92213390216 +166,61025.301204819276,54169.58744954755 +367,61102.12534059946,25381.578543313 +217,61188.01843317972,57184.118199863464 +268,61353.35820895522,34154.96997069272 +10816,61562.85965236687,55996.039197531405 +259,61680.50193050193,67521.4391458056 +155,61690.32258064516,29792.071145558497 +1712,62005.94976635514,82004.00011630662 +651,62063.133640552995,57119.25601766312 +704,62695.923295454544,51568.97804067012 +449,62741.98218262806,68033.4473468201 +1025,62830.55609756098,45748.607652384984 +587,63250.68143100511,62428.68316557238 +735,63420.91156462585,32404.27322144226 +162,63453.7037037037,31927.773962062136 +302,63934.17218543046,48465.51401749722 +663,64399.034690799395,95690.3002845804 +388,64507.47422680412,32780.25924528305 +671,64980.76005961252,39938.30308490889 +477,65159.95807127882,40091.931783288856 +212,65817.45283018867,41448.393026466496 +561,65985.20499108735,56380.00701872179 +1071,67327.55368814192,42492.081956091955 +1954,67511.62538382804,71960.90116183538 +5979,67784.04749958187,33363.825168154886 +5857,67795.41061977121,60494.66027027032 +525,68352.09523809524,42952.24731066717 +109,68780.73394495413,29726.978260384523 +443,69287.47178329571,32411.255550407794 +236,70715.25423728813,64920.68762121735 +457,71396.3238512035,49319.07527537085 +1476,71814.65447154471,53727.080294440704 +201,72201.99004975124,42954.59342165748 +889,72983.59955005624,52618.60901769051 +3955,73775.20960809103,83809.58210059197 +2484,74696.73510466989,72022.22140203201 +981,76734.77064220184,40100.00367796557 +5456,76824.88178152492,94015.14422905697 +1920,78107.23958333333,61634.14362029448 +1070,78638.73831775702,36784.3423953724 +999,79330.23023023023,92468.32050622275 +256,79656.6015625,43021.64179250923 +691,80269.60926193921,54467.91771833894 +1949,80882.58081067214,61989.21190635767 +133,81566.31578947368,36265.32857673165 +344,81724.04069767441,70318.5743263255 +396,81917.67676767676,54724.49654573229 +199,82389.94974874372,37013.348663561796 +405,82656.04938271605,31312.455912066045 +1242,83829.34782608696,43418.88769695796 +11079,84068.53678129795,78540.05144529899 +945,84390.61375661376,42853.130060461604 +154,85905.84415584416,66653.03119086035 +1038,85910.73217726397,92576.05740925772 +312,87788.94230769231,79559.47348408098 +4880,88004.68032786885,92082.56051412193 +2025,88097.89135802469,43108.77694566431 +77,88574.02597402598,58833.73853629576 +478,88867.78242677824,36799.042458869335 +1250,88905.3584,93591.33823782875 +1273,89346.09269442262,74507.52912062789 +359,89509.55431754874,54599.991408533984 +1482,90342.52091767882,79346.52467286953 +97,90355.67010309278,34388.97282400326 +460,91395.43478260869,38724.948762694294 +831,92410.26474127558,55721.05171320993 +2570,92907.28171206226,57043.36881076245 +603,93742.68656716419,85058.5568150245 +438,95017.35159817351,79864.18001371343 +105,98062.85714285714,65425.62168239192 +453,98758.25607064017,116134.01267427258 +292,99711.98630136986,53726.51594537541 +4496,100022.6859430605,56900.20721783729 +1690,102356.9349112426,56626.570667655185 +456,104803.48684210527,71311.26958940338 +2404,106926.18552412646,63719.8232169186 +71,110228.1690140845,73937.56668027153 +304,113066.94078947368,64585.43476419163 +505,113803.78217821782,60385.33964300992 +1835,116122.87738419618,116295.86226115789 +301,121848.67109634551,89534.09110628269 +1130,130472.74336283185,97564.62817055204 +337,145268.84272997032,105113.47047426814 +2825,146967.46548672568,148283.85967018566 +1874,151577.6152614728,163188.46424002026 +8874,156727.0739238224,135097.47769384919 +67,174783.58208955225,137227.9739979436 +1224,188562.70506535948,141456.3948276375 +7405,231459.67832545578,166980.91397517134 diff --git a/lectures/acs_data_summary.csv.yml b/lectures/acs_data_summary.csv.yml new file mode 100644 index 0000000..15ad6f9 --- /dev/null +++ b/lectures/acs_data_summary.csv.yml @@ -0,0 +1,155 @@ +# Manifest for acs_data_summary.csv — migrated out of +# QuantEcon/lecture-python-advanced.myst, where it sat at +# lectures/_static/lecture_specific/match_transport/ and was read by relative +# path (PLAN Phase 8, wave C2). +# +# The name is on the phase-2 rename review (QuantEcon/data-lectures#87, +# decision of 2026-08-17): it migrates under its current name. The org-wide +# Trees sweep (2026-08-17, re-derived in QuantEcon/data-lectures#96) found +# four copies of this basename org-wide and all four are THE SAME BLOB +# (29a0e219…) — inherited duplicates in lecture-dp, lecture-dp.monorepo and +# 2026-tom-course, none with a consuming lecture — so there is no collision +# and no sibling consumer to keep in step. The lecture-dp copy is already +# annotated as an orphan under committed_unreferenced in +# scripts/audit_annotations.yml; that entry stays. +# +# This file is the corpus's least documented: 351 rows of (count, mean, std) +# for unnamed occupation cells — no occupation identifiers, no year, no +# extraction script. The `unrecovered` below is honest and inherited-only +# (AGENTS.md, "Two inherited-file states"); recovery would mean re-running an +# ACS extraction whose filters, vintage and occupation coding are recorded +# nowhere. + +filename: acs_data_summary.csv +title: American Community Survey — earnings count, mean and dispersion by occupation cell +description: > + 351 occupation cells from the American Community Survey, each carrying the + number of individual observations (count) and the mean and standard + deviation of earnings (mean_Earnings, std_Earnings). The rows carry NO + occupation identifiers — order is the only key — and no survey year is + recorded anywhere. `match_transport` reads it to plot the mean-dispersion + scatter of earnings across occupations that motivates its composite-sorting + application, citing Boerma, Tsyvinski and Zimin's composite-matching paper. + +# Constructed: an aggregation (count/mean/std by occupation) of ACS microdata, +# performed by whoever assembled the lecture. Not verbatim — the ACS +# distributes microdata, not this summary. +class: constructed + +source: + name: American Community Survey (US Census Bureau) — occupation-level summary, assembly unrecorded + url: https://www.census.gov/programs-surveys/acs + doi: null + version: > + Unknown — the file records no survey year and no ACS vintage, and the + consuming lecture's prose says only that the data "is obtained from the + American Community Survey and contains individual level data on income, + age and occupation" before this summary was formed. + series: count, mean_Earnings, std_Earnings — by unnamed occupation cell + citation: > + US Census Bureau, American Community Survey (microdata; vintage + unrecorded). Aggregation to occupation cells by the lecture authors, + accompanying the treatment of Boerma, Joel, Aleh Tsyvinski, and Alexander + P. Zimin's composite-sorting framework in the `match_transport` lecture. + note: > + The construction (ACS filters, the occupation grouping, the earnings + definition, the vintage) is described nowhere beyond the lecture's + one-sentence prose. The rows are position-keyed only: a consumer that + sorts or filters silently changes which occupation is which. Treat the + file as frozen — there is no documented way to rebuild or extend it. + +license: + name: null + url: https://www.census.gov/programs-surveys/acs + # Recorded as found, per the record-and-track policy (AGENTS.md, "Licensing + # and attribution"). + redistribution: permitted + verified: 2026-08-18 + note: > + An inherited exposure, served publicly from lecture-python-advanced.myst. + `permitted` rests on what the numbers are: ACS data is a US Census Bureau + product (a US Government work in the public domain), and what is published + here is not even the microdata but a 351-cell statistical summary of it. + No permission has been sought and none is required for census-derived + aggregates. Nothing to register on QuantEcon/data-lectures#35. + +retrieved: null # inherited-undated: no retrieval date was + # recorded and none is reconstructed from git + # history (AGENTS.md). Paired, as the rule + # requires, with the honest `unverifiable` + # below — for this file the vintage is pinned + # by nothing at all, which is exactly what the + # integrity note records. +maintainer: QuantEcon + +# --------------------------------------------------------------------------- +# Integrity (PLAN Phase 7) +# --------------------------------------------------------------------------- +# Migration check (repoint gate, not a manifest field): the bytes landing here +# are byte-identical to the copy the lecture reads today — sha256 719a3353… +# against lecture-python-advanced.myst @ 6ff5ab3, +# lectures/_static/lecture_specific/match_transport/. The three other org +# copies are the identical git blob (29a0e219…) and none has a consuming +# lecture. A repoint to this file therefore cannot change a figure. + +integrity: + sha256: 719a3353a1308d00c03a93aa70d71ac5f279f9e0122175c750459b0e1fe20d90 + upstream: + # There is nothing to verify AGAINST: no vintage, no extraction script, + # and no occupation identifiers to line the rows up with a fresh ACS pull. + # An honest `unverifiable` with the reason recorded beats a silence or a + # guessed `verified`. + status: unverifiable + date: 2026-08-18 + against: null + note: > + Unverifiable by construction, not by neglect: the file has no occupation + key, no year, and no recorded filters, so no ACS extraction can be + compared cell-for-cell against it. Any future re-derivation would be a + NEW dataset under a new name ("Corrections vs vintages", AGENTS.md). + +# --------------------------------------------------------------------------- +# Shape +# --------------------------------------------------------------------------- +# Measured from the committed bytes 2026-08-18 (pandas 2.3.3). + +schema: + format: csv + columns: + - {name: count, dtype: int64, description: "number of individual ACS observations in the occupation cell. Observed 16-30711"} + - {name: mean_Earnings, dtype: float64, description: "mean earnings in the cell, dollars (vintage-year dollars, year unrecorded). Observed 14900.41-231459.68. Rows are sorted ascending on this column, which is the file's only ordering"} + - {name: std_Earnings, dtype: float64, description: "standard deviation of earnings in the cell. Observed 11427.22-166980.91"} + row_count_floor: 351 # exact by design: a frozen summary with no + # documented way to grow + date_range: null # the file records no dates at all + known_nulls: {} # genuinely none — 1,053 cells, zero nulls + +# --------------------------------------------------------------------------- +# Consumers — how a correction knows what to rebuild +# --------------------------------------------------------------------------- +# One consumer, established by basename grep across all nine lecture clones +# plus the org-wide Trees sweep (the four org-wide basename hits are one blob; +# the lecture-dp, lecture-dp.monorepo and 2026-tom-course copies have no +# consuming lecture — verified in QuantEcon/data-lectures#96). The generated +# lecture-python-advanced.notebooks mirror self-heals after a publish tag and +# is deliberately not listed (wave A4 precedent). +# +# Note for whoever repoints: the read assembles its path from a `data_path` +# variable (match_transport.md:2234-2235), so the repoint edits the PREFIX +# line, not the read line — and the prose sentence naming the dataset sits +# three lines above (match_transport.md:2231), the only prose pairing in the +# whole track. +consumers: + - repo: QuantEcon/lecture-python-advanced.myst + file: lectures/match_transport.md + note: > + Reads its own copy at + `lectures/_static/lecture_specific/match_transport/`, not this file — + the repoint PR in lecture-python-advanced.myst follows this one, and + migration.yml records the dataset as `landed` until it merges. Listed + now because a correction to these bytes must reach the lecture + regardless of where it currently reads them from + (QuantEcon/data-lectures#91). + +builder: null +builder_status: unrecovered diff --git a/lectures/dataBHS.csv b/lectures/dataBHS.csv new file mode 100644 index 0000000..3bde90f --- /dev/null +++ b/lectures/dataBHS.csv @@ -0,0 +1,237 @@ +c,rb,rs +-4.6556074258422715,0.992677102,0.987307565 +-4.64557122745485,0.992688408,1.105162208 +-4.646285244280673,0.994077267,0.916425357 +-4.637080445740114,1.005303926,0.9974338 +-4.633506596654735,1.011945318,1.022930338 +-4.631496502346037,1.00995677,0.96519588 +-4.6393310816353415,1.008716264,1.124403964 +-4.633707735885377,1.000793485,1.102723285 +-4.623312873746522,1.005108888,1.047004668 +-4.606122839618498,1.001156954,1.021085724 +-4.592499992959327,0.979866537,1.096356837 +-4.594105921334169,0.99008186,1.07339447 +-4.57064161759515,0.971070418,1.018588768 +-4.570627706417248,0.99844581,0.994708929 +-4.555743938448301,1.001301179,1.126956879 +-4.550526804414256,0.989882027,1.003265917 +-4.547574240101472,1.003136042,1.035861042 +-4.531190064864107,1.001049711,1.017762041 +-4.519635999507332,0.997437725,0.978888575 +-4.508427305716663,0.999229317,1.079891397 +-4.509512339354156,1.002921131,0.981466701 +-4.502450975507638,1.004477666,0.962366046 +-4.5048156594114594,0.998311134,0.976950294 +-4.511834864023374,0.99886155,1.07237344 +-4.505694783688423,0.998883817,1.107055517 +-4.4990334616581755,1.001301146,1.087724106 +-4.486636409985554,1.003424983,1.093642953 +-4.47645107885784,1.001986161,1.137020061 +-4.467717690943375,1.001056225,1.035208315 +-4.460135732975513,1.0018795,1.1116104 +-4.454755103395538,1.002344487,1.019704047 +-4.43788435125797,1.00310448,1.05790605 +-4.4304928553186835,1.001183193,1.074886603 +-4.428245834605974,0.9999192611,0.984861358 +-4.4242415989261525,0.999430452,0.960637152 +-4.415896931665582,1.00443729,1.04362837 +-4.413597470097327,0.998391458,0.963433798 +-4.4100197720086305,1.003597046,1.073173036 +-4.403522422557802,0.998472888,0.892895028 +-4.403103765004122,1.003892432,0.942784052 +-4.41051146263232,0.993120109,1.059925299 +-4.400481482632812,1.000200909,1.085385979 +-4.388282642748578,1.004069857,1.114594557 +-4.383734412095975,1.005398439,1.113327339 +-4.376688428284799,1.000126977,1.018323467 +-4.365856085591377,1.007033568,1.057794168 +-4.358751533595485,1.004755988,0.970146118 +-4.351626788151,1.005751923,1.057932883 +-4.35201240217586,1.007248555,0.933717625 +-4.344437867805469,0.999037506,1.034477316 +-4.350463163512791,1.001623734,0.945105944 +-4.347787926508504,1.001135776,1.089538716 +-4.342612094152492,1.004238893,1.1352125929999999 +-4.331053711599741,1.007486684,1.003881942 +-4.331693433027904,1.002461186,1.033236026 +-4.31704776050594,1.005598413,1.073960063 +-4.309740429501824,1.001728974,0.972956884 +-4.302383933145983,1.004098725,0.783681125 +-4.2986662611603,1.004490106,1.032796936 +-4.294244501459826,1.003780759,1.127687359 +-4.294730298289037,1.003904485,1.059447855 +-4.290830286903178,1.006811645,1.049367825 +-4.280046746904622,1.004597713,1.034975483 +-4.276057464283933,1.006287268,1.044675128 +-4.2636366365897995,1.003488263,1.056870633 +-4.251729025094168,1.005651997,1.034788467 +-4.240438881844639,1.004425147,1.034478957 +-4.235309250606346,1.006384975,1.012652265 +-4.22917803647281,1.005922278,1.030076038 +-4.220488225729308,1.00314415,0.97144017 +-4.209789016991614,1.004298243,1.075984753 +-4.18924121554029,1.004989909,1.037181009 +-4.182786409118862,1.001996026,0.968856236 +-4.174954385675396,1.002882442,0.952204382 +-4.1699521197305796,1.005267521,0.900020911 +-4.166696311284937,1.005492341,1.061003341 +-4.15968314732551,1.007318396,1.137768996 +-4.154376835761422,1.004412637,1.017988517 +-4.150726867900153,1.003117257,1.068632617 +-4.14835571506424,1.004429337,1.001839315 +-4.135867677470062,1.000801244,0.925299044 +-4.124214067326781,1.003252996,1.117750996 +-4.114261840890498,1.00205029,1.02510469 +-4.111590542926189,1.004453919,1.018502899 +-4.106100897920703,1.004267468,0.961501318 +-4.101698778299348,1.002849328,0.947265668 +-4.099461813045028,1.004699857,0.949105227 +-4.093819739965397,1.007500899,0.99158285 +-4.089340543234876,1.00328023,0.96282838 +-4.090590915975177,1.004604331,0.7899698310000001 +-4.086186232083147,1.005804832,1.166681832 +-4.084212366719192,0.999697541,1.087466351 +-4.083457086468004,1.000374025,1.103932125 +-4.081490135508679,1.000438757,0.988923117 +-4.081321926032994,1.000496757,0.982966506 +-4.074223338624289,1.00263241,1.04003281 +-4.073466865960882,0.998891545,1.054393635 +-4.061729309474289,1.00282605,0.994343222 +-4.053634093920663,1.002101435,1.019596985 +-4.040241451069559,1.002257262,1.061671722 +-4.035073741086294,1.001689063,0.916878343 +-4.037022611754995,0.998054941,0.907849031 +-4.036246310168478,1.000225404,1.053563864 +-4.039732805027899,0.995979795,0.873476395 +-4.049694671453867,0.988327458,0.946797718 +-4.04982688356291,0.992636438,0.878543618 +-4.050317552454664,0.995818348,0.730246248 +-4.05421597285164,0.99595296,1.07274491 +-4.051880364855367,0.995089207,1.219587907 +-4.04169727489684,1.000588848,1.148442448 +-4.040704065670079,0.995907291,0.864073391 +-4.0373308755212856,0.997321365,1.066535655 +-4.028026232134553,1.000874967,1.148356967 +-4.023469950319691,1.00477982,1.01798946 +-4.017769039833221,0.99715553,1.0030525 +-4.00907744557045,0.995869624,1.025662684 +-4.006291982472242,0.992646415,0.915154295 +-4.0077083105831885,0.994153946,1.023840956 +-4.002661808123991,0.999697935,0.956127945 +-3.993531641216127,1.000091864,0.991709021 +-3.9862395680929885,0.999253711,0.944283111 +-3.978106345851491,0.995598495,1.068142585 +-3.9756642225966954,1.001485419,1.071092619 +-3.9727085853524886,1.003081105,0.921888485 +-3.969009739888196,1.005141515,1.061274465 +-3.968951463357041,0.995275832,1.008762642 +-3.967082805252683,0.998367946,1.051605716 +-3.965011841697928,1.004892491,0.984443543 +-3.968625632569669,1.00848906,0.91490336 +-3.981041466528223,0.993891081,1.123633581 +-3.9779524668080732,1.002935473,1.0987404729999999 +-3.969910838170569,1.013778305,1.066434435 +-3.9739126113747445,1.004419009,0.986854869 +-3.970813703660906,1.021111088,0.968031838 +-3.974074594878794,1.020025695,0.875396995 +-3.9757703729436273,1.011569683,1.059639503 +-3.975463942464915,1.018780966,0.909111946 +-3.9746643406947997,1.021723357,0.982061501 +-3.969687553214627,1.002745118,1.098165318 +-3.957999982820713,1.008037286,1.174986986 +-3.9503553932592252,1.012495999,1.093185799 +-3.9401930823951625,1.012302233,1.107636333 +-3.928343564167737,1.008239722,0.984475173 +-3.920345642693546,1.015759609,0.991900743 +-3.920391322325556,1.011408954,0.958158814 +-3.9103494565925287,1.014068117,0.965266227 +-3.9036771789256934,1.017841245,1.088191565 +-3.8960781529227697,1.013588721,1.012252621 +-3.8838469548637216,1.01032603,1.08605057 +-3.87748100888107,1.009106722,1.067738262 +-3.8679129664555623,1.010480454,0.948537424 +-3.86229152253347,1.009115324,1.159026924 +-3.8612553691746276,1.007561273,1.132909973 +-3.857398181752254,1.015361063,1.053189383 +-3.85398940800643,1.005759182,0.926344622 +-3.84686954840241,1.006443629,1.037519239 +-3.838998033793551,1.002651488,1.192512488 +-3.832651030337733,1.005435622,1.030478152 +-3.828693857865169,1.005632241,1.052163981 +-3.823517927898152,1.005283221,0.764888921 +-3.814781056328545,1.003980287,1.0602750969999999 +-3.808991800509453,1.004023248,1.052815088 +-3.7997614518036746,1.005345764,0.990805022 +-3.7928716175039985,1.00980955,1.01722688 +-3.790369956830286,1.0092266,1.05742983 +-3.78914554618951,1.00558766,1.07042324 +-3.7840374375007233,1.012334829,1.094299129 +-3.7760843189120337,1.010021624,1.004210544 +-3.78194014591461,1.003421919,0.950957949 +-3.7743732922392614,1.0071236,1.04214687 +-3.7703614337092413,1.004265032,0.850953232 +-3.776924616184365,1.002825524,1.071078654 +-3.7800451939861537,1.007418635,1.137631435 +-3.772868986931704,1.007949724,0.993093998 +-3.7723709386543076,1.005641073,1.049515073 +-3.773685163559446,1.001741277,1.072180847 +-3.7624681272805662,1.001257919,0.972321699 +-3.7590373611341663,1.002458933,1.006000613 +-3.753263277884429,0.999526215,1.022926975 +-3.746079167630234,1.000947525,1.049545765 +-3.745745171970005,1.001566,1.04100166 +-3.745163751281674,1.001024222,0.99914432 +-3.7344400748772064,1.004148299,1.027917309 +-3.7306825547614495,1.00245674,1.01486875 +-3.7237988615601267,1.004670552,0.957593462 +-3.7194978495261024,1.004948297,0.994306518 +-3.715591875144725,1.0023815,1.03944634 +-3.7120403284145977,1.009136228,0.986205457 +-3.708836518134853,1.009322256,1.085258436 +-3.7031004143370745,1.007198233,1.077016933 +-3.699879978627473,1.007876967,1.077202187 +-3.6974234099761842,1.007771113,1.050989213 +-3.691024152335877,1.005493736,1.051731816 +-3.6858084943415377,1.004357189,1.029420619 +-3.682576241164825,1.007647036,1.023448586 +-3.678966312071075,1.004122324,1.065569134 +-3.6753934405659097,1.007069017,1.012469087 +-3.6717857868566246,1.008736917,1.160324517 +-3.66218922832594,1.008315171,1.078144171 +-3.6568010326640037,1.008330385,1.026899155 +-3.6471910957952005,1.010594715,1.124351315 +-3.6381494963706817,1.009546311,1.012820231 +-3.631253431842076,1.006630613,0.867015213 +-3.6273926722995595,1.005754305,1.189435305 +-3.619114763222892,1.007116223,1.012201933 +-3.61089478891158,1.003236441,1.068846201 +-3.605032655792246,1.004811123,0.907706473 +-3.5948684451394133,1.005361222,1.094611422 +-3.598792893684837,1.003510817,0.995939122 +-3.590909469274724,1.008438035,0.989140482 +-3.585226211016875,1.008770993,1.036502633 +-3.5789312261198347,1.008916375,0.987984918 +-3.5791818173778123,1.00122632,0.89821973 +-3.579196441919062,1.000338037,1.041849247 +-3.578458921959506,1.003967441,0.877686841 +-3.5738394838905716,1.002046945,1.087971215 +-3.570682139838347,1.000693687,1.016585817 +-3.567789075971493,0.99560028,0.88496248 +-3.568408797792177,0.998337715,0.834343215 +-3.565583606412248,0.997333072,1.062030802 +-3.567032969902794,0.992488986,0.952951486 +-3.56527835404335,0.998912555,1.157939455 +-3.557759406446244,0.994346647,1.015192917 +-3.5550399189701967,0.996771582,1.122753982 +-3.5443165677787434,0.992066685,1.025422485 +-3.5394573650426437,0.993101274,0.99665879 +-3.5344988692882864,0.998095604,0.992642304 +-3.5278811642027197,0.997057008,1.085320648 +-3.5236393583496297,1.000663502,0.986994218 +-3.5187167845727054,0.998523773,1.012290273 +-3.51402907049237,0.996017263,1.033717493 +-3.510907208543307,1.001163648,1.013463628 +-3.5059171026727323,1.00532985,1.0500210700000001 +-3.501679980391295,1.000377173,0.98209101 +-3.4990674872824004,1.005057705,1.037812005 +-3.491831169850087,1.014024135,1.074967115 diff --git a/lectures/dataBHS.csv.yml b/lectures/dataBHS.csv.yml new file mode 100644 index 0000000..c75bb27 --- /dev/null +++ b/lectures/dataBHS.csv.yml @@ -0,0 +1,196 @@ +# Manifest for dataBHS.csv — the format-converted successor to +# lecture-python-advanced.myst's lectures/dataBHS.mat (PLAN Phase 8, wave C2; +# conversion decided 2026-08-17 on QuantEcon/workspace-lectures#45). +# +# THE CONVERSION IS A BUG FIX, NOT TIDYING. The .mat sat at lectures/ — outside +# html_static_path — so Sphinx never copied it into the published tree: +# python-advanced.quantecon.org/dataBHS.mat is 404 while the published +# five_preferences.ipynb is 200 and calls loadmat('dataBHS.mat'). The +# downloadable notebook could not run. Serving the data from this repo as CSV +# removes the 404 and the scipy dependency at once. The stem is preserved so +# the real rename rides the phase-2 review (QuantEcon/data-lectures#87) with +# its wave-mates. +# +# Because the extension changes, this file is NOT a byte-identical migration +# and cannot claim the usual repoint gate ("a migration moves bytes"). The +# gate it claims instead, measured 2026-08-17 and re-run 2026-08-18: the CSV +# parses back bit-exactly under pandas' correctly-rounded reader, and the +# consuming lecture's histogram of consumption growth has identical counts and +# bin edges under pandas' DEFAULT parser, so the lecture keeps a plain +# read_csv. The lecture's hardcoded moments (mu_c = 0.004952, sigma_c = +# 0.005050) reproduce from the c series to printed precision, and the +# QuantEcon/data-lectures#96 validation reproduced the paper's Tables 1 and 2 +# from these arrays to printed precision. +# +# NOTE the old basename: scripts/audit_annotations.yml keeps its dataBHS.mat +# entry until the repoint lands — the lecture still reads the .mat, that +# annotation is the only record covering the read, and the basenames differ so +# the dual_recorded check does not fire. The flip PR deletes it. + +filename: dataBHS.csv +title: Barillas-Hansen-Sargent "Doubts or variability?" — quarterly US log consumption and real returns, 1948Q1-2006Q4 +description: > + The three quarterly US series behind Barillas, Hansen and Sargent's (JET, + 2009) detection-error calibration, 1948Q1-2006Q4 (236 quarters): log real + per-capita consumption (c) and gross real returns on bonds (rb) and stocks + (rs). The file carries no date column — the sample is stated in the paper + and in the consuming lecture's prose ("1948.I-2006.IV"). `five_preferences` + reads only c, differencing it into the consumption-growth histogram it plots + against the approximating and worst-case densities; rb and rs are carried so + the conversion preserves the authors' file whole. + +# Constructed, not verbatim: these bytes are a value-preserving container +# conversion of the authors' dataBHS.mat into CSV. No filtering, no rescaling, +# no reordering — which is exactly why builders/dataBHS.py reproduces the file +# from the committed input. +class: constructed + +source: + name: Barillas, Hansen and Sargent (2009), "Doubts or variability?" — the authors' dataBHS.mat + url: https://doi.org/10.1016/j.jet.2008.11.014 + # The ARTICLE doi. There is no dataset doi: the Journal of Economic Theory + # article carries no data supplement (checked 2026-08-18). + doi: 10.1016/j.jet.2008.11.014 + version: > + MAT-file header "MATLAB 5.0 MAT-file, Platform: PCWIN, Created on: Fri May + 11 10:34:23 2007" — a working file from the paper's drafting period + (published 2009, sample ends 2006Q4). + series: c, rb, rs — quarterly, 1948Q1-2006Q4, 236 observations each + citation: > + Barillas, Francisco, Lars Peter Hansen, and Thomas J. Sargent. 2009. + "Doubts or variability?" Journal of Economic Theory 144 (6): 2388-2418. + doi:10.1016/j.jet.2008.11.014. Data as distributed with the authors' MATLAB + replication files (dataBHS.mat). + note: > + NOT hosted by the authors or the journal. The underlying series are + standard US aggregates (real per-capita consumption and real asset + returns), assembled by the authors; the paper states the sample but ships + no data appendix. Where the MATLAB file itself could and could not be + located is recorded in sources/README.md, which is the audit trail for the + committed input. The consuming lecture treats the c series as canonical: + its hardcoded mu_c and sigma_c are moments of THIS vintage, frozen in + prose, so any substitute vintage would silently mis-plot the densities + against the histogram — the builder asserts both moments on every run. + +license: + name: null + url: https://doi.org/10.1016/j.jet.2008.11.014 + # Recorded as found, per the record-and-track policy (AGENTS.md, "Licensing + # and attribution"). There is no licence to name and no place that states + # one: the article has no data supplement, and no author-hosted distribution + # of the file exists to carry terms (searched 2026-08-18, sources/README.md). + redistribution: permitted + verified: 2026-08-18 + note: > + An inherited exposure — the .mat has been served publicly from + lecture-python-advanced.myst, and a byte-identical copy from + lecture-tools-techniques, since before this repo existed. `permitted` + rests on what the numbers are rather than on a licence grant: three + standard US aggregate series (consumption, bond and stock returns) of the + kind built from federal statistics, and the authors' contribution is the + assembly, credited in full by the citation above. No author or journal + permission has been sought. Registered for licence review on + QuantEcon/data-lectures#35. + +retrieved: null # inherited from lecture-python-advanced.myst + # with no recorded retrieval date; not + # reconstructed from git history (AGENTS.md). + # It costs nothing here — the builder + # reproduces these bytes from a committed, + # hash-pinned input, so the vintage is pinned + # by content rather than by a date. +maintainer: QuantEcon + +# --------------------------------------------------------------------------- +# Integrity (PLAN Phase 7) +# --------------------------------------------------------------------------- +# Migration check (repoint gate, not a manifest field): NOT byte identity — +# the format changes, deliberately (see header). The gate is value identity +# under the lecture's own read: all 708 values parse back bit-exactly under +# float_precision='round_trip', and the lecture's histogram (30 bins on +# c[1:]-c[:-1]) has identical counts and bin edges under the DEFAULT parser, +# both measured against the .mat via scipy.io.loadmat. The repoint swaps +# loadmat for read_csv, and because the .mat arrays are (236,1) while a CSV +# column parses 1-D, the repointed cell must go through .to_numpy() — a pandas +# Series would silently misalign the c[1:]-c[:-1] difference on index +# alignment. Recorded here because the repoint PR must carry that exact edit. + +integrity: + sha256: 13116a3d90ddc7f8b272b3ca903a136552147b21b8e9b3829472daa3d0d09c63 + upstream: + # `verified` in the sense AGENTS.md defines for a constructed dataset — + # "re-run the builder and compare". builders/dataBHS.py was run against + # the committed sources/dataBHS.mat on the date below and reproduced this + # file exactly, and its validate() stage asserts the round-trip and both + # lecture moments on every run. + status: verified + date: 2026-08-18 + against: sources/dataBHS.mat + note: > + The input hashes to + 28c5f85286718e70b205f6a3fb269ebb49bd635194e2d0d488409b017be5e890 and + holds exactly three (236,1) float64 arrays c, rb, rs (MATLAB 5.0, + PCWIN, created 2007-05-11). What could NOT be established is an authors' + own download to compare against: tomsargent.com's source-code page 404s, + larspeterhansen.org lists no code or data for the paper, the JET article + has no supplement, and a GitHub-wide code search (with a positive + control) finds only QuantEcon's own inherited copies of this blob. That + search is what put the input in sources/ rather than leaving the builder + fetching a copy of ourselves — see sources/README.md. + +# --------------------------------------------------------------------------- +# Shape +# --------------------------------------------------------------------------- +# Measured from the committed bytes 2026-08-18 (pandas 2.3.3), and asserted by +# the builder's validate() stage on every run. + +schema: + format: csv + columns: + - {name: c, dtype: float64, description: "log real per-capita US consumption, quarterly. Observed -4.6556 to -3.4918. The only column the lecture reads: it differences c into quarterly consumption growth (mean 0.004952, std 0.005050 — the lecture's hardcoded mu_c and sigma_c)"} + - {name: rb, dtype: float64, description: "gross real quarterly bond return. Observed 0.9711-1.0217. Carried from the authors' file; read by nothing today"} + - {name: rs, dtype: float64, description: "gross real quarterly stock return. Observed 0.7302-1.2196. Carried from the authors' file; read by nothing today"} + row_count_floor: 236 # exact by design, not a floor with headroom: + # 1948Q1-2006Q4 inclusive, 59 years x 4. The + # file has no date column; validate() asserts + # the count and the growth moments instead. + date_range: {start: 1948, end: 2006} # stated by the paper and the lecture + # prose ("1948.I-2006.IV"), not by any + # column in the file + known_nulls: {} # genuinely none — 708 cells, zero nulls + +# --------------------------------------------------------------------------- +# Consumers — how a correction knows what to rebuild +# --------------------------------------------------------------------------- +# Two consumers of these VALUES, established by the org-wide blob-SHA sweep +# (2026-08-17, QuantEcon/data-lectures#96): the migrating lecture, and a second +# published lecture series that carries a byte-identical .mat and reads its own +# copy. Byte-identical copies also sit in python-lecture-sandpit.myst and +# lecture-mapping, but nothing published reads them (orphans/sandboxes — not +# consumers). lecture-tools-techniques is OUTSIDE the audit's scan set, so it +# is recorded here or nowhere (NEWQDATA zh-cn precedent). The generated +# lecture-python-advanced.notebooks mirror self-heals after a publish tag and +# is deliberately not listed (wave A4 precedent). +consumers: + - repo: QuantEcon/lecture-python-advanced.myst + file: lectures/five_preferences.md + note: > + Reads its own copy — loadmat('dataBHS.mat') on a bare local filename at + lectures/dataBHS.mat, resolved against the notebook working directory — + not this file. The repoint PR in lecture-python-advanced.myst follows + this one (read_csv of this file plus .to_numpy(), dropping the + scipy.io import), and migration.yml records the dataset as `landed` + until it merges. Listed now because a correction to these bytes must + reach the lecture regardless of where it currently reads them from + (QuantEcon/data-lectures#91). + - repo: QuantEcon/lecture-tools-techniques + file: lectures/five_preferences.md + note: > + Reads its own byte-identical dataBHS.mat copy (five_preferences.md:1882), + not this file — same dataset, same lecture, second published series, with + the same downloadable-notebook 404. Mirroring the repoint there is + tracked in QuantEcon/lecture-tools-techniques#11; until it lands, a + correction to these values must be applied to that repo's .mat by hand. + +builder: builders/dataBHS.py +builder_status: committed diff --git a/lectures/fred_data.csv b/lectures/fred_data.csv new file mode 100644 index 0000000..bf41c8d --- /dev/null +++ b/lectures/fred_data.csv @@ -0,0 +1,862 @@ +DATE,GS1,GS5,GS10,DFII5,DFII10,USREC +1953-04-01,2.36,2.62,2.83,,,0 +1953-05-01,2.48,2.87,3.05,,,0 +1953-06-01,2.45,2.94,3.11,,,0 +1953-07-01,2.38,2.75,2.93,,,0 +1953-08-01,2.28,2.8,2.95,,,1 +1953-09-01,2.2,2.71,2.87,,,1 +1953-10-01,1.79,2.43,2.66,,,1 +1953-11-01,1.67,2.42,2.68,,,1 +1953-12-01,1.66,2.32,2.59,,,1 +1954-01-01,1.41,2.17,2.48,,,1 +1954-02-01,1.14,2.04,2.47,,,1 +1954-03-01,1.13,1.93,2.37,,,1 +1954-04-01,0.96,1.87,2.29,,,1 +1954-05-01,0.85,1.92,2.37,,,1 +1954-06-01,0.82,1.92,2.38,,,0 +1954-07-01,0.84,1.85,2.3,,,0 +1954-08-01,0.88,1.9,2.36,,,0 +1954-09-01,1.03,1.96,2.38,,,0 +1954-10-01,1.17,2.02,2.43,,,0 +1954-11-01,1.14,2.09,2.48,,,0 +1954-12-01,1.21,2.16,2.51,,,0 +1955-01-01,1.39,2.32,2.61,,,0 +1955-02-01,1.57,2.38,2.65,,,0 +1955-03-01,1.59,2.48,2.68,,,0 +1955-04-01,1.75,2.55,2.75,,,0 +1955-05-01,1.9,2.56,2.76,,,0 +1955-06-01,1.91,2.59,2.78,,,0 +1955-07-01,2.02,2.72,2.9,,,0 +1955-08-01,2.37,2.86,2.97,,,0 +1955-09-01,2.36,2.85,2.97,,,0 +1955-10-01,2.39,2.76,2.88,,,0 +1955-11-01,2.48,2.81,2.89,,,0 +1955-12-01,2.73,2.93,2.96,,,0 +1956-01-01,2.58,2.84,2.9,,,0 +1956-02-01,2.49,2.74,2.84,,,0 +1956-03-01,2.61,2.93,2.96,,,0 +1956-04-01,2.92,3.2,3.18,,,0 +1956-05-01,2.94,3.08,3.07,,,0 +1956-06-01,2.74,2.97,3.0,,,0 +1956-07-01,2.76,3.12,3.11,,,0 +1956-08-01,3.1,3.41,3.33,,,0 +1956-09-01,3.35,3.47,3.38,,,0 +1956-10-01,3.28,3.4,3.34,,,0 +1956-11-01,3.44,3.56,3.49,,,0 +1956-12-01,3.68,3.7,3.59,,,0 +1957-01-01,3.37,3.47,3.46,,,0 +1957-02-01,3.38,3.39,3.34,,,0 +1957-03-01,3.42,3.46,3.41,,,0 +1957-04-01,3.49,3.53,3.48,,,0 +1957-05-01,3.48,3.64,3.6,,,0 +1957-06-01,3.65,3.83,3.8,,,0 +1957-07-01,3.81,4.0,3.93,,,0 +1957-08-01,4.01,4.0,3.93,,,0 +1957-09-01,4.07,4.03,3.92,,,1 +1957-10-01,4.01,4.08,3.97,,,1 +1957-11-01,3.57,3.72,3.72,,,1 +1957-12-01,3.18,3.08,3.21,,,1 +1958-01-01,2.65,2.88,3.09,,,1 +1958-02-01,1.99,2.78,3.05,,,1 +1958-03-01,1.84,2.64,2.98,,,1 +1958-04-01,1.45,2.46,2.88,,,1 +1958-05-01,1.37,2.41,2.92,,,0 +1958-06-01,1.23,2.46,2.97,,,0 +1958-07-01,1.61,2.77,3.2,,,0 +1958-08-01,2.5,3.29,3.54,,,0 +1958-09-01,3.05,3.69,3.76,,,0 +1958-10-01,3.19,3.78,3.8,,,0 +1958-11-01,3.1,3.7,3.74,,,0 +1958-12-01,3.29,3.82,3.86,,,0 +1959-01-01,3.36,4.01,4.02,,,0 +1959-02-01,3.54,3.96,3.96,,,0 +1959-03-01,3.61,3.99,3.99,,,0 +1959-04-01,3.72,4.12,4.12,,,0 +1959-05-01,3.96,4.35,4.31,,,0 +1959-06-01,4.07,4.5,4.34,,,0 +1959-07-01,4.39,4.58,4.4,,,0 +1959-08-01,4.42,4.57,4.43,,,0 +1959-09-01,5.0,4.9,4.68,,,0 +1959-10-01,4.8,4.72,4.53,,,0 +1959-11-01,4.81,4.75,4.53,,,0 +1959-12-01,5.14,5.01,4.69,,,0 +1960-01-01,5.03,4.92,4.72,,,0 +1960-02-01,4.66,4.69,4.49,,,0 +1960-03-01,4.02,4.31,4.25,,,0 +1960-04-01,4.04,4.29,4.28,,,0 +1960-05-01,4.21,4.49,4.35,,,1 +1960-06-01,3.36,4.12,4.15,,,1 +1960-07-01,3.2,3.79,3.9,,,1 +1960-08-01,2.95,3.62,3.8,,,1 +1960-09-01,3.07,3.61,3.8,,,1 +1960-10-01,3.04,3.76,3.89,,,1 +1960-11-01,3.08,3.81,3.93,,,1 +1960-12-01,2.86,3.67,3.84,,,1 +1961-01-01,2.81,3.67,3.84,,,1 +1961-02-01,2.93,3.66,3.78,,,1 +1961-03-01,2.88,3.6,3.74,,,0 +1961-04-01,2.88,3.57,3.78,,,0 +1961-05-01,2.87,3.47,3.71,,,0 +1961-06-01,3.06,3.81,3.88,,,0 +1961-07-01,2.92,3.84,3.92,,,0 +1961-08-01,3.06,3.96,4.04,,,0 +1961-09-01,3.06,3.9,3.98,,,0 +1961-10-01,3.05,3.8,3.92,,,0 +1961-11-01,3.07,3.82,3.94,,,0 +1961-12-01,3.18,3.91,4.06,,,0 +1962-01-01,3.28,3.94,4.08,,,0 +1962-02-01,3.28,3.89,4.04,,,0 +1962-03-01,3.06,3.68,3.93,,,0 +1962-04-01,2.99,3.6,3.84,,,0 +1962-05-01,3.03,3.66,3.87,,,0 +1962-06-01,3.03,3.64,3.91,,,0 +1962-07-01,3.29,3.8,4.01,,,0 +1962-08-01,3.2,3.71,3.98,,,0 +1962-09-01,3.06,3.7,3.98,,,0 +1962-10-01,2.98,3.64,3.93,,,0 +1962-11-01,3.0,3.6,3.92,,,0 +1962-12-01,3.01,3.56,3.86,,,0 +1963-01-01,3.04,3.58,3.83,,,0 +1963-02-01,3.01,3.66,3.92,,,0 +1963-03-01,3.03,3.68,3.93,,,0 +1963-04-01,3.11,3.74,3.97,,,0 +1963-05-01,3.12,3.72,3.93,,,0 +1963-06-01,3.2,3.81,3.99,,,0 +1963-07-01,3.48,3.89,4.02,,,0 +1963-08-01,3.53,3.89,4.0,,,0 +1963-09-01,3.57,3.96,4.08,,,0 +1963-10-01,3.64,3.97,4.11,,,0 +1963-11-01,3.74,4.01,4.12,,,0 +1963-12-01,3.81,4.04,4.13,,,0 +1964-01-01,3.79,4.07,4.17,,,0 +1964-02-01,3.78,4.03,4.15,,,0 +1964-03-01,3.91,4.14,4.22,,,0 +1964-04-01,3.91,4.15,4.23,,,0 +1964-05-01,3.84,4.05,4.2,,,0 +1964-06-01,3.83,4.02,4.17,,,0 +1964-07-01,3.72,4.03,4.19,,,0 +1964-08-01,3.74,4.05,4.19,,,0 +1964-09-01,3.84,4.08,4.2,,,0 +1964-10-01,3.86,4.07,4.19,,,0 +1964-11-01,3.91,4.04,4.15,,,0 +1964-12-01,4.02,4.09,4.18,,,0 +1965-01-01,3.94,4.1,4.19,,,0 +1965-02-01,4.03,4.15,4.21,,,0 +1965-03-01,4.06,4.15,4.21,,,0 +1965-04-01,4.04,4.15,4.2,,,0 +1965-05-01,4.03,4.15,4.21,,,0 +1965-06-01,3.99,4.15,4.21,,,0 +1965-07-01,3.98,4.15,4.2,,,0 +1965-08-01,4.07,4.2,4.25,,,0 +1965-09-01,4.2,4.25,4.29,,,0 +1965-10-01,4.3,4.34,4.35,,,0 +1965-11-01,4.37,4.46,4.45,,,0 +1965-12-01,4.72,4.72,4.62,,,0 +1966-01-01,4.88,4.86,4.61,,,0 +1966-02-01,4.94,4.98,4.83,,,0 +1966-03-01,4.97,4.92,4.87,,,0 +1966-04-01,4.9,4.83,4.75,,,0 +1966-05-01,4.93,4.89,4.78,,,0 +1966-06-01,4.97,4.97,4.81,,,0 +1966-07-01,5.17,5.17,5.02,,,0 +1966-08-01,5.54,5.5,5.22,,,0 +1966-09-01,5.82,5.5,5.18,,,0 +1966-10-01,5.58,5.27,5.01,,,0 +1966-11-01,5.54,5.36,5.16,,,0 +1966-12-01,5.2,5.0,4.84,,,0 +1967-01-01,4.75,4.7,4.58,,,0 +1967-02-01,4.71,4.74,4.63,,,0 +1967-03-01,4.35,4.54,4.54,,,0 +1967-04-01,4.11,4.51,4.59,,,0 +1967-05-01,4.15,4.75,4.85,,,0 +1967-06-01,4.48,5.01,5.02,,,0 +1967-07-01,5.01,5.23,5.16,,,0 +1967-08-01,5.13,5.31,5.28,,,0 +1967-09-01,5.24,5.4,5.3,,,0 +1967-10-01,5.37,5.57,5.48,,,0 +1967-11-01,5.61,5.78,5.75,,,0 +1967-12-01,5.71,5.75,5.7,,,0 +1968-01-01,5.43,5.54,5.53,,,0 +1968-02-01,5.41,5.59,5.56,,,0 +1968-03-01,5.58,5.76,5.74,,,0 +1968-04-01,5.71,5.69,5.64,,,0 +1968-05-01,6.14,6.04,5.87,,,0 +1968-06-01,5.98,5.85,5.72,,,0 +1968-07-01,5.65,5.6,5.5,,,0 +1968-08-01,5.43,5.5,5.42,,,0 +1968-09-01,5.45,5.48,5.46,,,0 +1968-10-01,5.57,5.55,5.58,,,0 +1968-11-01,5.75,5.66,5.7,,,0 +1968-12-01,6.19,6.12,6.03,,,0 +1969-01-01,6.34,6.25,6.04,,,0 +1969-02-01,6.41,6.34,6.19,,,0 +1969-03-01,6.34,6.41,6.3,,,0 +1969-04-01,6.26,6.3,6.17,,,0 +1969-05-01,6.42,6.54,6.32,,,0 +1969-06-01,7.04,6.75,6.57,,,0 +1969-07-01,7.6,7.01,6.72,,,0 +1969-08-01,7.54,7.03,6.69,,,0 +1969-09-01,7.82,7.57,7.16,,,0 +1969-10-01,7.64,7.51,7.1,,,0 +1969-11-01,7.89,7.53,7.14,,,0 +1969-12-01,8.17,7.96,7.65,,,0 +1970-01-01,8.1,8.17,7.79,,,1 +1970-02-01,7.59,7.82,7.24,,,1 +1970-03-01,6.97,7.21,7.07,,,1 +1970-04-01,7.06,7.5,7.39,,,1 +1970-05-01,7.75,7.97,7.91,,,1 +1970-06-01,7.55,7.85,7.84,,,1 +1970-07-01,7.1,7.59,7.46,,,1 +1970-08-01,6.98,7.57,7.53,,,1 +1970-09-01,6.73,7.29,7.39,,,1 +1970-10-01,6.43,7.12,7.33,,,1 +1970-11-01,5.51,6.47,6.84,,,1 +1970-12-01,5.0,5.95,6.39,,,0 +1971-01-01,4.57,5.89,6.24,,,0 +1971-02-01,3.89,5.56,6.11,,,0 +1971-03-01,3.69,5.0,5.7,,,0 +1971-04-01,4.3,5.65,5.83,,,0 +1971-05-01,5.04,6.28,6.39,,,0 +1971-06-01,5.64,6.53,6.52,,,0 +1971-07-01,6.04,6.85,6.73,,,0 +1971-08-01,5.8,6.55,6.58,,,0 +1971-09-01,5.41,6.14,6.14,,,0 +1971-10-01,4.91,5.93,5.93,,,0 +1971-11-01,4.67,5.78,5.81,,,0 +1971-12-01,4.6,5.69,5.93,,,0 +1972-01-01,4.28,5.59,5.95,,,0 +1972-02-01,4.27,5.69,6.08,,,0 +1972-03-01,4.67,5.87,6.07,,,0 +1972-04-01,4.96,6.17,6.19,,,0 +1972-05-01,4.64,5.85,6.13,,,0 +1972-06-01,4.93,5.91,6.11,,,0 +1972-07-01,4.96,5.97,6.11,,,0 +1972-08-01,4.98,6.02,6.21,,,0 +1972-09-01,5.52,6.25,6.55,,,0 +1972-10-01,5.52,6.18,6.48,,,0 +1972-11-01,5.27,6.12,6.28,,,0 +1972-12-01,5.52,6.16,6.36,,,0 +1973-01-01,5.89,6.34,6.46,,,0 +1973-02-01,6.19,6.6,6.64,,,0 +1973-03-01,6.85,6.8,6.71,,,0 +1973-04-01,6.85,6.67,6.67,,,0 +1973-05-01,6.89,6.8,6.85,,,0 +1973-06-01,7.31,6.69,6.9,,,0 +1973-07-01,8.39,7.33,7.13,,,0 +1973-08-01,8.82,7.63,7.4,,,0 +1973-09-01,8.31,7.05,7.09,,,0 +1973-10-01,7.4,6.77,6.79,,,0 +1973-11-01,7.57,6.92,6.73,,,0 +1973-12-01,7.27,6.8,6.74,,,1 +1974-01-01,7.42,6.95,6.99,,,1 +1974-02-01,6.88,6.82,6.96,,,1 +1974-03-01,7.76,7.31,7.21,,,1 +1974-04-01,8.62,7.92,7.51,,,1 +1974-05-01,8.78,8.18,7.58,,,1 +1974-06-01,8.67,8.1,7.54,,,1 +1974-07-01,8.8,8.38,7.81,,,1 +1974-08-01,9.36,8.63,8.04,,,1 +1974-09-01,8.87,8.37,8.04,,,1 +1974-10-01,8.05,7.97,7.9,,,1 +1974-11-01,7.66,7.68,7.68,,,1 +1974-12-01,7.31,7.31,7.43,,,1 +1975-01-01,6.83,7.41,7.5,,,1 +1975-02-01,5.98,7.11,7.39,,,1 +1975-03-01,6.11,7.3,7.73,,,1 +1975-04-01,6.9,7.99,8.23,,,0 +1975-05-01,6.39,7.72,8.06,,,0 +1975-06-01,6.29,7.51,7.86,,,0 +1975-07-01,7.11,7.92,8.06,,,0 +1975-08-01,7.7,8.33,8.4,,,0 +1975-09-01,7.75,8.37,8.43,,,0 +1975-10-01,6.95,7.97,8.14,,,0 +1975-11-01,6.49,7.8,8.05,,,0 +1975-12-01,6.6,7.76,8.0,,,0 +1976-01-01,5.81,7.46,7.74,,,0 +1976-02-01,5.91,7.45,7.79,,,0 +1976-03-01,6.21,7.49,7.73,,,0 +1976-04-01,5.92,7.25,7.56,,,0 +1976-05-01,6.4,7.59,7.9,,,0 +1976-06-01,6.52,7.61,7.86,,,0 +1976-07-01,6.2,7.49,7.83,,,0 +1976-08-01,6.0,7.31,7.77,,,0 +1976-09-01,5.84,7.13,7.59,,,0 +1976-10-01,5.5,6.75,7.41,,,0 +1976-11-01,5.29,6.52,7.29,,,0 +1976-12-01,4.89,6.1,6.87,,,0 +1977-01-01,5.29,6.58,7.21,,,0 +1977-02-01,5.47,6.83,7.39,,,0 +1977-03-01,5.5,6.93,7.46,,,0 +1977-04-01,5.44,6.79,7.37,,,0 +1977-05-01,5.84,6.94,7.46,,,0 +1977-06-01,5.8,6.76,7.28,,,0 +1977-07-01,5.94,6.84,7.33,,,0 +1977-08-01,6.37,7.03,7.4,,,0 +1977-09-01,6.53,7.04,7.34,,,0 +1977-10-01,6.97,7.32,7.52,,,0 +1977-11-01,6.95,7.34,7.58,,,0 +1977-12-01,6.96,7.48,7.69,,,0 +1978-01-01,7.28,7.77,7.96,,,0 +1978-02-01,7.34,7.83,8.03,,,0 +1978-03-01,7.31,7.86,8.04,,,0 +1978-04-01,7.45,7.98,8.15,,,0 +1978-05-01,7.82,8.18,8.35,,,0 +1978-06-01,8.09,8.36,8.46,,,0 +1978-07-01,8.39,8.54,8.64,,,0 +1978-08-01,8.31,8.33,8.41,,,0 +1978-09-01,8.64,8.43,8.42,,,0 +1978-10-01,9.14,8.61,8.64,,,0 +1978-11-01,10.01,8.84,8.81,,,0 +1978-12-01,10.3,9.08,9.01,,,0 +1979-01-01,10.41,9.2,9.1,,,0 +1979-02-01,10.24,9.13,9.1,,,0 +1979-03-01,10.25,9.2,9.12,,,0 +1979-04-01,10.12,9.25,9.18,,,0 +1979-05-01,10.12,9.24,9.25,,,0 +1979-06-01,9.57,8.85,8.91,,,0 +1979-07-01,9.64,8.9,8.95,,,0 +1979-08-01,9.98,9.06,9.03,,,0 +1979-09-01,10.84,9.41,9.33,,,0 +1979-10-01,12.44,10.63,10.3,,,0 +1979-11-01,12.39,10.93,10.65,,,0 +1979-12-01,11.98,10.42,10.39,,,0 +1980-01-01,12.06,10.74,10.8,,,0 +1980-02-01,13.92,12.6,12.41,,,1 +1980-03-01,15.82,13.47,12.75,,,1 +1980-04-01,13.3,11.84,11.47,,,1 +1980-05-01,9.39,9.95,10.18,,,1 +1980-06-01,8.16,9.21,9.78,,,1 +1980-07-01,8.65,9.53,10.25,,,1 +1980-08-01,10.24,10.84,11.1,,,0 +1980-09-01,11.52,11.62,11.51,,,0 +1980-10-01,12.49,11.86,11.75,,,0 +1980-11-01,14.15,12.83,12.68,,,0 +1980-12-01,14.88,13.25,12.84,,,0 +1981-01-01,14.08,12.77,12.57,,,0 +1981-02-01,14.57,13.41,13.19,,,0 +1981-03-01,13.71,13.41,13.12,,,0 +1981-04-01,14.32,13.99,13.68,,,0 +1981-05-01,16.2,14.63,14.1,,,0 +1981-06-01,14.86,13.95,13.47,,,0 +1981-07-01,15.72,14.79,14.28,,,0 +1981-08-01,16.72,15.56,14.94,,,1 +1981-09-01,16.52,15.93,15.32,,,1 +1981-10-01,15.38,15.41,15.15,,,1 +1981-11-01,12.41,13.38,13.39,,,1 +1981-12-01,12.85,13.6,13.72,,,1 +1982-01-01,14.32,14.65,14.59,,,1 +1982-02-01,14.73,14.54,14.43,,,1 +1982-03-01,13.95,13.98,13.86,,,1 +1982-04-01,13.98,14.0,13.87,,,1 +1982-05-01,13.34,13.75,13.62,,,1 +1982-06-01,14.07,14.43,14.3,,,1 +1982-07-01,13.24,14.07,13.95,,,1 +1982-08-01,11.43,13.0,13.06,,,1 +1982-09-01,10.85,12.25,12.34,,,1 +1982-10-01,9.32,10.8,10.91,,,1 +1982-11-01,9.16,10.38,10.55,,,1 +1982-12-01,8.91,10.22,10.54,,,0 +1983-01-01,8.62,10.03,10.46,,,0 +1983-02-01,8.92,10.26,10.72,,,0 +1983-03-01,9.04,10.08,10.51,,,0 +1983-04-01,8.98,10.02,10.4,,,0 +1983-05-01,8.9,10.03,10.38,,,0 +1983-06-01,9.66,10.63,10.85,,,0 +1983-07-01,10.2,11.21,11.38,,,0 +1983-08-01,10.53,11.63,11.85,,,0 +1983-09-01,10.16,11.43,11.65,,,0 +1983-10-01,9.81,11.28,11.54,,,0 +1983-11-01,9.94,11.41,11.69,,,0 +1983-12-01,10.11,11.54,11.83,,,0 +1984-01-01,9.9,11.37,11.67,,,0 +1984-02-01,10.04,11.54,11.84,,,0 +1984-03-01,10.59,12.02,12.32,,,0 +1984-04-01,10.9,12.37,12.63,,,0 +1984-05-01,11.66,13.17,13.41,,,0 +1984-06-01,12.08,13.48,13.56,,,0 +1984-07-01,12.03,13.27,13.36,,,0 +1984-08-01,11.82,12.68,12.72,,,0 +1984-09-01,11.58,12.53,12.52,,,0 +1984-10-01,10.9,12.06,12.16,,,0 +1984-11-01,9.82,11.33,11.57,,,0 +1984-12-01,9.33,11.07,11.5,,,0 +1985-01-01,9.02,10.93,11.38,,,0 +1985-02-01,9.29,11.13,11.51,,,0 +1985-03-01,9.86,11.52,11.86,,,0 +1985-04-01,9.14,11.01,11.43,,,0 +1985-05-01,8.46,10.34,10.85,,,0 +1985-06-01,7.8,9.6,10.16,,,0 +1985-07-01,7.86,9.7,10.31,,,0 +1985-08-01,8.05,9.81,10.33,,,0 +1985-09-01,8.07,9.81,10.37,,,0 +1985-10-01,8.01,9.69,10.24,,,0 +1985-11-01,7.88,9.28,9.78,,,0 +1985-12-01,7.67,8.73,9.26,,,0 +1986-01-01,7.73,8.68,9.19,,,0 +1986-02-01,7.61,8.34,8.7,,,0 +1986-03-01,7.03,7.46,7.78,,,0 +1986-04-01,6.44,7.05,7.3,,,0 +1986-05-01,6.65,7.52,7.71,,,0 +1986-06-01,6.73,7.64,7.8,,,0 +1986-07-01,6.27,7.06,7.3,,,0 +1986-08-01,5.93,6.8,7.17,,,0 +1986-09-01,5.77,6.92,7.45,,,0 +1986-10-01,5.72,6.83,7.43,,,0 +1986-11-01,5.8,6.76,7.25,,,0 +1986-12-01,5.87,6.67,7.11,,,0 +1987-01-01,5.78,6.64,7.08,,,0 +1987-02-01,5.96,6.79,7.25,,,0 +1987-03-01,6.03,6.79,7.25,,,0 +1987-04-01,6.5,7.57,8.02,,,0 +1987-05-01,7.0,8.26,8.61,,,0 +1987-06-01,6.8,8.02,8.4,,,0 +1987-07-01,6.68,8.01,8.45,,,0 +1987-08-01,7.03,8.32,8.76,,,0 +1987-09-01,7.67,8.94,9.42,,,0 +1987-10-01,7.59,9.08,9.52,,,0 +1987-11-01,6.96,8.35,8.86,,,0 +1987-12-01,7.17,8.45,8.99,,,0 +1988-01-01,6.99,8.18,8.67,,,0 +1988-02-01,6.64,7.71,8.21,,,0 +1988-03-01,6.71,7.83,8.37,,,0 +1988-04-01,7.01,8.19,8.72,,,0 +1988-05-01,7.4,8.58,9.09,,,0 +1988-06-01,7.49,8.49,8.92,,,0 +1988-07-01,7.75,8.66,9.06,,,0 +1988-08-01,8.17,8.94,9.26,,,0 +1988-09-01,8.09,8.69,8.98,,,0 +1988-10-01,8.11,8.51,8.8,,,0 +1988-11-01,8.48,8.79,8.96,,,0 +1988-12-01,8.99,9.09,9.11,,,0 +1989-01-01,9.05,9.15,9.09,,,0 +1989-02-01,9.25,9.27,9.17,,,0 +1989-03-01,9.57,9.51,9.36,,,0 +1989-04-01,9.36,9.3,9.18,,,0 +1989-05-01,8.98,8.91,8.86,,,0 +1989-06-01,8.44,8.29,8.28,,,0 +1989-07-01,7.89,7.83,8.02,,,0 +1989-08-01,8.18,8.09,8.11,,,0 +1989-09-01,8.22,8.17,8.19,,,0 +1989-10-01,7.99,7.97,8.01,,,0 +1989-11-01,7.77,7.81,7.87,,,0 +1989-12-01,7.72,7.75,7.84,,,0 +1990-01-01,7.92,8.12,8.21,,,0 +1990-02-01,8.11,8.42,8.47,,,0 +1990-03-01,8.35,8.6,8.59,,,0 +1990-04-01,8.4,8.77,8.79,,,0 +1990-05-01,8.32,8.74,8.76,,,0 +1990-06-01,8.1,8.43,8.48,,,0 +1990-07-01,7.94,8.33,8.47,,,0 +1990-08-01,7.78,8.44,8.75,,,1 +1990-09-01,7.76,8.51,8.89,,,1 +1990-10-01,7.55,8.33,8.72,,,1 +1990-11-01,7.31,8.02,8.39,,,1 +1990-12-01,7.05,7.73,8.08,,,1 +1991-01-01,6.64,7.7,8.09,,,1 +1991-02-01,6.27,7.47,7.85,,,1 +1991-03-01,6.4,7.77,8.11,,,1 +1991-04-01,6.24,7.7,8.04,,,0 +1991-05-01,6.13,7.7,8.07,,,0 +1991-06-01,6.36,7.94,8.28,,,0 +1991-07-01,6.31,7.91,8.27,,,0 +1991-08-01,5.78,7.43,7.9,,,0 +1991-09-01,5.57,7.14,7.65,,,0 +1991-10-01,5.33,6.87,7.53,,,0 +1991-11-01,4.89,6.62,7.42,,,0 +1991-12-01,4.38,6.19,7.09,,,0 +1992-01-01,4.15,6.24,7.03,,,0 +1992-02-01,4.29,6.58,7.34,,,0 +1992-03-01,4.63,6.95,7.54,,,0 +1992-04-01,4.3,6.78,7.48,,,0 +1992-05-01,4.19,6.69,7.39,,,0 +1992-06-01,4.17,6.48,7.26,,,0 +1992-07-01,3.6,5.84,6.84,,,0 +1992-08-01,3.47,5.6,6.59,,,0 +1992-09-01,3.18,5.38,6.42,,,0 +1992-10-01,3.3,5.6,6.59,,,0 +1992-11-01,3.68,6.04,6.87,,,0 +1992-12-01,3.71,6.08,6.77,,,0 +1993-01-01,3.5,5.83,6.6,,,0 +1993-02-01,3.39,5.43,6.26,,,0 +1993-03-01,3.33,5.19,5.98,,,0 +1993-04-01,3.24,5.13,5.97,,,0 +1993-05-01,3.36,5.2,6.04,,,0 +1993-06-01,3.54,5.22,5.96,,,0 +1993-07-01,3.47,5.09,5.81,,,0 +1993-08-01,3.44,5.03,5.68,,,0 +1993-09-01,3.36,4.73,5.36,,,0 +1993-10-01,3.39,4.71,5.33,,,0 +1993-11-01,3.58,5.06,5.72,,,0 +1993-12-01,3.61,5.15,5.77,,,0 +1994-01-01,3.54,5.09,5.75,,,0 +1994-02-01,3.87,5.4,5.97,,,0 +1994-03-01,4.32,5.94,6.48,,,0 +1994-04-01,4.82,6.52,6.97,,,0 +1994-05-01,5.31,6.78,7.18,,,0 +1994-06-01,5.27,6.7,7.1,,,0 +1994-07-01,5.48,6.91,7.3,,,0 +1994-08-01,5.56,6.88,7.24,,,0 +1994-09-01,5.76,7.08,7.46,,,0 +1994-10-01,6.11,7.4,7.74,,,0 +1994-11-01,6.54,7.72,7.96,,,0 +1994-12-01,7.14,7.78,7.81,,,0 +1995-01-01,7.05,7.76,7.78,,,0 +1995-02-01,6.7,7.37,7.47,,,0 +1995-03-01,6.43,7.05,7.2,,,0 +1995-04-01,6.27,6.86,7.06,,,0 +1995-05-01,6.0,6.41,6.63,,,0 +1995-06-01,5.64,5.93,6.17,,,0 +1995-07-01,5.59,6.01,6.28,,,0 +1995-08-01,5.75,6.24,6.49,,,0 +1995-09-01,5.62,6.0,6.2,,,0 +1995-10-01,5.59,5.86,6.04,,,0 +1995-11-01,5.43,5.69,5.93,,,0 +1995-12-01,5.31,5.51,5.71,,,0 +1996-01-01,5.09,5.36,5.65,,,0 +1996-02-01,4.94,5.38,5.81,,,0 +1996-03-01,5.34,5.97,6.27,,,0 +1996-04-01,5.54,6.3,6.51,,,0 +1996-05-01,5.64,6.48,6.74,,,0 +1996-06-01,5.81,6.69,6.91,,,0 +1996-07-01,5.85,6.64,6.87,,,0 +1996-08-01,5.67,6.39,6.64,,,0 +1996-09-01,5.83,6.6,6.83,,,0 +1996-10-01,5.55,6.27,6.53,,,0 +1996-11-01,5.42,5.97,6.2,,,0 +1996-12-01,5.47,6.07,6.3,,,0 +1997-01-01,5.61,6.33,6.58,,,0 +1997-02-01,5.53,6.2,6.42,,,0 +1997-03-01,5.8,6.54,6.69,,,0 +1997-04-01,5.99,6.76,6.89,,,0 +1997-05-01,5.87,6.57,6.71,,,0 +1997-06-01,5.69,6.38,6.49,,,0 +1997-07-01,5.54,6.12,6.22,,,0 +1997-08-01,5.56,6.16,6.3,,,0 +1997-09-01,5.52,6.11,6.21,,,0 +1997-10-01,5.46,5.93,6.03,,,0 +1997-11-01,5.46,5.8,5.88,,,0 +1997-12-01,5.53,5.77,5.81,,,0 +1998-01-01,5.24,5.42,5.54,,,0 +1998-02-01,5.31,5.49,5.57,,,0 +1998-03-01,5.39,5.61,5.65,,,0 +1998-04-01,5.38,5.61,5.64,,,0 +1998-05-01,5.44,5.63,5.65,,,0 +1998-06-01,5.41,5.52,5.5,,,0 +1998-07-01,5.36,5.46,5.46,,,0 +1998-08-01,5.21,5.27,5.34,,,0 +1998-09-01,4.71,4.62,4.81,,,0 +1998-10-01,4.12,4.18,4.53,,,0 +1998-11-01,4.53,4.54,4.83,,,0 +1998-12-01,4.52,4.45,4.65,,,0 +1999-01-01,4.51,4.6,4.72,,,0 +1999-02-01,4.7,4.91,5.0,,,0 +1999-03-01,4.78,5.14,5.23,,,0 +1999-04-01,4.69,5.08,5.18,,,0 +1999-05-01,4.85,5.44,5.54,,,0 +1999-06-01,5.1,5.81,5.9,,,0 +1999-07-01,5.03,5.68,5.79,,,0 +1999-08-01,5.2,5.84,5.94,,,0 +1999-09-01,5.25,5.8,5.92,,,0 +1999-10-01,5.43,6.03,6.11,,,0 +1999-11-01,5.55,5.97,6.03,,,0 +1999-12-01,5.84,6.19,6.28,,,0 +2000-01-01,6.12,6.58,6.66,,,0 +2000-02-01,6.22,6.68,6.52,,,0 +2000-03-01,6.22,6.5,6.26,,,0 +2000-04-01,6.15,6.26,5.99,,,0 +2000-05-01,6.33,6.69,6.44,,,0 +2000-06-01,6.17,6.3,6.1,,,0 +2000-07-01,6.08,6.18,6.05,,,0 +2000-08-01,6.18,6.06,5.83,,,0 +2000-09-01,6.13,5.93,5.8,,,0 +2000-10-01,6.01,5.78,5.74,,,0 +2000-11-01,6.09,5.7,5.72,,,0 +2000-12-01,5.6,5.17,5.24,,,0 +2001-01-01,4.81,4.86,5.16,,,0 +2001-02-01,4.68,4.89,5.1,,,0 +2001-03-01,4.3,4.64,4.89,,,0 +2001-04-01,3.98,4.76,5.14,,,1 +2001-05-01,3.78,4.93,5.39,,,1 +2001-06-01,3.58,4.81,5.28,,,1 +2001-07-01,3.62,4.76,5.24,,,1 +2001-08-01,3.47,4.57,4.97,,,1 +2001-09-01,2.82,4.12,4.73,,,1 +2001-10-01,2.33,3.91,4.57,,,1 +2001-11-01,2.18,3.97,4.65,,,1 +2001-12-01,2.22,4.39,5.09,,,0 +2002-01-01,2.16,4.34,5.04,,,0 +2002-02-01,2.23,4.3,4.91,,,0 +2002-03-01,2.57,4.74,5.28,,,0 +2002-04-01,2.48,4.65,5.21,,,0 +2002-05-01,2.35,4.49,5.16,,,0 +2002-06-01,2.2,4.19,4.93,,,0 +2002-07-01,1.96,3.81,4.65,,,0 +2002-08-01,1.76,3.29,4.26,,,0 +2002-09-01,1.72,2.94,3.87,,,0 +2002-10-01,1.65,2.95,3.94,,,0 +2002-11-01,1.49,3.05,4.05,,,0 +2002-12-01,1.45,3.03,4.03,,,0 +2003-01-01,1.36,3.05,4.05,1.65,2.29,0 +2003-02-01,1.3,2.9,3.9,1.24,1.99,0 +2003-03-01,1.24,2.78,3.81,1.09,1.94,0 +2003-04-01,1.27,2.93,3.96,1.36,2.18,0 +2003-05-01,1.18,2.52,3.57,1.18,1.91,0 +2003-06-01,1.01,2.27,3.33,0.91,1.72,0 +2003-07-01,1.12,2.87,3.98,1.3,2.11,0 +2003-08-01,1.31,3.37,4.45,1.48,2.32,0 +2003-09-01,1.24,3.18,4.27,1.29,2.19,0 +2003-10-01,1.25,3.19,4.29,1.21,2.08,0 +2003-11-01,1.34,3.29,4.3,1.27,1.96,0 +2003-12-01,1.31,3.27,4.27,1.23,1.98,0 +2004-01-01,1.24,3.12,4.15,1.09,1.89,0 +2004-02-01,1.24,3.07,4.08,0.86,1.76,0 +2004-03-01,1.19,2.79,3.83,0.52,1.47,0 +2004-04-01,1.43,3.39,4.35,1.02,1.9,0 +2004-05-01,1.78,3.85,4.72,1.34,2.09,0 +2004-06-01,2.12,3.93,4.73,1.41,2.15,0 +2004-07-01,2.1,3.69,4.5,1.29,2.02,0 +2004-08-01,2.02,3.47,4.28,1.12,1.86,0 +2004-09-01,2.12,3.36,4.13,1.1,1.8,0 +2004-10-01,2.23,3.35,4.1,0.97,1.73,0 +2004-11-01,2.5,3.53,4.19,0.9,1.68,0 +2004-12-01,2.67,3.6,4.23,0.92,1.67,0 +2005-01-01,2.86,3.71,4.22,1.13,1.72,0 +2005-02-01,3.03,3.77,4.17,1.08,1.63,0 +2005-03-01,3.3,4.17,4.5,1.29,1.79,0 +2005-04-01,3.32,4.0,4.34,1.23,1.71,0 +2005-05-01,3.33,3.85,4.14,1.28,1.65,0 +2005-06-01,3.36,3.77,4.0,1.39,1.67,0 +2005-07-01,3.64,3.98,4.18,1.67,1.88,0 +2005-08-01,3.87,4.12,4.26,1.71,1.89,0 +2005-09-01,3.85,4.01,4.2,1.4,1.7,0 +2005-10-01,4.18,4.33,4.46,1.7,1.94,0 +2005-11-01,4.33,4.45,4.54,1.97,2.06,0 +2005-12-01,4.35,4.39,4.47,2.09,2.12,0 +2006-01-01,4.45,4.35,4.42,1.93,2.01,0 +2006-02-01,4.68,4.57,4.57,1.98,2.05,0 +2006-03-01,4.77,4.72,4.72,2.09,2.2,0 +2006-04-01,4.9,4.9,4.99,2.26,2.41,0 +2006-05-01,5.0,5.0,5.11,2.3,2.45,0 +2006-06-01,5.16,5.07,5.11,2.45,2.53,0 +2006-07-01,5.22,5.04,5.09,2.46,2.51,0 +2006-08-01,5.08,4.82,4.88,2.27,2.29,0 +2006-09-01,4.97,4.67,4.72,2.38,2.32,0 +2006-10-01,5.01,4.69,4.73,2.51,2.41,0 +2006-11-01,5.01,4.58,4.6,2.41,2.29,0 +2006-12-01,4.94,4.53,4.56,2.28,2.25,0 +2007-01-01,5.06,4.75,4.76,2.47,2.44,0 +2007-02-01,5.05,4.71,4.72,2.34,2.36,0 +2007-03-01,4.92,4.48,4.56,2.04,2.18,0 +2007-04-01,4.93,4.59,4.69,2.12,2.26,0 +2007-05-01,4.91,4.67,4.75,2.29,2.37,0 +2007-06-01,4.96,5.03,5.1,2.65,2.69,0 +2007-07-01,4.96,4.88,5.0,2.6,2.64,0 +2007-08-01,4.47,4.43,4.67,2.39,2.44,0 +2007-09-01,4.14,4.2,4.52,2.14,2.26,0 +2007-10-01,4.1,4.2,4.53,2.01,2.2,0 +2007-11-01,3.5,3.67,4.15,1.35,1.77,0 +2007-12-01,3.26,3.49,4.1,1.27,1.79,0 +2008-01-01,2.71,2.98,3.74,0.86,1.47,1 +2008-02-01,2.05,2.78,3.74,0.65,1.41,1 +2008-03-01,1.54,2.48,3.51,0.23,1.09,1 +2008-04-01,1.74,2.84,3.68,0.62,1.36,1 +2008-05-01,2.06,3.15,3.88,0.79,1.46,1 +2008-06-01,2.42,3.49,4.1,0.97,1.63,1 +2008-07-01,2.28,3.3,4.01,0.84,1.57,1 +2008-08-01,2.18,3.14,3.89,1.15,1.68,1 +2008-09-01,1.91,2.88,3.69,1.55,1.85,1 +2008-10-01,1.42,2.73,3.81,2.75,2.75,1 +2008-11-01,1.07,2.29,3.53,3.69,2.89,1 +2008-12-01,0.49,1.52,2.42,1.76,2.17,1 +2009-01-01,0.44,1.6,2.52,1.59,1.91,1 +2009-02-01,0.62,1.87,2.87,1.29,1.75,1 +2009-03-01,0.64,1.82,2.82,1.23,1.71,1 +2009-04-01,0.55,1.86,2.93,1.11,1.57,1 +2009-05-01,0.5,2.13,3.29,1.07,1.72,1 +2009-06-01,0.51,2.71,3.72,1.18,1.86,1 +2009-07-01,0.48,2.46,3.56,1.18,1.82,0 +2009-08-01,0.46,2.57,3.59,1.29,1.77,0 +2009-09-01,0.4,2.37,3.4,1.03,1.64,0 +2009-10-01,0.37,2.33,3.39,0.83,1.48,0 +2009-11-01,0.31,2.23,3.4,0.48,1.28,0 +2009-12-01,0.37,2.34,3.59,0.43,1.36,0 +2010-01-01,0.35,2.48,3.73,0.42,1.37,0 +2010-02-01,0.35,2.36,3.69,0.42,1.42,0 +2010-03-01,0.4,2.43,3.73,0.56,1.51,0 +2010-04-01,0.45,2.58,3.85,0.62,1.5,0 +2010-05-01,0.37,2.18,3.42,0.41,1.31,0 +2010-06-01,0.32,2.0,3.2,0.34,1.26,0 +2010-07-01,0.29,1.76,3.01,0.34,1.24,0 +2010-08-01,0.26,1.47,2.7,0.13,1.02,0 +2010-09-01,0.26,1.41,2.65,0.13,0.91,0 +2010-10-01,0.23,1.18,2.54,-0.32,0.53,0 +2010-11-01,0.25,1.35,2.76,-0.21,0.67,0 +2010-12-01,0.29,1.93,3.29,0.21,1.04,0 +2011-01-01,0.27,1.99,3.39,0.06,1.06,0 +2011-02-01,0.29,2.26,3.58,0.25,1.24,0 +2011-03-01,0.26,2.11,3.41,-0.09,0.96,0 +2011-04-01,0.25,2.17,3.46,-0.14,0.86,0 +2011-05-01,0.19,1.84,3.17,-0.34,0.78,0 +2011-06-01,0.18,1.58,3.0,-0.38,0.76,0 +2011-07-01,0.19,1.54,3.0,-0.49,0.62,0 +2011-08-01,0.11,1.02,2.3,-0.75,0.14,0 +2011-09-01,0.1,0.9,1.98,-0.72,0.08,0 +2011-10-01,0.11,1.06,2.15,-0.63,0.19,0 +2011-11-01,0.11,0.91,2.01,-0.85,0.0,0 +2011-12-01,0.12,0.89,1.98,-0.78,-0.03,0 +2012-01-01,0.12,0.84,1.97,-0.92,-0.11,0 +2012-02-01,0.16,0.83,1.97,-1.11,-0.25,0 +2012-03-01,0.19,1.02,2.17,-1.03,-0.14,0 +2012-04-01,0.18,0.89,2.05,-1.06,-0.21,0 +2012-05-01,0.19,0.76,1.8,-1.12,-0.34,0 +2012-06-01,0.19,0.71,1.62,-1.05,-0.5,0 +2012-07-01,0.19,0.62,1.53,-1.15,-0.6,0 +2012-08-01,0.18,0.71,1.68,-1.19,-0.59,0 +2012-09-01,0.18,0.67,1.72,-1.47,-0.71,0 +2012-10-01,0.18,0.71,1.75,-1.47,-0.75,0 +2012-11-01,0.18,0.67,1.65,-1.38,-0.77,0 +2012-12-01,0.16,0.7,1.72,-1.4,-0.76,0 +2013-01-01,0.15,0.81,1.91,-1.39,-0.61,0 +2013-02-01,0.16,0.85,1.98,-1.39,-0.57,0 +2013-03-01,0.15,0.82,1.96,-1.43,-0.59,0 +2013-04-01,0.12,0.71,1.76,-1.38,-0.65,0 +2013-05-01,0.12,0.84,1.93,-1.14,-0.36,0 +2013-06-01,0.14,1.2,2.3,-0.59,0.25,0 +2013-07-01,0.12,1.4,2.58,-0.45,0.46,0 +2013-08-01,0.13,1.52,2.74,-0.33,0.55,0 +2013-09-01,0.12,1.6,2.81,-0.17,0.66,0 +2013-10-01,0.12,1.37,2.62,-0.41,0.43,0 +2013-11-01,0.12,1.37,2.72,-0.38,0.55,0 +2013-12-01,0.13,1.58,2.9,-0.09,0.74,0 +2014-01-01,0.12,1.65,2.86,-0.09,0.63,0 +2014-02-01,0.12,1.52,2.71,-0.26,0.55,0 +2014-03-01,0.13,1.64,2.72,-0.14,0.56,0 +2014-04-01,0.11,1.7,2.71,-0.11,0.54,0 +2014-05-01,0.1,1.59,2.56,-0.34,0.37,0 +2014-06-01,0.1,1.68,2.6,-0.29,0.37,0 +2014-07-01,0.11,1.7,2.54,-0.27,0.28,0 +2014-08-01,0.11,1.63,2.42,-0.21,0.22,0 +2014-09-01,0.11,1.77,2.53,0.1,0.46,0 +2014-10-01,0.1,1.55,2.3,0.06,0.38,0 +2014-11-01,0.13,1.62,2.33,0.14,0.45,0 +2014-12-01,0.21,1.64,2.21,0.37,0.51,0 +2015-01-01,0.2,1.37,1.88,0.17,0.27,0 +2015-02-01,0.22,1.47,1.98,0.11,0.26,0 +2015-03-01,0.25,1.52,2.04,0.04,0.28,0 +2015-04-01,0.23,1.35,1.94,-0.26,0.08,0 +2015-05-01,0.24,1.54,2.2,-0.1,0.33,0 +2015-06-01,0.28,1.68,2.36,0.05,0.5,0 +2015-07-01,0.3,1.63,2.32,0.14,0.5,0 +2015-08-01,0.38,1.54,2.17,0.31,0.56,0 +2015-09-01,0.37,1.49,2.17,0.33,0.65,0 +2015-10-01,0.26,1.39,2.07,0.21,0.57,0 +2015-11-01,0.48,1.67,2.26,0.4,0.69,0 +2015-12-01,0.65,1.7,2.24,0.46,0.73,0 +2016-01-01,0.54,1.52,2.09,0.33,0.67,0 +2016-02-01,0.53,1.22,1.78,0.14,0.47,0 +2016-03-01,0.66,1.38,1.89,-0.03,0.34,0 +2016-04-01,0.56,1.26,1.81,-0.22,0.19,0 +2016-05-01,0.59,1.3,1.81,-0.22,0.21,0 +2016-06-01,0.55,1.17,1.64,-0.27,0.17,0 +2016-07-01,0.51,1.07,1.5,-0.32,0.04,0 +2016-08-01,0.57,1.13,1.56,-0.17,0.09,0 +2016-09-01,0.59,1.18,1.63,-0.17,0.12,0 +2016-10-01,0.66,1.27,1.76,-0.26,0.1,0 +2016-11-01,0.74,1.6,2.14,-0.07,0.32,0 +2016-12-01,0.87,1.96,2.49,0.15,0.56,0 +2017-01-01,0.83,1.92,2.43,0.03,0.42,0 +2017-02-01,0.82,1.9,2.42,0.01,0.4,0 +2017-03-01,1.01,2.01,2.48,0.18,0.49,0 +2017-04-01,1.04,1.82,2.3,0.08,0.39,0 +2017-05-01,1.12,1.84,2.3,0.09,0.47,0 +2017-06-01,1.2,1.77,2.19,0.14,0.46,0 +2017-07-01,1.22,1.87,2.32,0.23,0.55,0 +2017-08-01,1.23,1.78,2.21,0.16,0.43,0 +2017-09-01,1.28,1.8,2.2,0.12,0.37,0 +2017-10-01,1.4,1.98,2.36,0.25,0.5,0 +2017-11-01,1.56,2.05,2.35,0.3,0.5,0 +2017-12-01,1.7,2.18,2.4,0.42,0.5,0 +2018-01-01,1.8,2.38,2.58,0.45,0.54,0 +2018-02-01,1.96,2.6,2.86,0.63,0.76,0 +2018-03-01,2.06,2.63,2.84,0.61,0.75,0 +2018-04-01,2.15,2.7,2.87,0.65,0.74,0 +2018-05-01,2.27,2.82,2.98,0.72,0.84,0 +2018-06-01,2.33,2.78,2.91,0.71,0.79,0 +2018-07-01,2.39,2.78,2.89,0.74,0.77,0 +2018-08-01,2.45,2.77,2.89,0.79,0.79,0 +2018-09-01,2.56,2.89,3.0,0.89,0.88,0 +2018-10-01,2.65,3.0,3.15,1.01,1.04,0 +2018-11-01,2.7,2.95,3.12,1.1,1.11,0 +2018-12-01,2.66,2.68,2.83,1.08,1.02,0 +2019-01-01,2.58,2.54,2.71,0.91,0.92,0 +2019-02-01,2.55,2.49,2.68,0.73,0.8,0 +2019-03-01,2.49,2.37,2.57,0.56,0.66,0 +2019-04-01,2.42,2.33,2.53,0.49,0.6,0 +2019-05-01,2.34,2.19,2.4,0.48,0.57,0 +2019-06-01,2.0,1.83,2.07,0.28,0.37,0 +2019-07-01,1.96,1.83,2.06,0.25,0.31,0 +2019-08-01,1.77,1.49,1.63,0.11,0.04,0 +2019-09-01,1.8,1.57,1.7,0.17,0.11,0 +2019-10-01,1.61,1.53,1.71,0.12,0.15,0 +2019-11-01,1.57,1.64,1.81,0.09,0.17,0 +2019-12-01,1.55,1.68,1.86,0.06,0.14,0 +2020-01-01,1.53,1.56,1.76,-0.09,0.04,0 +2020-02-01,1.41,1.32,1.5,-0.26,-0.11,0 +2020-03-01,0.33,0.59,0.87,-0.08,-0.12,1 +2020-04-01,0.18,0.39,0.66,-0.37,-0.45,1 +2020-05-01,0.16,0.34,0.67,-0.43,-0.44,0 +2020-06-01,0.18,0.34,0.73,-0.67,-0.54,0 +2020-07-01,0.15,0.28,0.62,-1.03,-0.83,0 +2020-08-01,0.13,0.27,0.65,-1.28,-1.01,0 +2020-09-01,0.13,0.27,0.68,-1.26,-0.98,0 +2020-10-01,0.13,0.34,0.79,-1.23,-0.92,0 +2020-11-01,0.12,0.39,0.87,-1.24,-0.84,0 +2020-12-01,0.1,0.39,0.93,-1.48,-0.98,0 +2021-01-01,0.1,0.45,1.08,-1.66,-1.0,0 +2021-02-01,0.07,0.54,1.26,-1.77,-0.92,0 +2021-03-01,0.08,0.82,1.61,-1.67,-0.66,0 +2021-04-01,0.06,0.86,1.64,-1.67,-0.71,0 +2021-05-01,0.05,0.82,1.62,-1.83,-0.85,0 +2021-06-01,0.07,0.84,1.52,-1.63,-0.82,0 +2021-07-01,0.08,0.76,1.32,-1.73,-1.01,0 +2021-08-01,0.07,0.77,1.28,-1.72,-1.07,0 +2021-09-01,0.08,0.86,1.37,-1.63,-0.97,0 +2021-10-01,0.11,1.11,1.58,-1.64,-0.95,0 +2021-11-01,0.18,1.2,1.56,-1.78,-1.06,0 +2021-12-01,0.3,1.23,1.47,-1.52,-0.99,0 +2022-01-01,0.55,1.54,1.76,-1.26,-0.69,0 +2022-02-01,1.0,1.81,1.93,-1.06,-0.52,0 +2022-03-01,1.34,2.11,2.13,-1.3,-0.72,0 +2022-04-01,1.89,2.78,2.75,-0.54,-0.14,0 +2022-05-01,2.06,2.87,2.9,-0.15,0.21,0 +2022-06-01,2.65,3.19,3.14,0.3,0.53,0 +2022-07-01,3.02,2.96,2.9,0.38,0.53,0 +2022-08-01,3.28,3.03,2.9,0.34,0.39,0 +2022-09-01,3.89,3.7,3.52,1.25,1.14,0 +2022-10-01,4.43,4.18,3.98,1.71,1.59,0 +2022-11-01,4.73,4.06,3.89,1.61,1.52,0 +2022-12-01,4.68,3.76,3.62,1.45,1.36,0 +2023-01-01,4.69,3.64,3.53,1.41,1.29,0 +2023-02-01,4.93,3.94,3.75,1.5,1.41,0 +2023-03-01,4.68,3.82,3.66,1.45,1.36,0 +2023-04-01,4.68,3.54,3.46,1.23,1.19,0 +2023-05-01,4.91,3.59,3.57,1.44,1.36,0 +2023-06-01,5.24,3.95,3.75,1.81,1.55,0 +2023-07-01,5.37,4.14,3.9,1.93,1.6,0 +2023-08-01,5.37,4.31,4.17,2.07,1.83,0 +2023-09-01,5.44,4.49,4.38,2.23,2.04,0 +2023-10-01,5.42,4.77,4.8,2.46,2.41,0 +2023-11-01,5.28,4.49,4.5,2.24,2.2,0 +2023-12-01,4.96,4.0,4.02,1.88,1.84,0 +2024-01-01,4.79,3.98,4.06,1.76,1.79,0 +2024-02-01,4.92,4.19,4.21,1.87,1.93,0 +2024-03-01,4.99,4.2,4.21,1.82,1.9,0 +2024-04-01,5.14,4.56,4.54,2.11,2.15,0 +2024-05-01,5.16,4.5,4.48,2.17,2.15,0 +2024-06-01,5.11,4.32,4.31,2.1,2.05,0 +2024-07-01,4.9,4.16,4.25,1.98,1.97,0 +2024-08-01,4.43,3.71,3.87,1.74,1.76,0 +2024-09-01,4.03,3.5,3.72,1.53,1.62,0 +2024-10-01,4.2,3.91,4.1,1.68,1.81,0 +2024-11-01,4.33,4.23,4.36,1.84,2.03,0 +2024-12-01,4.23,4.25,4.39,1.89,2.09,0 diff --git a/lectures/fred_data.csv.yml b/lectures/fred_data.csv.yml new file mode 100644 index 0000000..0690897 --- /dev/null +++ b/lectures/fred_data.csv.yml @@ -0,0 +1,191 @@ +# Manifest for fred_data.csv — migrated out of +# QuantEcon/lecture-python-advanced.myst, where it sat at +# lectures/_static/lecture_specific/risk_aversion_or_mistaken_beliefs/ and was +# read over the repo's OWN raw URL (PLAN Phase 8, wave C2). +# +# The name is on the phase-2 rename review (QuantEcon/data-lectures#87, +# decision of 2026-08-17): it migrates under its current name, and the org-wide +# Trees sweep of 2026-08-17 (re-derived in the QuantEcon/data-lectures#96 +# validation) found no other file of this name anywhere in the org, so the +# deferral introduces no ambiguity. +# +# The builder below is a RECONSTRUCTION, not a recovered original: no build +# script for this file has ever existed in the lecture repo. Unlike the BBH +# snapshot files it reads LIVE FRED, deliberately — none of these six series is +# revised the way the national accounts are — and it reproduced the committed +# file byte for byte on 2026-08-18, three end-to-end runs. It therefore takes +# `committed`, not `unrecovered`. This supersedes the previous entry in +# scripts/audit_annotations.yml (provenance: constructed-lost, "no build script +# anywhere") — right when it was written, wrong now; delete that entry in the +# same PR that lands this manifest. + +filename: fred_data.csv +title: US Treasury yields and NBER recessions — GS1/GS5/GS10, DFII5/DFII10 and USREC, monthly 1953-04 to 2024-12 +description: > + Six monthly FRED series on one date index: three nominal Treasury + constant-maturity yields (GS1, GS5, GS10), two real (TIPS) constant-maturity + yields (DFII5, DFII10) and the NBER recession indicator (USREC), 1953-04 to + 2024-12. The `risk_aversion_or_mistaken_beliefs` lecture plots the yields + with recession shading in its opening figure; every series is a level in the + units FRED publishes. + +# Constructed: a six-series extract on a fixed window, with the two daily TIPS +# series averaged to monthly. Nothing here is republished as distributed +# (fredgraph serves one series per file), so it is not verbatim. +class: constructed + +source: + name: FRED (Federal Reserve Bank of St. Louis) — series GS1, GS5, GS10, DFII5, DFII10, USREC + url: https://fred.stlouisfed.org + doi: null + version: > + Live FRED as of 2026-08-18, and the distinction that matters for the BBH + files does not bite here: constant-maturity yields are historical market + rates from the Board of Governors' H.15 release and USREC is a dummy built + from NBER's published turning points, none of which is revised or rebased + the way the national accounts are. A fresh fetch on 2026-08-18 reproduced + the committed bytes exactly (see integrity). + series: > + GS1, GS5, GS10 — market yield on US Treasury securities at 1-, 5- and + 10-year constant maturity, percent per annum, monthly, NSA (Board of + Governors H.15). DFII5, DFII10 — market yield on US Treasury + inflation-indexed securities at 5- and 10-year constant maturity, percent + per annum, DAILY at source, here the monthly average (fredgraph + `fq=Monthly&fam=avg`), published from 2003-01. USREC — NBER-based recession + indicator, monthly 0/1, from the NBER's published peak/trough chronology. + citation: > + Board of Governors of the Federal Reserve System, "Selected Interest Rates + (H.15)", retrieved from FRED, Federal Reserve Bank of St. Louis (series + GS1, GS5, GS10, DFII5, DFII10); Federal Reserve Bank of St. Louis, "NBER + based Recession Indicators for the United States" (USREC), from the + National Bureau of Economic Research's business cycle chronology. + note: > + The lecture's prose names exactly these six series and FRED as the source, + and for once the prose survives measurement: the committed file reproduces + from live FRED byte for byte. One served-format drift worth recording: + fredgraph.csv now titles its date column `observation_date` where this file + (and fredgraph at the time it was cut) says `DATE` — the builder renames on + read, and a naive re-fetch that trusted the served header would produce a + file whose first cell differs. + +license: + name: null + url: https://fred.stlouisfed.org/series/USREC + redistribution: permitted + verified: 2026-08-18 + note: > + Five of the six series are the Board of Governors' H.15 release delivered + through FRED — US Government works. The sixth, USREC, is constructed by the + St. Louis Fed from the NBER's published business-cycle dates, which are + facts. The BBH macro manifest's systematic check of 2026-08-17 (fourteen + FRED series pages against FRED's SP500 page as a positive control, which + does carry S&P's reproduction-prohibited block) found no restriction on any + federal series page; the same check could not be re-run for this manifest + on 2026-08-18 because fred.stlouisfed.org returned 403 to both the target + pages and the control from the authoring environment, so `permitted` rests + on the series' federal provenance plus that dated precedent. Nothing to + register on QuantEcon/data-lectures#35. + +retrieved: null # inherited-undated: the bytes arrived in + # lecture-python-advanced.myst with no recorded + # retrieval date, and none was reconstructed + # from git history (AGENTS.md). It costs + # nothing — the builder reproduces these bytes + # from live FRED exactly, so the content is + # pinned by reproduction rather than by a date. + # The window end (2024-12-01) is the only + # vintage fact, and it is pinned in the builder. +maintainer: QuantEcon + +# --------------------------------------------------------------------------- +# Integrity (PLAN Phase 7) +# --------------------------------------------------------------------------- +# Migration check (repoint gate, not a manifest field): the bytes landing here +# are byte-identical to the copy the lecture reads today — sha256 45a4fd41… +# against lecture-python-advanced.myst @ 6ff5ab3, +# lectures/_static/lecture_specific/risk_aversion_or_mistaken_beliefs/. The +# org-wide Trees sweep (2026-08-17, re-derived in QuantEcon/data-lectures#96) +# found that path and no other copy anywhere, so there is no sibling to keep in +# step. A repoint to this file therefore cannot change a figure. + +integrity: + sha256: 45a4fd41aeadf55072ea8e753c12dcb50ccb89bc34fb7eeb5e451ea0796811bc + upstream: + # `verified` in the sense AGENTS.md defines for a constructed dataset — + # "re-run the builder and compare". Run three times on the date below + # against live FRED, each run reproducing this file byte for byte. That is + # also what licenses `builder_status: committed` below. + status: verified + date: 2026-08-18 + against: builders/fred_data.py + note: > + Reproduced exactly from live FRED — which is possible, where it is not + for the BBH snapshot files, because constant-maturity yields and the + recession dummy are stable history rather than revised aggregates. The + reproduction pins the DFII pair to fredgraph's Monthly/avg aggregation of + the daily series; a fetch of the daily series aggregated any other way + does not match. + +# --------------------------------------------------------------------------- +# Shape +# --------------------------------------------------------------------------- +# Measured from the committed bytes 2026-08-18 (pandas 2.3.3), and asserted by +# the builder's validate() stage on every run. + +schema: + format: csv + columns: + - {name: DATE, dtype: datetime, description: "first-of-month date stamp, 1953-04-01 to 2024-12-01, 861 rows on an unbroken monthly grid. The CSV index column; the lecture reads it with parse_dates and index_col"} + - {name: GS1, dtype: float64, description: "1-year Treasury constant-maturity yield, percent per annum, monthly, NSA. Observed 0.05-16.72"} + - {name: GS5, dtype: float64, description: "5-year Treasury constant-maturity yield, percent per annum. Observed 0.27-15.93"} + - {name: GS10, dtype: float64, description: "10-year Treasury constant-maturity yield, percent per annum. Observed 0.62-15.32"} + - {name: DFII5, dtype: float64, description: "5-year Treasury inflation-indexed (TIPS) constant-maturity yield, percent per annum, monthly average of daily. Observed -1.83-3.69, and EMPTY before 2003-01 — see known_nulls"} + - {name: DFII10, dtype: float64, description: "10-year TIPS constant-maturity yield, percent per annum, monthly average of daily. Observed -1.07-2.89, and EMPTY before 2003-01 — see known_nulls"} + - {name: USREC, dtype: int64, description: "NBER recession indicator, 1 in recession months and 0 otherwise. The lecture uses it for the figure's shaded bands"} + row_count_floor: 861 # exact by design, not a floor with headroom: + # the window is fixed at 1953-04..2024-12 and + # does not grow. validate() asserts equality. + date_range: {start: 1953-04-01, end: 2024-12-01} + + # The only nulls in the file, and they are structural: FRED publishes the + # TIPS yields from 2003-01, so 1953-04..2002-12 (597 months) is empty in both + # DFII columns and complete in every other. validate() asserts these exact + # holes and no others, so a new hole fails instead of shipping. + known_nulls: {DFII5: 597, DFII10: 597} + +# --------------------------------------------------------------------------- +# Consumers — how a correction knows what to rebuild +# --------------------------------------------------------------------------- +# One consumer, established by basename grep across all nine lecture clones +# plus the org-wide Trees sweep (2026-08-17, re-derived in +# QuantEcon/data-lectures#96: exactly one org-wide hit for this basename). +# lecture-python-advanced.myst has no translation repo; the generated +# lecture-python-advanced.notebooks mirror bakes the same URL into +# risk_aversion_or_mistaken_beliefs.ipynb, self-heals after a publish tag, and +# is not a repo anyone repoints, so it is deliberately not listed (wave A4 +# precedent). +# +# Note for whoever repoints: the read is ALREADY an HTTP read of the lecture +# repo's own raw URL, SPLIT ACROSS TWO ADJACENT STRING LITERALS +# (risk_aversion_or_mistaken_beliefs.md:1613-1617) with the split falling +# mid-path inside `refs/heads/`, so a whole-URL grep returns a confident zero — +# sweep by basename. The in-repo spelling is `refs/heads/main`, which this +# repo's audit REJECTS for data-lectures reads (ref must be exactly `main`), so +# the repoint must not copy the spelling. Deleting the lecture-repo copy before +# the repoint publishes breaks every notebook and Colab run AT RUNTIME with no +# stale-serving grace period — this file and the published notebooks fetch it +# live at cell execution. +consumers: + - repo: QuantEcon/lecture-python-advanced.myst + file: lectures/risk_aversion_or_mistaken_beliefs.md + note: > + Reads its own copy over an own-repo raw URL + (`lecture-python-advanced.myst/.../risk_aversion_or_mistaken_beliefs/`), + not this file — the repoint PR in lecture-python-advanced.myst follows + this one, and migration.yml records the dataset as `landed` until it + merges. Listed now because a correction to these bytes must reach the + lecture regardless of where it currently reads them from + (QuantEcon/data-lectures#91). + +builder: builders/fred_data.py +builder_status: committed diff --git a/lectures/test_pwt.csv b/lectures/test_pwt.csv new file mode 100644 index 0000000..03ae3be --- /dev/null +++ b/lectures/test_pwt.csv @@ -0,0 +1,9 @@ +"country","country isocode","year","POP","XRAT","tcgdp","cc","cg" +"Argentina","ARG","2000","37335.653","0.9995","295072.21869","75.716805379","5.5788042896" +"Australia","AUS","2000","19053.186","1.72483","541804.6521","67.759025993","6.7200975332" +"India","IND","2000","1006300.297","44.9416","1728144.3748","64.575551328","14.072205773" +"Israel","ISR","2000","6114.57","4.07733","129253.89423","64.436450847","10.266688415" +"Malawi","MWI","2000","11801.505","59.543808333","5026.2217836","74.707624181","11.658954494" +"South Africa","ZAF","2000","45064.098","6.93983","227242.36949","72.718710427","5.7265463933" +"United States","USA","2000","282171.957","1","9898700","72.347054303","6.0324539789" +"Uruguay","URY","2000","3219.793","12.099591667","25255.961693","78.978740282","5.108067988" diff --git a/lectures/test_pwt.csv.yml b/lectures/test_pwt.csv.yml new file mode 100644 index 0000000..ee30ffc --- /dev/null +++ b/lectures/test_pwt.csv.yml @@ -0,0 +1,204 @@ +# Manifest for test_pwt.csv — migrated out of +# QuantEcon/lecture-python-programming, where it sat at +# lectures/_static/lecture_specific/pandas/data/ and was read over the repo's +# OWN raw URL from two lectures and three synced translation repos (PLAN +# Phase 8, Track D). +# +# The name is on the phase-2 rename review (QuantEcon/data-lectures#87, +# decision of 2026-08-17): it migrates under its current name. +# +# PROVENANCE IS WRONG IN THE CONSUMING LECTURE, measurably. pandas.md's prose +# says the file "is taken from the Penn World Tables" and links PWT 7.0 — but +# the committed values match NO downloadable PWT vintage. Checked 2026-08-18 +# against four archives from rug.nl/ggdc: PWT 7.0 (pwt70_06032011version.zip, +# member pwt70_w_country_names.csv sha256 4623b92a…) has different tcgdp/cc/cg +# for every row and renamed the isocode column; PWT 6.3 +# (pwt63_nov182009version.zip, member sha256 f9609c42…) matches Argentina and +# Australia POP but nothing else and has no tcgdp column; PWT 6.2 +# (pwt62_data.xlsx, sha256 a3293337…) and PWT 6.1 (pwt61_data.xlsx, sha256 +# 1477a118…) match nothing. The committed +# header uses the 6.x-era column name `country isocode` while `tcgdp` appears +# only in the 7.x releases checked — so the extract mixes or predates the +# vintages still downloadable. Hence `unrecovered`, with this paragraph as the +# recovery-attempt record. + +filename: test_pwt.csv +title: Penn World Table teaching extract — eight countries, year 2000 +description: > + An eight-row teaching extract in Penn World Table layout: eight countries + (Argentina, Australia, India, Israel, Malawi, South Africa, United States, + Uruguay), year 2000, with population (POP, thousands), the exchange rate + (XRAT), total PPP-converted GDP (tcgdp, millions of international dollars), + and the consumption and government shares of GDP (cc, cg). The pandas and + polars lectures in lecture-python-programming use it as their first + read_csv example and as the small dataset in a timing comparison. Every + field in the file is quoted (csv QUOTE_ALL style) — any rewrite must + preserve that to keep byte identity. + +# Constructed: an eight-row, eight-column extract of a PWT release (vintage +# unestablished — see header), assembled by hand for teaching. +class: constructed + +source: + name: Penn World Table (Heston, Summers and Aten; PWT 6.x/7.x era) — exact vintage unestablished + url: https://www.rug.nl/ggdc/productivity/pwt/pwt-releases/pwt-7.0 + # The URL the lecture prose cites — kept as the recorded-as-found claim even + # though the bytes measurably do not come from that release (see header). + doi: null + version: > + Unestablished. The lecture prose cites PWT 7.0; the committed values match + neither PWT 7.0 nor any of 6.3/6.2/6.1 (comparison measured 2026-08-18, + archive hashes in the header comment). The header naming (`country + isocode`) is 6.x-era; `tcgdp` exists in the 7.x layout. + series: country, country isocode, year, POP, XRAT, tcgdp, cc, cg + citation: > + Heston, Alan, Robert Summers, and Bettina Aten. Penn World Table (Center + for International Comparisons of Production, Income and Prices, University + of Pennsylvania; now maintained by the Groningen Growth and Development + Centre). Exact version unrecorded; extract assembled for the QuantEcon + pandas lecture. + note: > + A TEACHING extract: its job is to be a small, familiar CSV, not to carry + a defensible vintage. The value mismatch matters only if someone treats + the numbers as citable PWT data — which the lectures do not; they compute + illustrative transformations. Recorded so nobody spends another afternoon + hunting the vintage: the four downloadable candidates are eliminated + above. + +license: + name: null + url: https://www.rug.nl/ggdc/productivity/pwt/ + # Recorded as found, per the record-and-track policy (AGENTS.md, "Licensing + # and attribution"). + redistribution: permitted + verified: 2026-08-18 + note: > + An inherited exposure, served publicly from lecture-python-programming + (and its three translation repos) for years. The Penn World Table is + distributed for free academic use and its current releases carry CC BY + 4.0; what is republished here is an eight-row extract with full credit. + No permission has been sought. Nothing to register on + QuantEcon/data-lectures#35. + +retrieved: null # inherited-undated: no retrieval date was + # recorded and none is reconstructed from git + # history (AGENTS.md). Paired, as the rule + # requires, with the resolved `unverifiable` + # below and the vintage-elimination record in + # the header. +maintainer: QuantEcon + +# --------------------------------------------------------------------------- +# Integrity (PLAN Phase 7) +# --------------------------------------------------------------------------- +# Migration check (repoint gate, not a manifest field): the bytes landing here +# are byte-identical to the copy every consumer reads today — sha256 229316b3… +# against lecture-python-programming @ 81d9734, +# lectures/_static/lecture_specific/pandas/data/. The three translation-repo +# copies carry the identical file and their lectures read the SOURCE repo's +# raw URL, not their local copies. A repoint to this file therefore cannot +# change a figure. + +integrity: + sha256: 229316b32902ded0231bcdae576da34d3113255e5d51265e72a7f3ee5e964fc2 + upstream: + # No downloadable PWT release reproduces these values, so there is nothing + # to verify against — recorded as the honest terminal state, with the + # elimination measurements in the header comment as evidence. + status: unverifiable + date: 2026-08-18 + against: null + note: > + Four candidate vintages fetched and eliminated 2026-08-18 (PWT 7.0, + 6.3, 6.2, 6.1 — hashes in the header comment). The values are + PWT-shaped but match no release still downloadable; the extract likely + predates the rug.nl re-hosting or was hand-edited. Any future + re-derivation from a current PWT would be a NEW dataset under a new + name ("Corrections vs vintages", AGENTS.md). + +# --------------------------------------------------------------------------- +# Shape +# --------------------------------------------------------------------------- +# Measured from the committed bytes 2026-08-18 (pandas 2.3.3). + +schema: + format: csv + columns: + - {name: country, dtype: object, description: "country name, 8 unique — Argentina, Australia, India, Israel, Malawi, South Africa, United States, Uruguay"} + - {name: country isocode, dtype: object, description: "ISO 3166-1 alpha-3 code. The SPACE in this column name is load-bearing for the lectures, which demonstrate column selection on it"} + - {name: year, dtype: int64, description: "2000 in every row"} + - {name: POP, dtype: float64, description: "population, thousands. Observed 3219.793-1006300.297"} + - {name: XRAT, dtype: float64, description: "exchange rate, national currency per US dollar. Observed 0.9995-59.543808"} + - {name: tcgdp, dtype: float64, description: "total PPP-converted GDP, millions of international dollars. Observed 5026.22-9898700.0"} + - {name: cc, dtype: float64, description: "consumption share of PPP-converted GDP per capita, percent. Observed 64.44-78.98"} + - {name: cg, dtype: float64, description: "government share of PPP-converted GDP per capita, percent. Observed 5.11-14.07"} + row_count_floor: 8 # exact by design: a frozen teaching extract + date_range: {start: 2000, end: 2000} + known_nulls: {} # genuinely none + +# --------------------------------------------------------------------------- +# Consumers — how a correction knows what to rebuild +# --------------------------------------------------------------------------- +# EIGHT consuming lecture files across four repos, all reading the SOURCE +# repo's raw URL (raw.githubusercontent.com/QuantEcon/lecture-python-programming/ +# main/...) — so deleting the source copy breaks all of them AT RUNTIME with no +# stale-serving grace period, translations included. Established 2026-08-18 by +# basename grep with size controls against the live default branches of all +# four repos. Three of polars.md's four reads build the URL across THREE +# adjacent string literals; a whole-URL grep returns a confident zero — sweep +# by basename. +# +# The translation repos are OUTSIDE the audit's scan set, so they are recorded +# here or nowhere. They are not direct repoint targets: their .md files are +# overwritten wholesale by lecture-python-programming's sync-translations-* +# workflows, so the repoint propagates by sync after the source repoint +# merges — but the DELETION gate must wait until all three have synced, +# rebuilt and republished, or their live sites break at cell execution. +consumers: + - repo: QuantEcon/lecture-python-programming + file: lectures/pandas.md + note: > + Reads the repo's own raw URL (one unsplit literal, pandas.md:172), not + this file — the repoint PR follows this one, and migration.yml records + the dataset as `landed` until it merges. The prose at pandas.md:156-176 + names the file and tabulates its columns, and must move with the read + (QuantEcon/data-lectures#91). + - repo: QuantEcon/lecture-python-programming + file: lectures/polars.md + note: > + Reads the repo's own raw URL three times (polars.md:161-163, 347-349, + 425-427), each split across three adjacent string literals — collapse + at the repoint, do not patch the stem (NEWQDATA precedent). One `url` + binding at 425 feeds two readers (pandas and polars timing comparison). + - repo: QuantEcon/lecture-python-programming.zh-cn + file: lectures/pandas.md + note: > + Synced translation; reads lecture-python-programming's raw URL, not this + file. Repointed indirectly when the source repoint syncs; do not delete + the source copy until this repo has synced and republished. Tracked with + the Track D record in QuantEcon/data-lectures migration.yml. + - repo: QuantEcon/lecture-python-programming.zh-cn + file: lectures/polars.md + note: > + Synced translation; same URL, same sync dependency as pandas.md above. + - repo: QuantEcon/lecture-python-programming.fr + file: lectures/pandas.md + note: > + Synced translation; reads lecture-python-programming's raw URL, not this + file. Same sync dependency as the zh-cn entries. + - repo: QuantEcon/lecture-python-programming.fr + file: lectures/polars.md + note: > + Synced translation; same URL, same sync dependency. + - repo: QuantEcon/lecture-python-programming.fa + file: lectures/pandas.md + note: > + Synced translation; reads lecture-python-programming's raw URL, not this + file. Same sync dependency as the zh-cn entries. + - repo: QuantEcon/lecture-python-programming.fa + file: lectures/polars.md + note: > + Synced translation; same URL, same sync dependency. + +builder: null +builder_status: unrecovered diff --git a/migration.yml b/migration.yml index ec6fc3d..596c348 100644 --- a/migration.yml +++ b/migration.yml @@ -709,10 +709,9 @@ datasets: cutover: null # Wave C1 — the three lecture-python-advanced.myst datasets that needed no - # rename decision. `landed` here and NOT yet `repointed`: the lecture still - # reads its own copies, and the repoint PR follows this one (advanced.myst - # caches notebook execution, so the repointed cells re-execute against - # whatever is on this repo's main at that moment — data first, always). + # rename decision. Landed and repointed 2026-08-17 (data first, always: + # advanced.myst caches notebook execution, so the repointed cells re-execute + # against whatever is on this repo's main at that moment). bbh_macro_quarterly.csv: pilot: C1 status: repointed @@ -755,6 +754,71 @@ datasets: date: 2026-08-17 cutover: null + # Wave C2 — the three lecture-python-advanced.myst datasets that needed + # decisions, all settled 2026-08-17 on QuantEcon/workspace-lectures#45: + # renames deferred to the phase-2 review (QuantEcon/data-lectures#87), and + # dataBHS.mat converts to dataBHS.csv (stem preserved). + + # Already read over an own-repo raw URL rather than a local path, so deleting + # it from advanced.myst breaks readers at RUNTIME with no stale-serving grace + # period. Repoint and publish before any deletion. The in-repo URL is split + # across adjacent string literals with the `refs/heads/main` spelling — + # sweep by basename, and do not copy the spelling into the repoint. + fred_data.csv: + pilot: C2 + status: landed + prior_pattern: own-repo + landed: + pr: QuantEcon/data-lectures#98 + date: 2026-08-18 + repoints: [] + cutover: null + + acs_data_summary.csv: + pilot: C2 + status: landed + prior_pattern: local-path + landed: + pr: QuantEcon/data-lectures#98 + date: 2026-08-18 + repoints: [] + cutover: null + + # Lands as a FORMAT CONVERSION of advanced.myst's lectures/dataBHS.mat — a + # bug fix, not tidying: the .mat sat outside html_static_path, so the + # published site 404s it while the published notebook calls loadmat on it. + # The audit keys on basenames, so until the repoint removes the .mat read, + # scripts/audit_annotations.yml keeps its dataBHS.mat entry (the basenames + # differ — dual_recorded cannot fire); the flip PR deletes it. + dataBHS.csv: + pilot: C2 + status: landed + prior_pattern: local-path + landed: + pr: QuantEcon/data-lectures#98 + date: 2026-08-18 + repoints: [] + cutover: null + + # Track D — the one lecture-python-programming dataset, riding with C2 (its + # rename is on the same phase-2 review). Read over the SOURCE repo's raw URL + # from pandas.md, polars.md AND the three synced translation repos + # (.zh-cn/.fr/.fa), so the deletion gate is wider than usual: repoint the + # source, wait for the sync-translations workflows to propagate it, and + # verify all four repos have rebuilt and republished before deleting — + # otherwise the translations break at cell execution with no grace period. + # Three of polars.md's four reads split the URL across three string + # literals; sweep by basename. + test_pwt.csv: + pilot: D + status: landed + prior_pattern: own-repo + landed: + pr: QuantEcon/data-lectures#98 + date: 2026-08-18 + repoints: [] + cutover: null + # Planned waves that have not landed anything here yet. `datasets` names the # files as the audit sees them today, so the dashboard can join the two views. # `title` is the reader-facing milestone name (the dashboard is read by people diff --git a/requirements.txt b/requirements.txt index 67d70a0..aae4e80 100644 --- a/requirements.txt +++ b/requirements.txt @@ -14,5 +14,5 @@ openpyxl==3.1.5 # the .xlsx reader pandas defers to — builders/ # need it. The last two have needed it since they landed; # it was never declared because CI does not run builders. PyYAML==6.0.3 # scripts/build_catalog.py — parse the sidecar manifests -scipy==1.16.3 # builders/NEWQDATA.py — loadmat, the only way to read a - # MATLAB 5.0 .MAT; pandas cannot +scipy==1.16.3 # builders/NEWQDATA.py and builders/dataBHS.py — loadmat, + # the only way to read a MATLAB 5.0 .MAT; pandas cannot diff --git a/scripts/audit_annotations.yml b/scripts/audit_annotations.yml index 232de0e..0f386e4 100644 --- a/scripts/audit_annotations.yml +++ b/scripts/audit_annotations.yml @@ -18,18 +18,15 @@ # toy invented in-lecture teaching table datasets: - acs_data_summary.csv: - description: ACS occupation summary - provenance: constructed-lost - note: construction (ACS filters, grouping, sorting) described in lecture prose only dataBHS.mat: description: US consumption/income series, MATLAB replication bundle provenance: verbatim flags: [lectures-root] - fred_data.csv: - description: FRED snapshot — GS1, GS5, GS10, DFII5, DFII10, USREC - provenance: constructed-lost - note: lecture names the FRED series; no build script anywhere + note: > + Successor manifest landed as lectures/dataBHS.csv.yml (wave C2 format + conversion); this entry covers the lecture's remaining loadmat read of + the .mat and is deleted by the flip PR once the repoint removes that + read. The basenames differ, so dual_recorded cannot fire meanwhile. graph.txt: description: 100-node weighted digraph for the shortest-path problem provenance: toy @@ -42,9 +39,6 @@ datasets: all. Two of the four repos also commit a copy the %%file cell overwrites before reading (intro, dp) — those are shadowed; jax and wasm commit no copy at all, which is the cleaner shape - test_pwt.csv: - description: Penn World Table 7.0 extract - provenance: author-assembled # --------------------------------------------------------------------------- # Live-API uses, keyed repo:lecture:access. `pedagogy` says WHY the call is diff --git a/sources/README.md b/sources/README.md index d9c464d..8cdd1a3 100644 --- a/sources/README.md +++ b/sources/README.md @@ -171,3 +171,53 @@ rather than by analogy to a verbatim re-fetch. 7,104 B — four orders of magnitude below any limit that matters. It is here on the un-refetchability test alone, which is the test, and a useful counterweight to reading this directory as "the big-file directory". + +--- + +## `dataBHS.mat` + +| | | +| --- | --- | +| **Origin** | The authors' MATLAB data file for "Doubts or variability?", inherited as `lectures/dataBHS.mat` in `QuantEcon/lecture-python-advanced.myst` (where it sat at the lectures root, outside `html_static_path`, and was never served by the published site). Byte-identical copies exist in `lecture-tools-techniques` (which reads its own), `python-lecture-sandpit.myst` and `lecture-mapping` — all QuantEcon-internal descendants of the same inheritance, not an upstream | +| **Upstream** | Barillas, Francisco, Lars Peter Hansen, and Thomas J. Sargent (2009), *Doubts or variability?*, Journal of Economic Theory 144(6), 2388-2418 | +| **Upstream identifier** | DOI [10.1016/j.jet.2008.11.014](https://doi.org/10.1016/j.jet.2008.11.014) — the **article**. The JET article carries no data supplement | +| **Retrieved** | `null` — no retrieval date was recorded upstream; the lecture-repo commit dates record when QuantEcon acquired it, not when it came from the authors. Do not promote one to the other | +| **Licence** | `null` — no licence statement exists at the journal or the authors' pages. Registered on [#35](https://github.com/QuantEcon/data-lectures/issues/35) | +| **`sha256`** | `28c5f85286718e70b205f6a3fb269ebb49bd635194e2d0d488409b017be5e890` | +| **Size** | 5,588 B | +| **Consumed by** | `builders/dataBHS.py` (`builder_status: committed`) | +| **Produces** | `lectures/dataBHS.csv` | + +### Why this file is here rather than fetched at run time + +Un-refetchable from any authoritative source, on the NEWQDATA precedent. +Searched 2026-08-18, each zero beside a passing control: + +- **tomsargent.com**'s source-code page returns 404 (and the site's https + endpoint does not answer); +- **larspeterhansen.org** hosts the paper's PDF but lists no code or data for + it on the research pages; +- the **Journal of Economic Theory** article (ScienceDirect) shows no + supplementary material; +- a **GitHub-wide code search** for `dataBHS` returns only QuantEcon's own + inherited copies of this blob (plus token-collision noise in unrelated + JavaScript), with the same search finding `NEWQDATA` in five QuantEcon + files as the positive control. + +Unlike NEWQDATA there is not even a third-party mirror of an authors' +directory — every locatable copy descends from the QuantEcon inheritance. So +the input is committed here, and `builders/dataBHS.py` reads it from disk. + +The file is MATLAB 5.0 (PCWIN, created 2007-05-11) holding exactly three +(236,1) float64 arrays `c`, `rb`, `rs` — 1948Q1-2006Q4, the paper's sample. +The conversion is value-preserving (three arrays to three columns, no +filtering, no rescaling, no reordering); the CSV parses back **bit-exactly** +under `float_precision='round_trip'`, and the consuming lecture's histogram is +identical under pandas' default parser (PLAN-QELD-PACKAGE.md §4.3 measured why +those are different claims). That is what earns the manifest's +`integrity.upstream.status: verified` under AGENTS.md's definition for a +`constructed` dataset. + +### Size + +5,588 B — here on the un-refetchability test alone, like NEWQDATA above. diff --git a/sources/dataBHS.mat b/sources/dataBHS.mat new file mode 100644 index 0000000..6b8e5b1 --- /dev/null +++ b/sources/dataBHS.mat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:28c5f85286718e70b205f6a3fb269ebb49bd635194e2d0d488409b017be5e890 +size 5588