Skip to content

VALIDATION: independent review of the 2026-08-18 Track C wave C2 + Track D migration and prune work #100

Description

@mmcky

Everything below was verified during the work, by the same session that did it — same tools, same URL forms, same environment, same mental model. This issue exists to break that circularity: run these checks in a fresh session with a diversified toolchain and report what survives.

Bias to test for: this session's verification monoculture was curl-based HTTP status/byte probes plus basename greps, and a single pinned venv (python 3.13, pandas 2.3.3, scipy 1.16.3) for every numeric claim; scripts/build_audit.py was run by the session itself for all four strict/dry-run measurements. Confirm reader-facing outcomes independently of all of that: open the published pages and notebooks in a browser, execute code rather than grepping it, use a different Python/pandas where a check involves parsing, and re-derive counts with your own parser rather than confirming the session's numbers.

What landed (2026-08-18, all merged the same day)

repo landed
QuantEcon/lecture-python-advanced.myst #374 (prune jb clean . --html in ci+publish, 6ff5ab3), #375 (wave C2 repoint, 878b87a), #376 (wave C2 deletion, 0c96791), tag publish-2026aug18 (rerun of run 32087672065 after a 75-min hang on the texlive apt step; deploy had not run when cancelled)
QuantEcon/data-lectures #98 (land wave C2 + Track D: 4 datasets, 2 builders, sources/dataBHS.mat under LFS, f42eeaf), #99 (flip all four to repointed, f09485c)
QuantEcon/lecture-python-programming #612 (Track D repoint, 55c87c9) — whose merge auto-opened translation-sync PRs lecture-python-programming.zh-cn#92, lecture-python-programming.fr#33, lecture-python-programming.fa#153 (all still open, deliberately)
QuantEcon/lecture-tools-techniques issue #11 filed (byte-identical dataBHS.mat, same broken-downloadable-notebook bug)
QuantEcon/workspace-lectures comments: #41 (prune repo 1 of 9 landed), #40 (re-audit booked for ~2026-08-24 with checklist)

1. Reader-facing outcomes

  • The clearest incident to confirm fixed: https://python-advanced.quantecon.org/_notebooks/five_preferences.ipynb downloads, and executes end-to-end in a clean environment with no dataBHS.mat on disk (Colab or local jupyter). Before this work the notebook called loadmat('dataBHS.mat') against a file the site serves at 404, so it could not run. The session verified the URL swap by grep, never by executing the published notebook — this check is the unexercised surface.
  • Same execution check for /_notebooks/risk_aversion_or_mistaken_beliefs.ipynb (live FRED-shaped read from data-lectures) and /_notebooks/match_transport.ipynb — at minimum their data-read cells.
  • The three published pages (risk_aversion_or_mistaken_beliefs.html, match_transport.html, five_preferences.html) render their data-driven figures — in particular five_preferences' consumption-growth histogram with two density curves (its figure was claimed shape-preserved through the loadmat→read_csv conversion).
  • https://github.com/QuantEcon/data-lectures/raw/main/lectures/<f> serves 200 with sizes 27963 / 14365 / 10160 / 793 for fred_data.csv / acs_data_summary.csv / dataBHS.csv / test_pwt.csv, and a never-existed control path 404s.
  • The old raw blob https://raw.githubusercontent.com/QuantEcon/lecture-python-advanced.myst/main/lectures/_static/lecture_specific/risk_aversion_or_mistaken_beliefs/fred_data.csv is 404, and the published site still serves /_static/.../fred_data.csv at 200 — expected under the settle policy until the next cache+publish cycle, and the distinction is the point: runtime reads were repointed before the blob left main, stale-serving clears later (re-audit on QuantEcon/workspace-lectures#40).

2. Artifact integrity

  • lectures/dataBHS.csv (sha256 13116a3d90ddc7f8b272b3ca903a136552147b21b8e9b3829472daa3d0d09c63) parses back bit-exactly from sources/dataBHS.mat (sha256 28c5f85286718e70b205f6a3fb269ebb49bd635194e2d0d488409b017be5e890) under float_precision='round_trip', and the lecture's 30-bin histogram of c[1:]-c[:-1] has identical counts and edges under pandas' default parser. Use a pandas other than 2.3.3.
  • sources/dataBHS.mat is a real LFS object (pointer oid = the sha256 above), and git check-attr filter -- sources/dataBHS.mat prints lfs.
  • The four lectures/<f> files are byte-identical to the copies the lectures read before migration: check against advanced.myst @ 878b87a^ paths and lecture-python-programming @ 55c87c9^ (git history, not the session's recorded hashes).

3. Highest-value claim: fred_data.csv reproduces from live FRED byte-for-byte

Everything about this file's committed builder status rests on it. Re-run builders/fred_data.py (pinned env per requirements.txt, then once more on a different pandas) and diff against the committed file. The claim's load-bearing details, each falsifiable: DFII5/DFII10 must be fetched with fredgraph's fq=Monthly&fam=avg aggregation (the bare series is daily); fredgraph now serves header observation_date where the file says DATE (the builder renames on read); the window is pinned 1953-04-01..2024-12-01. If the diff is non-empty, check whether FRED revised a value before concluding the builder is wrong — that distinction decides between fixing the builder and re-recording integrity.upstream.

  • Three fresh runs reproduce sha256 45a4fd41aeadf55072ea8e753c12dcb50ccb89bc34fb7eeb5e451ea0796811bc on separate days if possible (the session's three runs were minutes apart — stability over time is the unexercised margin).

4. Records written

  • The four new sidecar manifests parse (PyYAML and one non-Python YAML parser), their schema blocks match the measured bytes (row counts 861/351/236/8, the declared null structure — DFII pair 597 nulls each before 2003-01, zero nulls elsewhere), and each integrity.sha256 matches the committed file.
  • lectures/test_pwt.csv.yml's central negative claim: the committed values match no downloadable PWT vintage — re-derive for at least PWT 7.0 (pwt70_06032011version.zip member pwt70_w_country_names.csv) and PWT 6.3 (pwt63_nov182009version.zip): Argentina/Australia POP match 6.3 but cc/cg do not, and 7.0 differs on tcgdp/cc/cg for every row. The archive sha256s are in the manifest header; re-derive rather than trust.
  • sources/README.md's dataBHS un-refetchability trail: tomsargent.com/source_code.html 404s, larspeterhansen.org lists no code/data for the 2009 JET paper, the article page shows no supplement, and a GitHub-wide code search for dataBHS finds only QuantEcon-descended copies — with NEWQDATA as the search's positive control.

5. Tracker consistency

  • migration.yml parsed (never grep): 37 records, all status: repointed, zero landed/pending/final; the four new records cite Land wave C2 + Track D — the four remaining static datasets #98 and the repoint PRs #375/#612 with date 2026-08-18.
  • Strict audit green on data-lectures main against current clones, and CATALOG.md regenerates with an empty diff.
  • scripts/audit_annotations.yml no longer contains entries for fred_data.csv, acs_data_summary.csv, test_pwt.csv, or dataBHS.mat — but does still contain the lecture-dp orphan entry for acs_data_summary.csv under committed_unreferenced: (deleting that one would have been a mistake; confirm it survived).
  • builders/README.md's re-derived coverage counts (28 constructed, 18 with builders across 16 files, 10 unrecovered) match a fresh parse of the manifests.

6. Known blind spots

  • The deletion-time org sweep covered 278/278 default branches by Trees API. Its documented margins: non-default branches, fork content, binary contents. Probe at least advanced.myst's branch list for the three deleted basenames (the C1 validation found a .pkl variant on origin/hansen this way).
  • The session claims lecture-python-programming.ml holds test_pwt.csv as an orphan (no pandas.md/polars.md lectures exist there). Confirm from its tree, and confirm the manifest's consumer list (8 entries, three translation repos × two files plus the source repo × two) against your own sweep.
  • The .notebooks mirror self-heal claim: lecture-python-advanced.notebooks/{risk_aversion_or_mistaken_beliefs,five_preferences,match_transport}.ipynb on main each carry exactly one data-lectures/raw/main read and zero refs/heads / loadmat residue.

7. Decisions and mechanisms settled today

  • The prune discriminated on #376's preview: Monday's restored cache (built 2026-08-17 03:20, before the C2 deletion) contained the three files, and the preview 404'd all three while serving risk_aversion_or_mistaken_beliefs.html and eggs_backus.png at 200. Note the preview URL form: only the deploy-id-prefixed https://<id>--lustrous-melomakarona-3ee73e.netlify.app serves content; the deploy-preview-N-- form 404s everything including controls.
  • The five_preferences repoint's shape argument: with a pandas Series, c[1:] - c[:-1] misaligns on index and the figure silently changes; the merged cell uses data[['c']].to_numpy() to keep (236,1). Execute both variants and confirm the Series variant is actually wrong (the session asserted but did not run the broken variant).
  • Translation sync mechanics: the three sync PRs (zh-cn#92, fr#33, fa#153) carry all four URL swaps each with surrounding translations preserved, and the sync opens PRs rather than committing (auto-merge off) — which is why test_pwt.csv deletion is gated and was deliberately not done.

8. Deliberately not done

Where the reasoning lives

AGENTS.md and PLAN.md in this repo (URL forms, sources/ rules, repoint ordering); the work plan and its decisions on QuantEcon/workspace-lectures#45; the PR bodies of #98/#99 here and #374/#375/#376 in advanced.myst and #612 in lecture-python-programming; the re-audit checklist on QuantEcon/workspace-lectures#40.

For the validator: work in a session that did not do this work. Do not use the tool named under "Bias to test for" except where a check explicitly says to run it. Re-derive counts rather than confirming them. Where a check can be run against a surface the original session did not exercise, do that too — the margin beyond the checklist is where regressions hide. Deliver: one comment on this issue with a per-item verdict (confirmed / confirmed with caveat / refuted / not completable, with evidence), a new issue for any regression found (do not bury findings in the comment), and leave the checkboxes to the issue owner unless told otherwise.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions