From 20966f3c5bac85bd3c985edb495ff5684deae5db Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Mon, 10 Aug 2026 16:47:11 +1000 Subject: [PATCH 1/3] Add PLAN-QELD-PACKAGE.md: the consumer-side data package, re-scoped MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Distils a design session into an operative plan. The package is a URL resolver, not a fetch-and-cache library: `qeld.url('mpd2020.xlsx')` returns a context-aware URL and the lecture reads it with pandas, in the open. What changed from the first design pass: - The version pins the key/URL contract, not bytes. CalVer retired for semver; `>=1,<2` in install cells, `==` in env files. The `qbn==1.6` precedent does not transfer: quantecon_book_networks ships its data inside the wheel, so its pin is immortal because nothing is fetched. qeld inverts that. - No fetch(), load(), cache, or runtime integrity. After the format convention is applied, `dataBHS.mat` is the only file in the endgame that cannot be read from a URL, which is what justifies dropping fetch(). - Context detection may change transport, never semantics: the CORS-correct URL form, and the emscripten shim that retires pyodide_http.patch_all(). - Integrity moves to CI. The committed-bytes leg already landed in #56; the remaining gap is a live serving-URL check, which belongs post-merge and on a schedule because a PR branch's bytes are never on the serving host. The call-site rule was chosen by measurement, not taste. Every static data read in six repos (115 sites, 40 lectures) was swept and clustered by idiom; the worklist lands as scripts/qeld_callsites.yml. Substituting in place of the URL expression the lecture already uses is the only form that wins on every idiom — inlining destroys reused variables across 47 sites, and always-two-step turns one line into two across 18 inline literals. The plan also records the honest case: 13 sites are structural, 70 are cosmetic shortenings of literals adopted in the recent repoint PRs. The package earns its place on those 13, the wasm shim, the host cutover, and the relative-path portability bugs — not on the read-site tally. Section 9 lists corrections to the original report so they are not re-proposed. Section 8 carries the ambiguities, each tagged with the phase that needs it. Co-Authored-By: Claude Opus 5 (1M context) --- PLAN-QELD-PACKAGE.md | 377 ++++++++++++++++ scripts/qeld_callsites.yml | 874 +++++++++++++++++++++++++++++++++++++ 2 files changed, 1251 insertions(+) create mode 100644 PLAN-QELD-PACKAGE.md create mode 100644 scripts/qeld_callsites.yml diff --git a/PLAN-QELD-PACKAGE.md b/PLAN-QELD-PACKAGE.md new file mode 100644 index 0000000..71122ce --- /dev/null +++ b/PLAN-QELD-PACKAGE.md @@ -0,0 +1,377 @@ +# PLAN — `qeld`, the consumer-side data package + +**Status:** design settled, nothing implemented · **Last updated:** 2026-08-10 +**Relationship to `PLAN.md`:** that document migrates *bytes* into this repo. This one gives *consumers* a +stable way to read them. They are independent — the migration completes with or without `qeld` — but the +call-site convention here replaces repoint rules 5–6 for any lecture that adopts it. + +--- + +## 1. What `qeld` is + +A tiny package that hands a lecture a **context-aware URL** for a published dataset. The lecture then reads +it with pandas, in the open: + +```python +url = qeld.url('mpd2020.xlsx') # or inline, see §4.1 +data = pd.read_excel(url, sheet_name='Full data') +``` + +**Purpose: transit assistance and simplified fetching.** Not data management — data history is git's job. + +It exists to solve three concrete problems: + +1. **The host is spelled six ways.** The same lecture reads the same file via + `github.com/…/raw/main/…` in `lecture-python-intro` and `raw.githubusercontent.com/…` in `lecture-wasm`, + because the first fails CORS in the browser. That split *is* repoint rules 5–6. `qeld.url()` erases it. +2. **`pyodide_http.patch_all()` is in every wasm lecture.** Importing `qeld` under emscripten installs the + transport shim, and those two lines leave the lectures. +3. **The host cutover.** When `data.quantecon.org` lands (#37, #15), it is one constant in one package + rather than an edit in every lecture. + +**What it is not:** a cache, a fetcher, a loader, a data-version manager, or an integrity client. See §3.1. + +--- + +## 2. Decisions + +D1–D5 were taken 2026-08-10 against the original design report. D2, D3 and D5 were **revised the same day** +in a working session that re-scoped the package from fetch-and-cache to URL-resolver; D1 and D4 stand +unchanged. D6–D10 are new. + +| # | decision | status | +|---|---|---| +| **D1** | Name `qeld`. Free on PyPI (verified 2026-08-10; `quantecon-data` also free, `qeds` is taken and still installable at 0.7.0) | unchanged | +| **D2** | **Call style: `qeld.url()` substituted in place of the URL expression the lecture already uses.** No `fetch()`, no `load()` | **revised** — see §4.1 | +| **D3** | **Pin policy: semver, `qeld>=1,<2` in install cells; `==` in env files if lockfile determinism is wanted.** CalVer retired | **revised** — see §3.4 | +| **D4** | Home: in-repo `packages/qeld/`. Bytes, manifest and package in one commit | unchanged | +| **D5** | v1 scope: `url()` and `info()` only. Non-goals: live APIs (#26), Julia, datascience/networks, mirrors, a MyST provenance directive | **narrowed** | +| **D6** | **Context detection may change transport; never semantics** | new — §3.2 | +| **D7** | **Catalog bundled in the wheel, advisory not authoritative** | new — §3.3 | +| **D8** | **Data format: tier 1 binds at intake, tier 2 is a forward-looking preference** | new — §4.2 | +| **D9** | **Integrity lives in CI, not at the call site** | new — §6 | +| **D10** | **Rollout is ordered by win, not by migration track** | new — §7 | + +**The scoping principle throughout:** ship the minimum; add sophistication as demand requires. + +### 2.1 Why D3 changed — the `qbn` precedent does not transfer + +The original rationale for exact CalVer pins was "the `qbn==1.6` house pattern for data packages". It fails +on inspection, and the reasoning is recorded here so it is not re-proposed: + +- `quantecon_book_networks` ships its data **inside the wheel** (`data.py:1-16`, `importlib.resources`). Its + pin is immortal because nothing is fetched. `qeld` inverts that exactly. +- As a house pattern it does not generalise: of `lecture-python-intro`'s 16 install cells, the **only** exact + pins are the two `qbn` cells; `quantecon` itself is unpinned in five. `lecture-wasm` pins nothing. +- "The pin names the data vintage" is a category error. The serving tree holds one version per filename, so + a version pins the **key → URL contract**, never the bytes. + +**What `<2` protects, and therefore what the major version means:** catalog keys are append-only, and a +key's meaning never changes within a major version. + +--- + +## 3. Design + +### 3.1 API surface — two functions + +| call | returns | notes | +|---|---|---| +| `qeld.url(name)` | `str` | the context-correct URL. Unknown key → **warning with near-key suggestions, still returns a URL** | +| `qeld.info(name)` | provenance record | title, source, licence, citation, `citation_policy`, `redistribution`, `sha256`. Offline, from the bundled catalog | + +`info()` earns its place because Maddison's `citation_policy` is a live obligation a lecture must discharge, +and the manifest already carries it. + +**No `fetch()`, `load()`, `open()`, cache, or path.** The full static corpus is reachable with `url()` alone +once the format convention (§4.2) is applied — see §5.3. + +### 3.2 Context detection (D6) + +**The rule: context may change transport; it may never change semantics.** Same key, same bytes, same result +everywhere. A design where context changes *what you get* produces "works in my notebook, fails in CI". + +Exactly two jobs: + +1. **URL form** — `raw.githubusercontent.com` under Pyodide (the `github.com/…/raw/` form 302s and fails + CORS); the 302-tolerant form elsewhere; `data.quantecon.org` for both after #37. +2. **Browser transport shim** — under emscripten, `import qeld` installs the fetch shim, because the right + URL is still not enough: `pd.read_excel(url)` goes through urllib, which fails in the browser. + +Explicitly **not** detection's job: caching, fallback hosts, graceful degradation on fetch failure. + +### 3.3 Catalog (D7) + +`url()` does not need a catalog to *resolve* — keys are filenames and the tree is flat, so the URL is +`base + name`. The catalog exists to validate a key and to answer `info()`. + +- **Bundled in the wheel, advisory not authoritative.** Unknown key → warning + near-key suggestions, and the + URL is still returned; the subsequent 404 confirms it. +- **Why bundled:** typo diagnosis is the common failure and `qeld` cannot catch it after the fact — pandas + does the fetch, so a bad key surfaces as a pandas 404 that never mentions `qeld`. Keys carry extensions and + the repo serves both `longprices.xls` and `mpd2020.xlsx`, so `.xls`/`.xlsx` confusion is live. +- **Why advisory:** fail-open means a dataset that landed after your wheel still works. **No new dataset ever + requires a package release.** +- **One compiler, three outputs.** `scripts/build_catalog.py` already emits `CATALOG.md`; it gains + `catalog.json` for the wheel, sharing a freshness gate. This is the CI that PLAN Phase 2 promised. +- **Rejected, on the record:** a catalog-free package (`url()` = `base + name`, `info()` fetching the served + sidecar — the sidecars *are* served, `text/yaml`, `acao: *`). Attractive because the package would only + change when its code changes, but it gives up the one diagnostic that matters day to day. + +### 3.4 Versioning and pinning (D3) + +- **Semver**, frozen at 1.0 after the first pilots pass. +- **Install cells: `qeld>=1,<2`.** Install cells resolve fresh every session (Colab, Binder, wasm/piplite), + so a corrected release is picked up automatically. +- **Env files: `==` if wanted.** These are resolved once and cached, so a maintainer is present to act on a + break. +- **A byte correction behaves as `AGENTS.md` already intends:** same filename, same URL, every consumer gets + the fix, nothing breaks. + +### 3.5 Zero runtime dependencies + +Pure `py3-none-any` wheel, stdlib only, so one artifact serves CPython, Colab and Pyodide via +micropip/piplite. `info()` reads the bundled catalog with `json` — no YAML parser at runtime. + +--- + +## 4. Conventions this creates + +### 4.1 The call-site rule (D2) — substitute in place + +> Lecture code reads published data through `qeld.url('')`, substituted **in place of the URL +> expression the lecture already uses** — an inline literal becomes an inline call, an assigned variable +> keeps its assignment. The reader (`pd.read_*`, `pl.read_*`, …) and all its kwargs stay visible and +> unchanged. + +Chosen by testing three candidates against every idiom in the corpus (§5). Inline-always destroys reused +variables across 47 sites; two-step-always turns one line into two across 18 inline literals. Substitute-in- +place is the only form that wins everywhere, and it has the property that matters for a ~78-site sweep: + +**The diff is always and only the URL expression**, so a reviewer verifies a migration PR by reading the +changed lines alone. It satisfies the maintainer's bar — *"no more complex than it currently is just fetching +a url"* — by construction, since the lecture's shape is unchanged. + +The install cell is **accepted cost**, not added complexity. This answers #58's stated objection by decision +rather than by design. + +**Carve-outs — do not convert:** + +| carve-out | why | sites | +|---|---|---| +| Live parameterised API endpoints | `url()` keys bare filenames and cannot express a query string; in `pandas.md` the raw URL is the *subject* of a teaching section | 4 | +| Reads whose literal URL **is the lesson** | `pandas_panel` prose reads *"The dataset can be accessed with the following link:"* then shows it. All three files already resolve to data-lectures in the form `qeld` emits, so there is no gain to offset it | 3 | +| Reads whose file is also named by a prose/`{download}` link | `url()` cannot appear in markdown; converting the code alone leaves two spellings that drift | 3, until §8.2 | + +### 4.2 Data format (D8) + +> **Tier 1 (binding, at intake):** a published dataset ships in a format its consumer can read **directly +> from a URL**. +> **Tier 2 (preference, forward-looking):** where the format is *not itself part of what is being taught*, +> prefer text. + +Tier 1 binds — stricter than the house pattern elsewhere ("licensing does not gate migration"), deliberately: +a missing licence field is a gap you record, whereas a non-URL-readable format produces a lecture a reader +cannot run. Every file failing tier 1 today is read from a **bare local path** — the Colab breakage the +programme exists to retire. Enforceable from the catalog alone: assert every catalogued extension is on the +readable list. + +Tier 2's test is *"is the format the lesson?"*, not *"does the format carry information?"* — `.dta` carries +no information beyond its values, but `pd.read_stata` **is** the lesson in `ols.md`, so it is never +converted. + +### 4.3 CSV conversion is lossless in the file, not on read + +Verified 2026-08-10. `to_csv` writes shortest-round-trip repr, but **pandas' default CSV parser is fast, not +correctly rounded.** On `dataBHS.mat` → CSV: default `float_precision='high'` differs in 18 of 708 values +(max relative error 2.1e-16); `'legacy'` differs in 164; `'round_trip'` is bit-exact. + +So any binary→text conversion must **state its read**, and its gate must run `np.array_equal` **under the +reader the lecture will actually use**. 2.1e-16 is immaterial to a figure, but PLAN rule 4's discipline is +"provably cannot change output", and this is the difference between provable and negligible. + +--- + +## 5. The call-site audit — input to the package + +**Snapshot: 2026-08-10**, six repos, 40 lectures, 115 read sites. Machine-readable worklist: +`scripts/qeld_callsites.yml`. **Regenerate before the sweep** — `lecture-python-intro` and this repo both +moved during the session that produced it. + +### 5.1 The unit of work is ~78 URL definitions, not 115 reads + +23 of the 115 sites are downstream uses of a variable assigned earlier (`pd.read_excel(data_url, …)`). +`french_rev` reads `dette_url` six times. One `qeld.url()` call serves N reads — which is why the call-site +rule preserves variables. + +| idiom | definitions | notes | +|---|---:|---| +| A — split string literal across lines | 23 | the clearest win; `polars.md` carries the same 3-line literal twice | +| B — `url` variable = single literal | 32 | one-line RHS swap | +| C — inline literal inside the read | 18 | `ols.md` alone has five 150-char `read_stata` literals | +| D — relative / local path | 5 | **portability bugs today** — these break in Colab | +| F — `requests`+`BytesIO` → `np.load` | 4 | resolved by §4.2, not by the package | +| G — live API endpoint | 4 | carve-out | +| H — `scipy.io.loadmat` | 1 | `dataBHS.mat`, see §8.4 | + +### 5.2 The honest case for the package + +Classified in the worklist: **13 sites are structural** — `qeld` deletes real logic — of which **6 are +convertible today** and the rest are blocked on the format decision (§8.3, §8.4). **70 are cosmetic** +shortenings of literals adopted in the *recent repoint PRs*; a plain one-line literal would win those back +without a package. 32 are neutral or excluded. + +**The case rests on:** those ~10 structural sites, the wasm shim, the host-cutover property, and group D's +portability bugs. **Not** on the read-site tally. Anyone re-reading this plan should weigh it on that basis. + +The best diff in the corpus is `lecture-python-advanced.myst/lectures/hansen_jagannathan_1991.md:182`, where +`qeld` deletes a hand-rolled version of itself — a 3-fragment URL literal, a local-vs-remote branch and a +`Path("lectures") / url` fallback, 16 lines to 3, plus an unused `pathlib` import. + +### 5.3 `url()` alone covers the corpus + +After the `.npy` conversion (§8.3), **`dataBHS.mat` is the only file in the endgame that cannot be read from +a URL**. That is what justifies dropping `fetch()`. + +### 5.4 Two latent bugs found, worth fixing regardless + +- `inequality.md` imports `pyodide_http` and **never calls `patch_all()`**. +- `short_path.md` calls `requests.get` under Pyodide **with no shim at all**. + +Both are incidentally fixed by §3.2's shim, but neither should wait for it. + +### 5.5 Two problems no syntax solves + +**Generic filenames collide in a flat namespace.** `fred_data.csv`, `fp.dta`, `test_pwt.csv`, +`acs_data_summary.csv` all need renaming at migration — and `mle.md:160` names `mle/fp.dta` **in prose**, so +a rename must be paired with a prose edit found by grep, not by the audit. + +**Prose and `{download}` links.** 28 refs; **11 name a data file in two places**. The acute case is +`simple_linear_regression.md` (intro:411/416, zh-cn:421/426), where a `{download}` role and the code read sit +five lines apart containing byte-identical 158-character strings. + +--- + +## 6. Integrity and CI (D9) + +**Baseline, stated honestly:** a bare URL read has no integrity guarantee today either. Git at the source is +the guarantee, and it remains the guarantee. This design **declines to add a property rather than removing +one** — important, so nobody later re-adds `fetch()` to recover something the series never had. + +| leg | status | +|---|---| +| Committed bytes vs `integrity.sha256`, every manifested file | ✅ **already landed** (#56) — hashes whenever a manifest records a hash, with or without consumers | +| URL *spelling* assertions (`ref == main`, path shape, media host, CORS form) | ✅ **already landed** (#55, #48, #47) — static, from the parsed URL | +| **Live serving-URL fetch vs the manifest hash** | ❌ **the remaining gap** — see below | +| Catalog compile + freshness, shared with `CATALOG.md` | ❌ to build (§3.3) | +| Format tier-1 assertion over the catalog | ❌ to build (§4.2) | + +**The live leg belongs post-merge and on a schedule, never on PRs.** `audit-dashboard.yml` gates deploy on +`if: github.event_name != 'pull_request'`, so a PR branch's bytes are never on the host: a new-dataset PR +would 404 and a correction PR would compare old served bytes against a new manifest hash. Both fail by +construction — and those are exactly the PR classes such a gate would exist for. + +This leg catches what the committed-bytes check structurally cannot: an LFS pointer served instead of +content, a Pages misconfiguration, a stale CDN, a 404. It opens an issue on failure rather than blocking. + +**Recorded gap:** a reader running a downloaded notebook years later against a drifted host gets no warning +in their own session. That is **also true today**; the scheduled leg catches drift centrally, so protection +is indirect. Net position after this work: strictly better than today, short of a hashing client. + +--- + +## 7. Development plan + +Ordering is forced by one constraint: **the audit must learn the call form before any consumer adopts it**, +or every migrated read classifies `local-path` and the dashboard inverts. + +| phase | work | gate | +|---|---|---| +| **Q1 — Audit first** | `build_audit.py` learns `qeld.url('X')` → pattern `qeld`, counted migrated **and terminal**. For `pattern == 'qeld'`, assert the key exists in `lectures/` and is not deprecated — otherwise the qeld path loses every assertion #55/#48/#47 added. `migration.yml`: `final` := canonical-host *or* qeld | `audit.json` `stats` and `problems` unchanged on today's repos (**not** "byte-identical" — the audit stamps `date.today()`) | +| **Q2 — Schema hygiene** | Document `read_as` (used in 6 manifests) and `sheets` (8) in `manifest-schema.yml` — both are in use and neither appears in the file `AGENTS.md` calls "the authoritative, commented field reference". Add `deprecated:` (new, used nowhere yet) since §3.3 warns on it. `shape` is already documented. Delete `then: "iloc[1:]"` from `longprices.xls.yml:70` by moving `iloc[1:]` into the lecture — a post-read transform encoded as a string to evaluate is exactly what D5 excludes | `manifest-schema.yml` covers every field any manifest uses. Needs none of #14's decisions — do not block on it | +| **Q3 — Package** | `packages/qeld/`: `url()`, `info()`, context detection, advisory catalog. Catalog compiler shares a freshness gate with `CATALOG.md`. Format tier-1 assertion. First release to PyPI via trusted publishing | Offline suite green on every PR: catalog compiles and is fresh; unknown key warns and still returns a URL; URL form correct per detected context; suffix fidelity incl. `.csv.gz`; `info()` fields present. CPython matrix | +| **Q4 — Live leg** | Post-merge + scheduled job: fetch each served URL, compare to the manifest hash, open an issue on failure | Green on `main`; an induced failure opens an issue | +| **Q5 — Browser session** | `%pip install qeld==` in a real `lecture-wasm` page (**`%pip` routes through piplite, not micropip** — a console `micropip.install` is a false pass); `pd.read_excel(qeld.url('mpd2020.xlsx'), sheet_name='Regional data', header=[0,1,2], index_col=0)`; a `.csv.gz` read; record observed Pyodide and pyodide-kernel versions | Written pass/fail. Fail ⇒ wasm keeps URLs and the plan proceeds for the CPython repos | +| **Q6 — Pilots, by win** | `hansen_jagannathan_1991` (deletes the dual-path loader — the best diff); `french_rev` (the `base_url` block, intro **and** wasm together per repoint rule 2); `subjective_beliefs_business_cycles` or `match_transport` (group D, a real portability bug) | Diff is the URL expression only; intro figure-hash equality; wasm = re-run Q5's checklist on the pilot page with the console transcript filed. On pass: tag 1.0, freeze the API | +| **Q7 — Sweep by win** | The remaining structural sites, then cosmetic sites at leisure or never. `AGENTS.md` gains §4.1 verbatim, §4.2, and the note that repoint rules 5–6 do not apply to qeld call sites | `migrated` meter reflects qeld refs; no lecture regresses | + +**Effort note:** Q3 is not a weekend. The package is small, but the catalog compiler, the tier-1 assertion, +the offline suite and PyPI trusted-publisher configuration are four separate pieces, and the last is not a +code task. + +**Pyodide version pinning is accidental** — the version ships inside `thebe-lite.min.js` inside the theme +zip, so a CSS-only theme bump moves the Python runtime. Record Q5's verdict as "Pyodide 0.27 as shipped by +thebe-lite in quantecon-theme v2.1.0" and re-run when the theme moves. + +--- + +## 8. Ambiguities — decide before the phase that needs them + +### 8.1 `pandas_panel`'s carve-out — permanent or revisited? + +The prose advertises the URL as something a beginner can paste into a browser. Converting is a pedagogical +loss with no offsetting gain (those files already resolve to data-lectures in the form `qeld` emits). The +alternative is rewriting the prose, which is a larger edit than the one being justified. **Needed by Q7.** + +### 8.2 The markdown-time story for `{download}` links + +`qeld.url()` cannot appear in markdown. Options: convert both and accept two spellings; leave those lectures +alone; or give `qeld` a markdown-time story (a MyST substitution, or point prose at `CATALOG.md`). Until +answered, "convert neither" is the default. **Needed by Q7.** + +### 8.3 The `.npy` pair — now a breaking change + +`caron.npy` and `nom_balances.npy` are (63, 2) and (81, 2) float64 arrays whose manifests already name the +columns (`date`/`specie_value`, `date`/`nominal_balances`). Converting to CSV deletes `requests`, `BytesIO` +and two imports from `french_rev` in every consuming repo. + +**But the window closed.** When this was analysed both files had `consumers: []`; the A3 set has since been +repointed (#49) and both now have two consumers. So this is no longer a free replacement — it needs the +`AGENTS.md` "new vintage → new filename" treatment (`caron.csv` lands alongside, consumers opt in, the `.npy` +is swept later) or a coordinated set under repoint rules 1–3. **Decide before Q6**, since `french_rev` is a +pilot. + +### 8.4 `dataBHS.mat` — convert at migration, or exclude? + +5,588 bytes; `c`, `rb`, `rs`, each (236, 1) float64; the lecture uses only `data['c']` and the read is inside +a `hide-input` cell, so nothing about it is taught. Trivially a 236×3 CSV — but see §4.3 on the read. A Track +C decision; the only true impossibility among static files. + +### 8.5 Is `lecture-intro.zh-cn` in scope? + +It carries data reads, appears in **zero** `consumers` blocks, is excluded from `SCAN_REPOS` by decision, has +no data CI, publishes on a `publish*` tag, and inherits install cells automatically via the `.md`-only sync — +so it acquires whatever intro acquires without anyone deciding. It also has files with no data-lectures key +and no business having one (`country_code_cn.csv`, a translation asset). +**Recommendation: explicit non-goal for v1, with one fixed rule instead of machinery — any sweep touching an +intro file also touches zh-cn.** + +### 8.6 The rename list for generic filenames + +§5.5. Needs a pass before Tracks B and C migrate, and each rename needs its prose pairing found by grep. + +### 8.7 Open from the original report + +- Ask Spencer/Tom what actually retired `qeds` — it is dead as a project but **still installable** (PyPI + returns 200 for 0.7.0, not yanked), one letter-transposition from `qeld`. Consider reserving + `quantecon-data` and `qedata` as stubs. +- `data.quantecon.org` DNS (#37) is the nearest unblocked item and races this work for the same weekend. +- Publishing `qeld` does **not** move #35's licensing gate — it rehosts nothing and fetches the same public + URLs `pandas_panel` reads today. Do not add a release check that fails on `redistribution: restricted`; + `countries.csv` is restricted and unresolved, so it would block every release from day one. + +--- + +## 9. Corrections to the original design report + +Recorded so they are not re-proposed. The report is superseded by this document. + +| claim | status | +|---|---| +| "18/18 manifests", "all 18, not a sample" | stale before the ink dried — 24 manifests, 27 files on `main` | +| Exact CalVer pins, "the install cell names the data vintage" | a version cannot pin bytes — §2.1 | +| "CI fails with `IntegrityError`; that failure *is* the rebuild signal" | wasm CI does not execute; intro is `execute_notebooks: "cache"` and a data correction changes no code, so PRs are cache hits | +| "no lecture-content edits, ever" | the install cell is lecture content, and prose data URLs already exist | +| "six different URL ways" | the audit says five in use | +| "closes #8" | #8's catalog box was already ticked; this adds the freshness CI | +| A `~200-line prototype` and an evidence document | neither exists on disk; `git log --all` has zero hits for `qeld`. Both load-bearing prototype claims re-derive from the manifests | diff --git a/scripts/qeld_callsites.yml b/scripts/qeld_callsites.yml new file mode 100644 index 0000000..e30fc78 --- /dev/null +++ b/scripts/qeld_callsites.yml @@ -0,0 +1,874 @@ +# qeld call-site worklist — audit snapshot +# +# Generated 2026-08-10 from a source sweep of six lecture repos (40 lectures, +# 115 read sites). Input to PLAN-QELD-PACKAGE.md; see that file for the +# call-site rule (section 4.1), the carve-outs, and the rollout ordering. +# +# THIS IS A SNAPSHOT, NOT A LIVE VIEW. lecture-python-intro and this repo both +# moved during the session that produced it (the A3 french_rev set was +# repointed in #49). REGENERATE BEFORE THE SWEEP. +# +# action: +# convert apply the substitute-in-place rule +# carve-out do not convert - see the note +# blocked needs a format decision first +# exclude not a data-lectures candidate at all +# downstream a read of a variable assigned elsewhere; changes by construction +# +# win: indicative only. `structural` means qeld deletes logic, not just shortens +# a literal. The rollout takes structural sites first. + +callsites: + - repo: "QuantEcon" + lecture: "lectures/risk_aversion_or_mistaken_beliefs.md:1613" + filename: "fred_data.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "lecture-python-programming" + lecture: "lectures/polars.md:160-163" + filename: "test_pwt.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "lecture-python-programming" + lecture: "lectures/polars.md:346-349" + filename: "test_pwt.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "lecture-python-programming" + lecture: "lectures/polars.md:424-428" + filename: "test_pwt.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "lecture-python.myst" + lecture: "lectures/phillips_drifts_volatilities.md:102" + filename: "NEWQDATA.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/fitting_distributions.md:77-79" + filename: "ames_house_prices.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/fitting_distributions.md:257-259" + filename: "us_adult_heights.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/fitting_distributions.md:467-469" + filename: "epl_match_goals.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/fitting_distributions.md:667-669" + filename: "japan_earthquakes.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/fitting_distributions.md:753-755" + filename: "japan_deaths_by_age.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/observed_distributions.md:137-139" + filename: "us_adult_heights.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/observed_distributions.md:152-154" + filename: "ames_house_prices.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/observed_distributions.md:187-189" + filename: "japan_deaths_by_age.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/prob_dist.md:57" + filename: "us_adult_heights.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/prob_dist.md:130" + filename: "japan_population_by_age.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/observed_distributions.md:133" + filename: "us_adult_heights.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/observed_distributions.md:151" + filename: "ames_house_prices.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/observed_distributions.md:192" + filename: "japan_deaths_by_age.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/fitting_distributions.md:72" + filename: "ames_house_prices.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/fitting_distributions.md:277" + filename: "us_adult_heights.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/fitting_distributions.md:522" + filename: "epl_match_goals.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/fitting_distributions.md:757" + filename: "japan_earthquakes.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/fitting_distributions.md:855" + filename: "japan_deaths_by_age.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "lecture-python-programming" + lecture: "lectures/pandas_panel.md:78,90" + filename: "realwage.csv" + idiom: "B-url-variable" + verdict: "same" + action: "carve-out" + win: "cosmetic" + note: "the literal URL is the lesson - see PLAN section 8.1" + - repo: "lecture-python-programming" + lecture: "lectures/pandas_panel.md:198,202" + filename: "countries.csv" + idiom: "B-url-variable" + verdict: "same" + action: "carve-out" + win: "cosmetic" + note: "the literal URL is the lesson - see PLAN section 8.1" + - repo: "lecture-python-programming" + lecture: "lectures/pandas_panel.md:507,526" + filename: "employ.csv" + idiom: "B-url-variable" + verdict: "same" + action: "carve-out" + win: "cosmetic" + note: "the literal URL is the lesson - see PLAN section 8.1" + - repo: "lecture-python.myst" + lecture: "lectures/pandas_panel.md:68" + filename: "realwage.csv" + idiom: "B-url-variable" + verdict: "more_complex" + action: "carve-out" + win: "cosmetic" + note: "the literal URL is the lesson - see PLAN section 8.1" + - repo: "lecture-python.myst" + lecture: "lectures/pandas_panel.md:188" + filename: "countries.csv" + idiom: "B-url-variable" + verdict: "more_complex" + action: "carve-out" + win: "cosmetic" + note: "the literal URL is the lesson - see PLAN section 8.1" + - repo: "lecture-python.myst" + lecture: "lectures/pandas_panel.md:505" + filename: "employ.csv" + idiom: "B-url-variable" + verdict: "more_complex" + action: "carve-out" + win: "cosmetic" + note: "the literal URL is the lesson - see PLAN section 8.1" + - repo: "QuantEcon" + lecture: "lectures/long_run_growth.md:96-99" + filename: "mpd2020.xlsx" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/inflation_history.md:75-80" + filename: "longprices.xls" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/inflation_history.md:350-351" + filename: "chapter_3.xlsx" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:80-85" + filename: "fig_3.xlsx, dette.xlsx, assignat.xlsx" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "structural" + - repo: "QuantEcon" + lecture: "lectures/simple_linear_regression.md:426-427" + filename: "life-expectancy-vs-gdp-per-capita.csv" + idiom: "B-url-variable" + verdict: "same" + action: "carve-out" + win: "cosmetic" + note: "paired with a download link - see PLAN section 8.2" + - repo: "QuantEcon" + lecture: "lectures/mle.md:95-96" + filename: "SCF_plus_mini_no_weights.csv" + idiom: "B-url-variable" + verdict: "same" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/short_path.md:274-277" + filename: "graph.txt" + idiom: "B-url-variable" + verdict: "same" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/inequality.md:250-251" + filename: "SCF_plus_mini.csv" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/inequality.md:619-620" + filename: "usa-gini-nwealth-tincome-lincome.csv" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/inflation_history.md:102-107" + filename: "longprices.xls" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/inflation_history.md:374-375" + filename: "chapter_3.xlsx" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/long_run_growth.md:106-108" + filename: "mpd2020.xlsx" + idiom: "B-url-variable" + verdict: "more_complex" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/french_rev.md:84-89" + filename: "fig_3.xlsx, dette.xlsx, assignat.xlsx" + idiom: "B-url-variable" + verdict: "same" + action: "convert" + win: "structural" + - repo: "" + lecture: "lectures/inequality.md:256-257" + filename: "SCF_plus_mini.csv" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/inequality.md:613-614" + filename: "usa-gini-nwealth-tincome-lincome.csv" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/mle.md:105-106" + filename: "SCF_plus_mini_no_weights.csv" + idiom: "B-url-variable" + verdict: "same" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/simple_linear_regression.md:426-427" + filename: "life-expectancy-vs-gdp-per-capita.csv" + idiom: "B-url-variable" + verdict: "more_complex" + action: "carve-out" + win: "cosmetic" + note: "paired with a download link - see PLAN section 8.2" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:75" + filename: "fig_3.xlsx" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "structural" + - repo: "QuantEcon" + lecture: "lectures/long_run_growth.md:89" + filename: "mpd2020.xlsx" + idiom: "B-url-variable" + verdict: "same" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/inflation_history.md:86" + filename: "longprices.xls" + idiom: "B-url-variable" + verdict: "same" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/inflation_history.md:358" + filename: "chapter_3.xlsx" + idiom: "B-url-variable" + verdict: "same" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/msy_fishery.md:596" + filename: "lingcod_msy_recovery.csv" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/mle.md:93" + filename: "SCF_plus_mini_no_weights.csv" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/inequality.md:249" + filename: "SCF_plus_mini.csv" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/inequality.md:621" + filename: "usa-gini-nwealth-tincome-lincome.csv" + idiom: "B-url-variable" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/simple_linear_regression.md:416" + filename: "life-expectancy-vs-gdp-per-capita.csv" + idiom: "B-url-variable" + verdict: "more_complex" + action: "carve-out" + win: "cosmetic" + note: "paired with a download link - see PLAN section 8.2" + - repo: "lecture-python-programming" + lecture: "lectures/pandas.md:165" + filename: "test_pwt.csv" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "lecture-python.myst" + lecture: "lectures/ols.md:94" + filename: "maketable1.dta" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "lecture-python.myst" + lecture: "lectures/ols.md:326" + filename: "maketable2.dta" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "lecture-python.myst" + lecture: "lectures/ols.md:478" + filename: "maketable4.dta" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "lecture-python.myst" + lecture: "lectures/ols.md:606" + filename: "maketable4.dta" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "lecture-python.myst" + lecture: "lectures/ols.md:679" + filename: "maketable1.dta" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "lecture-python.myst" + lecture: "lectures/mle.md:165" + filename: "fp.dta" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/heavy_tails.md:827" + filename: "forbes-global2000.csv" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/heavy_tails.md:854" + filename: "cities_us.csv" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/heavy_tails.md:855" + filename: "cities_brazil.csv" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/heavy_tails.md:879" + filename: "forbes-billionaires.csv" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/heavy_tails.md:811" + filename: "forbes-global2000.csv" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/heavy_tails.md:838-839" + filename: "cities_us.csv, cities_brazil.csv" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "" + lecture: "lectures/heavy_tails.md:863" + filename: "forbes-billionaires.csv" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/heavy_tails.md:827" + filename: "forbes-global2000.csv" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/heavy_tails.md:854" + filename: "cities_us.csv" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/heavy_tails.md:855" + filename: "cities_brazil.csv" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/heavy_tails.md:879" + filename: "forbes-billionaires.csv" + idiom: "C-inline-literal" + verdict: "simpler" + action: "convert" + win: "cosmetic" + - repo: "QuantEcon" + lecture: "lectures/subjective_beliefs_business_cycles.md:149" + filename: "bbh_macro_quarterly.csv" + idiom: "D-local-path" + verdict: "same" + action: "convert" + win: "structural" + - repo: "QuantEcon" + lecture: "lectures/subjective_beliefs_business_cycles.md:152" + filename: "bbh_michigan_monthly.csv" + idiom: "D-local-path" + verdict: "simpler" + action: "convert" + win: "structural" + - repo: "QuantEcon" + lecture: "lectures/match_transport.md:2234" + filename: "acs_data_summary.csv" + idiom: "D-local-path" + verdict: "simpler" + action: "convert" + win: "structural" + - repo: "" + lecture: "lectures/long_run_growth.md:141" + filename: "country_code_cn.csv" + idiom: "D-local-path" + verdict: "impossible" + action: "exclude" + win: "structural" + note: "zh-cn translation asset; no data-lectures key" + - repo: "" + lecture: "lectures/business_cycle.md:62" + filename: "country_code_cn.csv" + idiom: "D-local-path" + verdict: "impossible" + action: "exclude" + win: "structural" + note: "zh-cn translation asset; no data-lectures key" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:725-728" + filename: "caron.npy, nom_balances.npy" + idiom: "F-npy-bytesio" + verdict: "same" + action: "blocked" + win: "structural" + note: "npy cannot be read from a URL - see PLAN section 8.3" + - repo: "" + lecture: "lectures/french_rev.md:671-674" + filename: "caron.npy, nom_balances.npy" + idiom: "F-npy-bytesio" + verdict: "same" + action: "blocked" + win: "structural" + note: "npy cannot be read from a URL - see PLAN section 8.3" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:711" + filename: "caron.npy" + idiom: "F-npy-bytesio" + verdict: "same" + action: "blocked" + win: "structural" + note: "npy cannot be read from a URL - see PLAN section 8.3" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:712" + filename: "nom_balances.npy" + idiom: "F-npy-bytesio" + verdict: "same" + action: "blocked" + win: "structural" + note: "npy cannot be read from a URL - see PLAN section 8.3" + - repo: "lecture-python-programming" + lecture: "lectures/pandas.md:514" + filename: "fredgraph.csv (FRED query endpoint)" + idiom: "G-live-api" + verdict: "impossible" + action: "exclude" + win: "none" + note: "live parameterised API endpoint, not a file" + - repo: "lecture-python-programming" + lecture: "lectures/pandas.md:532,552" + filename: "fredgraph.csv (FRED query endpoint)" + idiom: "G-live-api" + verdict: "impossible" + action: "exclude" + win: "none" + note: "live parameterised API endpoint, not a file" + - repo: "lecture-python-programming" + lecture: "lectures/polars.md:557-572" + filename: "fredgraph.csv (FRED query endpoint)" + idiom: "G-live-api" + verdict: "impossible" + action: "exclude" + win: "none" + note: "live parameterised API endpoint, not a file" + - repo: "lecture-python.myst" + lecture: "lectures/phillips_drifts_volatilities.md:2213" + filename: "fredgraph.csv (FRED query endpoint, not a file)" + idiom: "G-live-api" + verdict: "impossible" + action: "exclude" + win: "none" + note: "live parameterised API endpoint, not a file" + - repo: "QuantEcon" + lecture: "lectures/five_preferences.md:1848" + filename: "dataBHS.mat" + idiom: "H-loadmat" + verdict: "impossible" + action: "blocked" + win: "none" + note: "scipy.io.loadmat cannot take a URL - see PLAN section 8.4" + - repo: "QuantEcon" + lecture: "lectures/hansen_jagannathan_1991.md:182" + filename: "hansen_jagannathan_1991_data.json" + idiom: "I-generated" + verdict: "simpler" + action: "exclude" + win: "structural" + note: "file is generated by the lecture itself" + - repo: "lecture-python.myst" + lecture: "lectures/hansen_singleton_1982.md:993" + filename: "hansen_singleton_1982_data.csv" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "lecture-python.myst" + lecture: "lectures/hansen_singleton_1983.md:1432" + filename: "hansen_singleton_1983_data.csv" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/long_run_growth.md:488" + filename: "mpd2020.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/long_run_growth.md:593-596" + filename: "mpd2020.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/inflation_history.md:374-375" + filename: "chapter_3.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:106-108" + filename: "dette.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:111-113" + filename: "dette.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:155-156" + filename: "dette.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:233-234" + filename: "dette.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:235-236" + filename: "dette.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:276-277" + filename: "fig_3.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:438-439" + filename: "dette.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:475-476" + filename: "assignat.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:522-523" + filename: "assignat.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:586-587" + filename: "assignat.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:588-589" + filename: "assignat.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:656-657" + filename: "assignat.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:658-659" + filename: "assignat.xlsx" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/simple_linear_regression.md:444" + filename: "life-expectancy-vs-gdp-per-capita.csv" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "" + lecture: "lectures/prob_dist.md:74-77" + filename: "us_adult_heights.csv" + idiom: "Z-downstream-read" + verdict: "simpler" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "" + lecture: "lectures/prob_dist.md:152-159" + filename: "japan_population_by_age.xlsx (data-lectures key: japan_population_by_age.csv)" + idiom: "Z-downstream-read" + verdict: "simpler" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "" + lecture: "lectures/long_run_growth.md:486" + filename: "mpd2020.xlsx (reuse)" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "" + lecture: "lectures/long_run_growth.md:589-592" + filename: "mpd2020.xlsx (reuse)" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "" + lecture: "lectures/french_rev.md:110,115,158,231,233,413,447,492,555,557,615,617" + filename: "dette.xlsx / assignat.xlsx / fig_3.xlsx (12 reuses)" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "" + lecture: "lectures/simple_linear_regression.md:444" + filename: "life-expectancy-vs-gdp-per-capita.csv (reuse)" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:76" + filename: "dette.xlsx" + idiom: "Z-downstream-read" + verdict: "simpler" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon" + lecture: "lectures/french_rev.md:77" + filename: "assignat.xlsx" + idiom: "Z-downstream-read" + verdict: "simpler" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" From 137e1f3b4ba9e9ac580d1e73386b9586e0234252 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Mon, 10 Aug 2026 16:53:50 +1000 Subject: [PATCH 2/3] Move the qeld working artifacts to .dev/qeld/, and add the migration catalog MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `callsites.yml` says *where* to convert; it does not say *how*. The migration catalog carries the worked before/after per idiom and the head-to-head of the three candidate syntaxes, so it is the spec a sweep is checked against. Both are evidence rather than tooling, so they move out of scripts/ into .dev/qeld/ with a README stating what they are, that the plan wins on any disagreement, and that both are 2026-08-10 snapshots which must be re-verified before use — the lecture repos moved during the session that produced them. Nothing here is published: the Pages job assembles _site from site/, lectures/ and audit.json only. Co-Authored-By: Claude Opus 5 (1M context) --- .dev/qeld/README.md | 20 ++ .../qeld/callsites.yml | 0 .dev/qeld/migration-catalog.md | 274 ++++++++++++++++++ PLAN-QELD-PACKAGE.md | 7 +- 4 files changed, 298 insertions(+), 3 deletions(-) create mode 100644 .dev/qeld/README.md rename scripts/qeld_callsites.yml => .dev/qeld/callsites.yml (100%) create mode 100644 .dev/qeld/migration-catalog.md diff --git a/.dev/qeld/README.md b/.dev/qeld/README.md new file mode 100644 index 0000000..2bde288 --- /dev/null +++ b/.dev/qeld/README.md @@ -0,0 +1,20 @@ +# `.dev/qeld/` — working artifacts behind `PLAN-QELD-PACKAGE.md` + +Evidence and worklists produced while designing the `qeld` consumer-side data package. These are **inputs to +the plan, not the plan**. Where anything here disagrees with `PLAN-QELD-PACKAGE.md`, the plan wins. + +Nothing here is published — the Pages job assembles `_site` from `site/`, `lectures/` and `audit.json` only. + +| file | what it is | +|---|---| +| `callsites.yml` | The 115-site call-site worklist: every static data read in six lecture repos, with its idiom, the verdict against the "no more complex than fetching a url" bar, the action to take, and whether the win is structural or cosmetic. Drives the rollout ordering (`PLAN-QELD-PACKAGE.md` §7). | +| `migration-catalog.md` | The worked evidence behind the call-site rule — verbatim before/after per idiom, and the head-to-head comparison of the three candidate syntaxes. This is the spec for what a correct conversion produces. | + +## Both are snapshots dated 2026-08-10 + +They were produced from a source sweep on that date, and **both this repo and the lecture repos moved during +the session that produced them**. The most consequential drift: the A3 `french_rev` set was repointed (#49), +so the two `.npy` files gained consumers and their conversion to CSV is no longer a free replacement. + +**Regenerate the worklist before running the sweep.** Treat the catalog's examples as illustrative of the +*pattern*, and re-read the current source before converting any specific site. diff --git a/scripts/qeld_callsites.yml b/.dev/qeld/callsites.yml similarity index 100% rename from scripts/qeld_callsites.yml rename to .dev/qeld/callsites.yml diff --git a/.dev/qeld/migration-catalog.md b/.dev/qeld/migration-catalog.md new file mode 100644 index 0000000..0433208 --- /dev/null +++ b/.dev/qeld/migration-catalog.md @@ -0,0 +1,274 @@ +# qeld migration catalog — how the lectures consume data today, and what each idiom becomes + +**Date:** 2026-08-10 · **Status:** design input, superseded on decisions by `PLAN-QELD-PACKAGE.md`. +**Purpose:** the worked evidence behind the call-site rule — what each conversion actually looks like. +**Method:** every static data read in six lecture repos (40 lectures, 115 read sites) enumerated from source, +clustered by idiom, with verbatim current code. + +> **Snapshot, not a live view.** Both this repo and `lecture-python-intro` moved after this was produced — +> notably the A3 `french_rev` set was repointed in #64's predecessor #49, which changes §3 F's status from a +> free conversion to a breaking one (`PLAN-QELD-PACKAGE.md` §8.3). Re-verify any example before acting on it. + +--- + +## 1. The unit of work is not 115 reads + +115 read *sites*, but **23 of them are downstream uses of a variable assigned earlier** — +`pd.read_excel(data_url, …)` where `data_url` was set once. `french_rev` reads `dette_url` six times; +`long_run_growth` reads `data_url` three times. + +So the real unit is **~78 URL definitions**. One `qeld.url()` call serves N reads. This is the single most +important fact for choosing syntax: a form that preserves the variable converts one line and leaves N reads +untouched; a form that inlines has to touch all N. + +| idiom | definitions | current verdicts | +|---|---:|---| +| **A** — split string literal across lines | 23 | all simpler | +| **B** — `url` variable = single literal | 32 | 15 simpler, 11 same, **all 6 more_complex** | +| **C** — inline literal URL inside the read | 18 | all simpler | +| **D** — relative / local path | 5 | 2 impossible | +| **F** — `requests`+`BytesIO` → `np.load` | 4 | all same | +| **G** — live parameterised API endpoint | 4 | all impossible | +| **H** — `scipy.io.loadmat` | 1 | impossible | +| *(downstream reads of an existing variable)* | *23* | *unchanged by construction* | + +--- + +## 2. The three candidate syntaxes + +- **S1 — inline:** `pd.read_csv(qeld.url('x.csv'))` +- **S2 — two-step always:** `url = qeld.url('x.csv')` then `pd.read_csv(url)` +- **S3 — mirror the existing shape:** replace only the URL expression; inline stays inline, a variable stays + a variable. + +--- + +## 3. The catalog — verbatim, by idiom + +### A. Split string literal (23) — the pattern the design was built for + +Two sub-shapes. **A1, assigned to a variable** (`polars.md:160-163`): + +```python +# CURRENT +url = ('https://raw.githubusercontent.com/QuantEcon/' + 'lecture-python-programming/main/lectures/_static/' + 'lecture_specific/pandas/data/test_pwt.csv') +df = pl.read_csv(url) + +# S2 / S3 (identical here) +url = qeld.url('test_pwt.csv') +df = pl.read_csv(url) +``` + +**A2, split inline inside the read** (`risk_aversion_or_mistaken_beliefs.md:1613`): + +```python +# CURRENT — 5 lines +data = pd.read_csv( + 'https://raw.githubusercontent.com/QuantEcon/lecture-python-advanced.myst/refs/heads/' + 'main/lectures/_static/lecture_specific/risk_aversion_or_mistaken_beliefs/fred_data.csv', + parse_dates=['DATE'], index_col='DATE' +) + +# S1 / S3 — 1 line +data = pd.read_csv(qeld.url('fred_data.csv'), parse_dates=['DATE'], index_col='DATE') +``` + +**Verdict:** the clearest win in the corpus. `polars.md` carries the *same* three-line literal twice +(`:160` and `:346`), which a student has to re-read to confirm it is the same file. + +### B. `url` variable = single literal (32) — where every failure lives + +```python +# CURRENT (pandas_panel.md:68 / :78) +url1 = 'https://github.com/QuantEcon/data-lectures/raw/main/lectures/realwage.csv' +realwage = pd.read_csv(url1) + +# S1 — destroys the variable; every downstream read must change too +realwage = pd.read_csv(qeld.url('realwage.csv')) + +# S2 / S3 — one-line RHS swap, downstream reads untouched +url1 = qeld.url('realwage.csv') +realwage = pd.read_csv(url1) +``` + +**Verdict:** the six `more_complex` gradings in this group were scored against **S1**, which needlessly +destroys a variable the lecture reuses. Under S2/S3 the complaint largely evaporates — the diff is one +line's right-hand side. + +**What survives the fix:** `pandas_panel`'s three reads still lose something real. The prose immediately +above reads *"The dataset can be accessed with the following link:"* followed by a cell containing nothing +but the URL, so a beginner can paste it into a browser. All three files already point at data-lectures in +the exact form qeld emits off-Pyodide, so there is no host-migration gain to offset it. This is a +**pedagogical** objection, not a syntactic one, and no syntax fixes it. *(Note: `pandas_panel` exists in both +`lecture-python.myst` and `lecture-python-programming` — the two sweeps graded the same content differently, +which is why the `more_complex` count is 4–8 rather than exactly 6.)* + +### C. Inline literal URL inside the read (18) — where S2 loses + +```python +# CURRENT (ols.md:94) — a 150-char literal that overflows every page width +df1 = pd.read_stata('https://github.com/QuantEcon/lecture-python.myst/raw/refs/heads/main/lectures/_static/lecture_specific/ols/maketable1.dta') + +# S1 / S3 — 1 line +df1 = pd.read_stata(qeld.url('maketable1.dta')) + +# S2 — 1 line becomes 2, for nothing +url = qeld.url('maketable1.dta') +df1 = pd.read_stata(url) +``` + +**Verdict:** `ols.md` alone carries five of these. The prose above each already names the file +(`maketable2.dta`), so the qeld key matches the words on the page better than the URL does. **S2 is worse +than the status quo here** — this is the group that rules out "always two-step". + +### D. Relative / local path (5) + +```python +# CURRENT (subjective_beliefs_business_cycles.md:149) +data_path = '_static/lecture_specific/subjective_beliefs_business_cycles/' +macro_q = pd.read_csv(data_path + 'bbh_macro_quarterly.csv', index_col='YYYYQ') + +# any syntax +macro_q = pd.read_csv(qeld.url('bbh_macro_quarterly.csv'), index_col='YYYYQ') +``` + +**Verdict:** these relative paths are **portability bugs today** — they break in Colab and in downloaded +notebooks. Two of the five are `country_code_cn.csv`, a zh-cn translation asset with no data-lectures key +and no business having one. + +### F. `requests` + `BytesIO` → `np.load` (4) — resolved by the format decision, not by syntax + +```python +# CURRENT (french_rev.md:725-728) — plus `import requests` and `from io import BytesIO` at :73-74 +caron_response = requests.get(base_url + 'caron.npy') +nom_balances_response = requests.get(base_url + 'nom_balances.npy') +caron = np.load(BytesIO(caron_response.content)) +nom_balances = np.load(BytesIO(nom_balances_response.content)) + +# AFTER the .npy → .csv conversion (review §6.3) +caron = pd.read_csv(qeld.url('caron.csv')).to_numpy() +nom_balances = pd.read_csv(qeld.url('nom_balances.csv')).to_numpy() +``` + +**Verdict:** graded "same" by the sweep only because it did not know about the CSV conversion. Converted, +four lines and two imports go, and the intro copy's `np.load('datasets/caron.npy')` local path goes with +them. This is the second-best diff in the corpus and it is bought by a **data** change, not a package +feature. + +### G. Live parameterised API endpoints (4) — out of scope, and say so loudly + +FRED graph queries with ~700-character query strings encoding series id, date range and chart options: +`phillips_drifts_volatilities.md:2213`, `pandas.md:514`, `pandas.md:532/552`, `polars.md:557-572`. + +`qeld.url()` keys bare filenames and cannot express `?id=CPIAUCSL%2CUNRATE%2CTB3MS`. **The trap:** three of +these end in `.csv` and carry the ugliest literals in the corpus (`polars.md`'s is 16 continuation lines), so +they are the most tempting to convert by mistake. In `pandas.md` the raw URL is the explicit *subject* of a +teaching section on `requests` and error handling — hiding it would be a pedagogical regression on top of an +impossibility. + +### H. `scipy.io.loadmat` (1) + +```python +# CURRENT (five_preferences.md:1848) +data = loadmat('dataBHS.mat') + +# with qeld and no format change — nobody would ship this +data = loadmat(io.BytesIO(urllib.request.urlopen(qeld.url('dataBHS.mat')).read())) +``` + +**Verdict:** the only true impossibility among static files. Resolved by the format convention (convert to +CSV) or by exclusion — never by syntax. + +--- + +## 4. Syntax verdict + +| idiom | n | S1 inline | S2 two-step always | S3 mirror shape | +|---|---:|---|---|---| +| A1 split → variable | ~15 | ✗ kills the variable | ✅ | ✅ | +| A2 split inline | ~8 | ✅ | ✗ adds a line | ✅ | +| B variable = literal | 32 | ✗ kills the variable, touches N downstream reads | ✅ | ✅ | +| C inline literal | 18 | ✅ | ✗ 1 line → 2 | ✅ | +| D local path | 5 | ✅ | ✅ | ✅ | + +**S3 is the only candidate that wins on every idiom**, and it has a property the others lack: + +> Under S3 the diff is *always and only* the URL expression. A reviewer can verify a migration PR by reading +> the changed lines alone — there is no restructuring to audit. + +That matters for a ~78-site sweep, and it satisfies the maintainer's bar by **construction** rather than by +argument: the lecture's shape is unchanged, so it cannot have become harder to read. + +**The rule, for AGENTS.md:** + +> Lecture code reads published data through `qeld.url('')`, substituted **in place of the URL +> expression the lecture already uses** — an inline literal becomes an inline call, an assigned variable +> keeps its assignment. The reader (`pd.read_*`, `pl.read_*`, …) and all its kwargs stay visible and +> unchanged. + +Carve-outs: live-API reads (group G); reads whose literal URL is itself the lesson (`pandas_panel`); and +reads whose file is also named by a prose or `{download}` link, until that has an answer (§5). + +--- + +## 5. Two problems no syntax solves + +### 5.1 Generic filenames collide in a flat namespace + +The published tree is flat, so the key is a bare filename. Several pending files are far too generic to +survive it: + +| file | repo of origin | problem | +|---|---|---| +| `fred_data.csv` | advanced | names a source, not a dataset | +| `fp.dta` | python.myst | two letters | +| `test_pwt.csv` | programming | "test" | +| `data.csv` | programming (`about_py.md`) | referenced but never exists | +| `acs_data_summary.csv` | advanced | plausible but claims a whole survey | + +These need renaming at migration. **And a rename breaks text qeld cannot reach**: `mle.md:160` names +`mle/fp.dta` in prose. So the rename must be paired with a prose edit, in the same PR, found by grep rather +than by the audit. + +### 5.2 Prose and `{download}` links + +28 prose/`{download}` refs; **11 name a data file in two places**. The acute case is +`simple_linear_regression.md` (intro:411/416, zh-cn:421/426), where a `{download}` role and the code read sit +**five lines apart containing byte-identical 158-character strings**. Convert the code and the page shows a +raw URL in prose and an opaque call in the cell, with no way for a student to see they agree — and they can +drift silently. + +Options: convert both and accept two spellings; leave those lectures alone; or give qeld a markdown-time +story (a MyST substitution, or point the prose at `CATALOG.md` instead of the file). Currently unanswered, +and it is the strongest argument for the "convert both or neither" carve-out. + +--- + +## 6. What this implies for the package + +1. **`url()` returning a string is confirmed as the right primitive** — S3 only works because the return + value can sit anywhere a URL literal sits today. +2. **No second function is needed for the static corpus.** Groups F and H are resolved by the *format* + convention; group G is out of scope. `open()`/`fetch()` would exist only to serve files the format rule + says should not exist. +3. **The win is concentrated, not broad.** Of ~78 definitions, roughly 10 see qeld delete real logic; ~50 are + cosmetic shortenings of literals adopted in recent repoint PRs; the rest are neutral. The case for the + package rests on those ~10, the wasm shim, the host-cutover property, and the portability bugs in group D + — **not** on the read-site tally. +4. **Rollout should be ordered by win, not by migration track.** Structural sites first + (`hansen_jagannathan_1991`'s dual-path loader, `french_rev`'s `base_url` block, `ols.md`'s five literals, + `polars.md`'s duplicated literal, group D's portability bugs). Cosmetic-only sites can wait indefinitely. +5. **Two latent bugs qeld incidentally fixes**, worth their own issue regardless: `inequality.md` imports + `pyodide_http` and never calls `patch_all()`; `short_path.md` calls `requests.get` under Pyodide with no + shim at all. + +--- + +## 7. Open + +- The `pandas_panel` pedagogical carve-out — permanent, or revisited if the prose is rewritten? +- The markdown-time story for `{download}` links (§5.2). +- The rename list and its prose pairings (§5.1) — needs a pass before Tracks B and C migrate. +- `dataBHS.mat`: convert at migration, or exclude. diff --git a/PLAN-QELD-PACKAGE.md b/PLAN-QELD-PACKAGE.md index 71122ce..4bb7cef 100644 --- a/PLAN-QELD-PACKAGE.md +++ b/PLAN-QELD-PACKAGE.md @@ -194,9 +194,10 @@ reader the lecture will actually use**. 2.1e-16 is immaterial to a figure, but P ## 5. The call-site audit — input to the package -**Snapshot: 2026-08-10**, six repos, 40 lectures, 115 read sites. Machine-readable worklist: -`scripts/qeld_callsites.yml`. **Regenerate before the sweep** — `lecture-python-intro` and this repo both -moved during the session that produced it. +**Snapshot: 2026-08-10**, six repos, 40 lectures, 115 read sites. Working artifacts live in `.dev/qeld/`: +the machine-readable worklist (`callsites.yml`) and the worked before/after evidence per idiom +(`migration-catalog.md`, the spec for what a correct conversion produces). **Regenerate the worklist before +the sweep** — `lecture-python-intro` and this repo both moved during the session that produced it. ### 5.1 The unit of work is ~78 URL definitions, not 115 reads From 487955e464cd269d74600d3395312fd832f72cb6 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Mon, 10 Aug 2026 17:01:33 +1000 Subject: [PATCH 3/3] Fix the repo field in the call-site worklist, and a manifest count in the plan The worklist's `repo` was normalised with a regex that truncated at the first `(` or `/`, so `QuantEcon/lecture-python-advanced.myst (...)` became `QuantEcon` and a path-leading string became the empty string -- 28 of 115 entries carried no repo at all. Regenerated from the raw sweep output with a normaliser that matches against the known repo set and fails loudly on anything unrecognised, so every entry is now `owner/repo`. That also explains the four apparent duplicate entries: heavy_tails.md exists in lecture-python-intro, lecture-wasm and lecture-intro.zh-cn, and the truncation collapsed the intro and wasm reads onto the same key. They are distinct call sites -- and specifically the intro/wasm pairs repoint rule 2 requires be converted together -- so they stay. The header now says so, since (lecture, filename) is not a unique key in this file. Counts are unchanged: 115 entries, per-repo totals matching the sweep, and the same idiom, action and win distributions. Separately, the plan said `sheets` is used in 8 manifests; it is 5. The 8 came from grepping for the string, which also matches prose in comments. `read_as` at 6 was right. Co-Authored-By: Claude Opus 5 (1M context) --- .dev/qeld/callsites.yml | 731 ++++++++++++++++++++-------------------- PLAN-QELD-PACKAGE.md | 2 +- 2 files changed, 370 insertions(+), 363 deletions(-) diff --git a/.dev/qeld/callsites.yml b/.dev/qeld/callsites.yml index e30fc78..044e133 100644 --- a/.dev/qeld/callsites.yml +++ b/.dev/qeld/callsites.yml @@ -8,6 +8,13 @@ # moved during the session that produced it (the A3 french_rev set was # repointed in #49). REGENERATE BEFORE THE SWEEP. # +# `repo` is always owner/repo and is never empty. Several lectures exist in +# more than one repo -- heavy_tails.md, long_run_growth.md and french_rev.md +# are each present in lecture-python-intro, lecture-wasm and lecture-intro.zh-cn +# -- so (lecture, filename) is NOT a unique key, and entries differing only by +# repo are distinct call sites rather than duplicates. Repoint rule 2 requires +# the intro/wasm pair be converted together. +# # action: # convert apply the substitute-in-place rule # carve-out do not convert - see the note @@ -19,572 +26,564 @@ # a literal. The rollout takes structural sites first. callsites: - - repo: "QuantEcon" - lecture: "lectures/risk_aversion_or_mistaken_beliefs.md:1613" - filename: "fred_data.csv" - idiom: "A-split-literal" - verdict: "simpler" - action: "convert" - win: "cosmetic" - - repo: "lecture-python-programming" - lecture: "lectures/polars.md:160-163" - filename: "test_pwt.csv" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/fitting_distributions.md:257-259" + filename: "us_adult_heights.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "lecture-python-programming" - lecture: "lectures/polars.md:346-349" - filename: "test_pwt.csv" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/fitting_distributions.md:467-469" + filename: "epl_match_goals.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "lecture-python-programming" - lecture: "lectures/polars.md:424-428" - filename: "test_pwt.csv" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/fitting_distributions.md:667-669" + filename: "japan_earthquakes.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "lecture-python.myst" - lecture: "lectures/phillips_drifts_volatilities.md:102" - filename: "NEWQDATA.csv" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/fitting_distributions.md:753-755" + filename: "japan_deaths_by_age.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" + - repo: "QuantEcon/lecture-intro.zh-cn" lecture: "lectures/fitting_distributions.md:77-79" filename: "ames_house_prices.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/fitting_distributions.md:257-259" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/observed_distributions.md:137-139" filename: "us_adult_heights.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/fitting_distributions.md:467-469" - filename: "epl_match_goals.csv" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/observed_distributions.md:152-154" + filename: "ames_house_prices.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/fitting_distributions.md:667-669" - filename: "japan_earthquakes.csv" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/observed_distributions.md:187-189" + filename: "japan_deaths_by_age.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/fitting_distributions.md:753-755" - filename: "japan_deaths_by_age.csv" + - repo: "QuantEcon/lecture-python-advanced.myst" + lecture: "lectures/risk_aversion_or_mistaken_beliefs.md:1613" + filename: "fred_data.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/observed_distributions.md:137-139" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/fitting_distributions.md:277" filename: "us_adult_heights.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/observed_distributions.md:152-154" - filename: "ames_house_prices.csv" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/fitting_distributions.md:522" + filename: "epl_match_goals.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/observed_distributions.md:187-189" - filename: "japan_deaths_by_age.csv" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/fitting_distributions.md:72" + filename: "ames_house_prices.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/prob_dist.md:57" - filename: "us_adult_heights.csv" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/fitting_distributions.md:757" + filename: "japan_earthquakes.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/prob_dist.md:130" - filename: "japan_population_by_age.csv" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/fitting_distributions.md:855" + filename: "japan_deaths_by_age.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-python-intro" lecture: "lectures/observed_distributions.md:133" filename: "us_adult_heights.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-python-intro" lecture: "lectures/observed_distributions.md:151" filename: "ames_house_prices.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-python-intro" lecture: "lectures/observed_distributions.md:192" filename: "japan_deaths_by_age.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/fitting_distributions.md:72" - filename: "ames_house_prices.csv" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/prob_dist.md:130" + filename: "japan_population_by_age.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/fitting_distributions.md:277" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/prob_dist.md:57" filename: "us_adult_heights.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/fitting_distributions.md:522" - filename: "epl_match_goals.csv" + - repo: "QuantEcon/lecture-python-programming" + lecture: "lectures/polars.md:160-163" + filename: "test_pwt.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/fitting_distributions.md:757" - filename: "japan_earthquakes.csv" + - repo: "QuantEcon/lecture-python-programming" + lecture: "lectures/polars.md:346-349" + filename: "test_pwt.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/fitting_distributions.md:855" - filename: "japan_deaths_by_age.csv" + - repo: "QuantEcon/lecture-python-programming" + lecture: "lectures/polars.md:424-428" + filename: "test_pwt.csv" idiom: "A-split-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "lecture-python-programming" - lecture: "lectures/pandas_panel.md:78,90" - filename: "realwage.csv" - idiom: "B-url-variable" - verdict: "same" - action: "carve-out" - win: "cosmetic" - note: "the literal URL is the lesson - see PLAN section 8.1" - - repo: "lecture-python-programming" - lecture: "lectures/pandas_panel.md:198,202" - filename: "countries.csv" - idiom: "B-url-variable" - verdict: "same" - action: "carve-out" + - repo: "QuantEcon/lecture-python.myst" + lecture: "lectures/phillips_drifts_volatilities.md:102" + filename: "NEWQDATA.csv" + idiom: "A-split-literal" + verdict: "simpler" + action: "convert" win: "cosmetic" - note: "the literal URL is the lesson - see PLAN section 8.1" - - repo: "lecture-python-programming" - lecture: "lectures/pandas_panel.md:507,526" - filename: "employ.csv" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/french_rev.md:84-89" + filename: "fig_3.xlsx, dette.xlsx, assignat.xlsx" idiom: "B-url-variable" verdict: "same" - action: "carve-out" - win: "cosmetic" - note: "the literal URL is the lesson - see PLAN section 8.1" - - repo: "lecture-python.myst" - lecture: "lectures/pandas_panel.md:68" - filename: "realwage.csv" - idiom: "B-url-variable" - verdict: "more_complex" - action: "carve-out" - win: "cosmetic" - note: "the literal URL is the lesson - see PLAN section 8.1" - - repo: "lecture-python.myst" - lecture: "lectures/pandas_panel.md:188" - filename: "countries.csv" - idiom: "B-url-variable" - verdict: "more_complex" - action: "carve-out" - win: "cosmetic" - note: "the literal URL is the lesson - see PLAN section 8.1" - - repo: "lecture-python.myst" - lecture: "lectures/pandas_panel.md:505" - filename: "employ.csv" + action: "convert" + win: "structural" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/inequality.md:256-257" + filename: "SCF_plus_mini.csv" idiom: "B-url-variable" - verdict: "more_complex" - action: "carve-out" + verdict: "simpler" + action: "convert" win: "cosmetic" - note: "the literal URL is the lesson - see PLAN section 8.1" - - repo: "QuantEcon" - lecture: "lectures/long_run_growth.md:96-99" - filename: "mpd2020.xlsx" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/inequality.md:613-614" + filename: "usa-gini-nwealth-tincome-lincome.csv" idiom: "B-url-variable" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/inflation_history.md:75-80" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/inflation_history.md:102-107" filename: "longprices.xls" idiom: "B-url-variable" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/inflation_history.md:350-351" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/inflation_history.md:374-375" filename: "chapter_3.xlsx" idiom: "B-url-variable" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/french_rev.md:80-85" - filename: "fig_3.xlsx, dette.xlsx, assignat.xlsx" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/long_run_growth.md:106-108" + filename: "mpd2020.xlsx" idiom: "B-url-variable" - verdict: "simpler" + verdict: "more_complex" action: "convert" - win: "structural" - - repo: "QuantEcon" - lecture: "lectures/simple_linear_regression.md:426-427" - filename: "life-expectancy-vs-gdp-per-capita.csv" - idiom: "B-url-variable" - verdict: "same" - action: "carve-out" win: "cosmetic" - note: "paired with a download link - see PLAN section 8.2" - - repo: "QuantEcon" - lecture: "lectures/mle.md:95-96" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/mle.md:105-106" filename: "SCF_plus_mini_no_weights.csv" idiom: "B-url-variable" verdict: "same" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/short_path.md:274-277" - filename: "graph.txt" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/simple_linear_regression.md:426-427" + filename: "life-expectancy-vs-gdp-per-capita.csv" idiom: "B-url-variable" - verdict: "same" - action: "convert" + verdict: "more_complex" + action: "carve-out" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/inequality.md:250-251" - filename: "SCF_plus_mini.csv" + note: "paired with a download link - see PLAN section 8.2" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/french_rev.md:75" + filename: "fig_3.xlsx" idiom: "B-url-variable" verdict: "simpler" action: "convert" - win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/inequality.md:619-620" - filename: "usa-gini-nwealth-tincome-lincome.csv" + win: "structural" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/inequality.md:249" + filename: "SCF_plus_mini.csv" idiom: "B-url-variable" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/inflation_history.md:102-107" - filename: "longprices.xls" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/inequality.md:621" + filename: "usa-gini-nwealth-tincome-lincome.csv" idiom: "B-url-variable" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/inflation_history.md:374-375" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/inflation_history.md:358" filename: "chapter_3.xlsx" idiom: "B-url-variable" - verdict: "simpler" + verdict: "same" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/long_run_growth.md:106-108" - filename: "mpd2020.xlsx" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/inflation_history.md:86" + filename: "longprices.xls" idiom: "B-url-variable" - verdict: "more_complex" + verdict: "same" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/french_rev.md:84-89" - filename: "fig_3.xlsx, dette.xlsx, assignat.xlsx" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/long_run_growth.md:89" + filename: "mpd2020.xlsx" idiom: "B-url-variable" verdict: "same" action: "convert" - win: "structural" - - repo: "" - lecture: "lectures/inequality.md:256-257" - filename: "SCF_plus_mini.csv" + win: "cosmetic" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/mle.md:93" + filename: "SCF_plus_mini_no_weights.csv" idiom: "B-url-variable" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/inequality.md:613-614" - filename: "usa-gini-nwealth-tincome-lincome.csv" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/msy_fishery.md:596" + filename: "lingcod_msy_recovery.csv" idiom: "B-url-variable" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/mle.md:105-106" - filename: "SCF_plus_mini_no_weights.csv" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/simple_linear_regression.md:416" + filename: "life-expectancy-vs-gdp-per-capita.csv" + idiom: "B-url-variable" + verdict: "more_complex" + action: "carve-out" + win: "cosmetic" + note: "paired with a download link - see PLAN section 8.2" + - repo: "QuantEcon/lecture-python-programming" + lecture: "lectures/pandas_panel.md:198,202" + filename: "countries.csv" idiom: "B-url-variable" verdict: "same" - action: "convert" + action: "carve-out" win: "cosmetic" - - repo: "" - lecture: "lectures/simple_linear_regression.md:426-427" - filename: "life-expectancy-vs-gdp-per-capita.csv" + note: "the literal URL is the lesson - see PLAN section 8.1" + - repo: "QuantEcon/lecture-python-programming" + lecture: "lectures/pandas_panel.md:507,526" + filename: "employ.csv" + idiom: "B-url-variable" + verdict: "same" + action: "carve-out" + win: "cosmetic" + note: "the literal URL is the lesson - see PLAN section 8.1" + - repo: "QuantEcon/lecture-python-programming" + lecture: "lectures/pandas_panel.md:78,90" + filename: "realwage.csv" + idiom: "B-url-variable" + verdict: "same" + action: "carve-out" + win: "cosmetic" + note: "the literal URL is the lesson - see PLAN section 8.1" + - repo: "QuantEcon/lecture-python.myst" + lecture: "lectures/pandas_panel.md:188" + filename: "countries.csv" idiom: "B-url-variable" verdict: "more_complex" action: "carve-out" win: "cosmetic" - note: "paired with a download link - see PLAN section 8.2" - - repo: "QuantEcon" - lecture: "lectures/french_rev.md:75" - filename: "fig_3.xlsx" + note: "the literal URL is the lesson - see PLAN section 8.1" + - repo: "QuantEcon/lecture-python.myst" + lecture: "lectures/pandas_panel.md:505" + filename: "employ.csv" + idiom: "B-url-variable" + verdict: "more_complex" + action: "carve-out" + win: "cosmetic" + note: "the literal URL is the lesson - see PLAN section 8.1" + - repo: "QuantEcon/lecture-python.myst" + lecture: "lectures/pandas_panel.md:68" + filename: "realwage.csv" + idiom: "B-url-variable" + verdict: "more_complex" + action: "carve-out" + win: "cosmetic" + note: "the literal URL is the lesson - see PLAN section 8.1" + - repo: "QuantEcon/lecture-wasm" + lecture: "lectures/french_rev.md:80-85" + filename: "fig_3.xlsx, dette.xlsx, assignat.xlsx" idiom: "B-url-variable" verdict: "simpler" action: "convert" win: "structural" - - repo: "QuantEcon" - lecture: "lectures/long_run_growth.md:89" - filename: "mpd2020.xlsx" + - repo: "QuantEcon/lecture-wasm" + lecture: "lectures/inequality.md:250-251" + filename: "SCF_plus_mini.csv" idiom: "B-url-variable" - verdict: "same" + verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/inflation_history.md:86" - filename: "longprices.xls" + - repo: "QuantEcon/lecture-wasm" + lecture: "lectures/inequality.md:619-620" + filename: "usa-gini-nwealth-tincome-lincome.csv" idiom: "B-url-variable" - verdict: "same" + verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/inflation_history.md:358" + - repo: "QuantEcon/lecture-wasm" + lecture: "lectures/inflation_history.md:350-351" filename: "chapter_3.xlsx" idiom: "B-url-variable" - verdict: "same" + verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/msy_fishery.md:596" - filename: "lingcod_msy_recovery.csv" + - repo: "QuantEcon/lecture-wasm" + lecture: "lectures/inflation_history.md:75-80" + filename: "longprices.xls" idiom: "B-url-variable" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/mle.md:93" - filename: "SCF_plus_mini_no_weights.csv" + - repo: "QuantEcon/lecture-wasm" + lecture: "lectures/long_run_growth.md:96-99" + filename: "mpd2020.xlsx" idiom: "B-url-variable" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/inequality.md:249" - filename: "SCF_plus_mini.csv" + - repo: "QuantEcon/lecture-wasm" + lecture: "lectures/mle.md:95-96" + filename: "SCF_plus_mini_no_weights.csv" idiom: "B-url-variable" - verdict: "simpler" + verdict: "same" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/inequality.md:621" - filename: "usa-gini-nwealth-tincome-lincome.csv" + - repo: "QuantEcon/lecture-wasm" + lecture: "lectures/short_path.md:274-277" + filename: "graph.txt" idiom: "B-url-variable" - verdict: "simpler" + verdict: "same" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/simple_linear_regression.md:416" + - repo: "QuantEcon/lecture-wasm" + lecture: "lectures/simple_linear_regression.md:426-427" filename: "life-expectancy-vs-gdp-per-capita.csv" idiom: "B-url-variable" - verdict: "more_complex" + verdict: "same" action: "carve-out" win: "cosmetic" note: "paired with a download link - see PLAN section 8.2" - - repo: "lecture-python-programming" - lecture: "lectures/pandas.md:165" - filename: "test_pwt.csv" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/heavy_tails.md:811" + filename: "forbes-global2000.csv" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "lecture-python.myst" - lecture: "lectures/ols.md:94" - filename: "maketable1.dta" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/heavy_tails.md:838-839" + filename: "cities_us.csv, cities_brazil.csv" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "lecture-python.myst" - lecture: "lectures/ols.md:326" - filename: "maketable2.dta" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/heavy_tails.md:863" + filename: "forbes-billionaires.csv" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "lecture-python.myst" - lecture: "lectures/ols.md:478" - filename: "maketable4.dta" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/heavy_tails.md:827" + filename: "forbes-global2000.csv" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "lecture-python.myst" - lecture: "lectures/ols.md:606" - filename: "maketable4.dta" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/heavy_tails.md:854" + filename: "cities_us.csv" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "lecture-python.myst" - lecture: "lectures/ols.md:679" - filename: "maketable1.dta" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/heavy_tails.md:855" + filename: "cities_brazil.csv" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "lecture-python.myst" - lecture: "lectures/mle.md:165" - filename: "fp.dta" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/heavy_tails.md:879" + filename: "forbes-billionaires.csv" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/heavy_tails.md:827" - filename: "forbes-global2000.csv" + - repo: "QuantEcon/lecture-python-programming" + lecture: "lectures/pandas.md:165" + filename: "test_pwt.csv" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/heavy_tails.md:854" - filename: "cities_us.csv" + - repo: "QuantEcon/lecture-python.myst" + lecture: "lectures/mle.md:165" + filename: "fp.dta" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/heavy_tails.md:855" - filename: "cities_brazil.csv" + - repo: "QuantEcon/lecture-python.myst" + lecture: "lectures/ols.md:326" + filename: "maketable2.dta" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/heavy_tails.md:879" - filename: "forbes-billionaires.csv" + - repo: "QuantEcon/lecture-python.myst" + lecture: "lectures/ols.md:478" + filename: "maketable4.dta" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/heavy_tails.md:811" - filename: "forbes-global2000.csv" + - repo: "QuantEcon/lecture-python.myst" + lecture: "lectures/ols.md:606" + filename: "maketable4.dta" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/heavy_tails.md:838-839" - filename: "cities_us.csv, cities_brazil.csv" + - repo: "QuantEcon/lecture-python.myst" + lecture: "lectures/ols.md:679" + filename: "maketable1.dta" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "" - lecture: "lectures/heavy_tails.md:863" - filename: "forbes-billionaires.csv" + - repo: "QuantEcon/lecture-python.myst" + lecture: "lectures/ols.md:94" + filename: "maketable1.dta" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/heavy_tails.md:827" filename: "forbes-global2000.csv" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/heavy_tails.md:854" filename: "cities_us.csv" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/heavy_tails.md:855" filename: "cities_brazil.csv" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/heavy_tails.md:879" filename: "forbes-billionaires.csv" idiom: "C-inline-literal" verdict: "simpler" action: "convert" win: "cosmetic" - - repo: "QuantEcon" - lecture: "lectures/subjective_beliefs_business_cycles.md:149" - filename: "bbh_macro_quarterly.csv" - idiom: "D-local-path" - verdict: "same" - action: "convert" - win: "structural" - - repo: "QuantEcon" - lecture: "lectures/subjective_beliefs_business_cycles.md:152" - filename: "bbh_michigan_monthly.csv" - idiom: "D-local-path" - verdict: "simpler" - action: "convert" - win: "structural" - - repo: "QuantEcon" - lecture: "lectures/match_transport.md:2234" - filename: "acs_data_summary.csv" - idiom: "D-local-path" - verdict: "simpler" - action: "convert" - win: "structural" - - repo: "" - lecture: "lectures/long_run_growth.md:141" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/business_cycle.md:62" filename: "country_code_cn.csv" idiom: "D-local-path" verdict: "impossible" action: "exclude" win: "structural" note: "zh-cn translation asset; no data-lectures key" - - repo: "" - lecture: "lectures/business_cycle.md:62" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/long_run_growth.md:141" filename: "country_code_cn.csv" idiom: "D-local-path" verdict: "impossible" action: "exclude" win: "structural" note: "zh-cn translation asset; no data-lectures key" - - repo: "QuantEcon" - lecture: "lectures/french_rev.md:725-728" - filename: "caron.npy, nom_balances.npy" - idiom: "F-npy-bytesio" + - repo: "QuantEcon/lecture-python-advanced.myst" + lecture: "lectures/match_transport.md:2234" + filename: "acs_data_summary.csv" + idiom: "D-local-path" + verdict: "simpler" + action: "convert" + win: "structural" + - repo: "QuantEcon/lecture-python-advanced.myst" + lecture: "lectures/subjective_beliefs_business_cycles.md:149" + filename: "bbh_macro_quarterly.csv" + idiom: "D-local-path" verdict: "same" - action: "blocked" + action: "convert" win: "structural" - note: "npy cannot be read from a URL - see PLAN section 8.3" - - repo: "" + - repo: "QuantEcon/lecture-python-advanced.myst" + lecture: "lectures/subjective_beliefs_business_cycles.md:152" + filename: "bbh_michigan_monthly.csv" + idiom: "D-local-path" + verdict: "simpler" + action: "convert" + win: "structural" + - repo: "QuantEcon/lecture-intro.zh-cn" lecture: "lectures/french_rev.md:671-674" filename: "caron.npy, nom_balances.npy" idiom: "F-npy-bytesio" @@ -592,7 +591,7 @@ callsites: action: "blocked" win: "structural" note: "npy cannot be read from a URL - see PLAN section 8.3" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-python-intro" lecture: "lectures/french_rev.md:711" filename: "caron.npy" idiom: "F-npy-bytesio" @@ -600,7 +599,7 @@ callsites: action: "blocked" win: "structural" note: "npy cannot be read from a URL - see PLAN section 8.3" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-python-intro" lecture: "lectures/french_rev.md:712" filename: "nom_balances.npy" idiom: "F-npy-bytesio" @@ -608,7 +607,15 @@ callsites: action: "blocked" win: "structural" note: "npy cannot be read from a URL - see PLAN section 8.3" - - repo: "lecture-python-programming" + - repo: "QuantEcon/lecture-wasm" + lecture: "lectures/french_rev.md:725-728" + filename: "caron.npy, nom_balances.npy" + idiom: "F-npy-bytesio" + verdict: "same" + action: "blocked" + win: "structural" + note: "npy cannot be read from a URL - see PLAN section 8.3" + - repo: "QuantEcon/lecture-python-programming" lecture: "lectures/pandas.md:514" filename: "fredgraph.csv (FRED query endpoint)" idiom: "G-live-api" @@ -616,7 +623,7 @@ callsites: action: "exclude" win: "none" note: "live parameterised API endpoint, not a file" - - repo: "lecture-python-programming" + - repo: "QuantEcon/lecture-python-programming" lecture: "lectures/pandas.md:532,552" filename: "fredgraph.csv (FRED query endpoint)" idiom: "G-live-api" @@ -624,7 +631,7 @@ callsites: action: "exclude" win: "none" note: "live parameterised API endpoint, not a file" - - repo: "lecture-python-programming" + - repo: "QuantEcon/lecture-python-programming" lecture: "lectures/polars.md:557-572" filename: "fredgraph.csv (FRED query endpoint)" idiom: "G-live-api" @@ -632,7 +639,7 @@ callsites: action: "exclude" win: "none" note: "live parameterised API endpoint, not a file" - - repo: "lecture-python.myst" + - repo: "QuantEcon/lecture-python.myst" lecture: "lectures/phillips_drifts_volatilities.md:2213" filename: "fredgraph.csv (FRED query endpoint, not a file)" idiom: "G-live-api" @@ -640,7 +647,7 @@ callsites: action: "exclude" win: "none" note: "live parameterised API endpoint, not a file" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-python-advanced.myst" lecture: "lectures/five_preferences.md:1848" filename: "dataBHS.mat" idiom: "H-loadmat" @@ -648,7 +655,7 @@ callsites: action: "blocked" win: "none" note: "scipy.io.loadmat cannot take a URL - see PLAN section 8.4" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-python-advanced.myst" lecture: "lectures/hansen_jagannathan_1991.md:182" filename: "hansen_jagannathan_1991_data.json" idiom: "I-generated" @@ -656,47 +663,87 @@ callsites: action: "exclude" win: "structural" note: "file is generated by the lecture itself" - - repo: "lecture-python.myst" - lecture: "lectures/hansen_singleton_1982.md:993" - filename: "hansen_singleton_1982_data.csv" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/french_rev.md:110,115,158,231,233,413,447,492,555,557,615,617" + filename: "dette.xlsx / assignat.xlsx / fig_3.xlsx (12 reuses)" idiom: "Z-downstream-read" verdict: "same" action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "lecture-python.myst" - lecture: "lectures/hansen_singleton_1983.md:1432" - filename: "hansen_singleton_1983_data.csv" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/long_run_growth.md:486" + filename: "mpd2020.xlsx (reuse)" idiom: "Z-downstream-read" verdict: "same" action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" - lecture: "lectures/long_run_growth.md:488" - filename: "mpd2020.xlsx" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/long_run_growth.md:589-592" + filename: "mpd2020.xlsx (reuse)" idiom: "Z-downstream-read" verdict: "same" action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" - lecture: "lectures/long_run_growth.md:593-596" - filename: "mpd2020.xlsx" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/prob_dist.md:152-159" + filename: "japan_population_by_age.xlsx (data-lectures key: japan_population_by_age.csv)" + idiom: "Z-downstream-read" + verdict: "simpler" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/prob_dist.md:74-77" + filename: "us_adult_heights.csv" + idiom: "Z-downstream-read" + verdict: "simpler" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon/lecture-intro.zh-cn" + lecture: "lectures/simple_linear_regression.md:444" + filename: "life-expectancy-vs-gdp-per-capita.csv (reuse)" idiom: "Z-downstream-read" verdict: "same" action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" - lecture: "lectures/inflation_history.md:374-375" - filename: "chapter_3.xlsx" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/french_rev.md:76" + filename: "dette.xlsx" + idiom: "Z-downstream-read" + verdict: "simpler" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon/lecture-python-intro" + lecture: "lectures/french_rev.md:77" + filename: "assignat.xlsx" + idiom: "Z-downstream-read" + verdict: "simpler" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon/lecture-python.myst" + lecture: "lectures/hansen_singleton_1982.md:993" + filename: "hansen_singleton_1982_data.csv" + idiom: "Z-downstream-read" + verdict: "same" + action: "downstream" + win: "none" + note: "read of a variable assigned elsewhere" + - repo: "QuantEcon/lecture-python.myst" + lecture: "lectures/hansen_singleton_1983.md:1432" + filename: "hansen_singleton_1983_data.csv" idiom: "Z-downstream-read" verdict: "same" action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/french_rev.md:106-108" filename: "dette.xlsx" idiom: "Z-downstream-read" @@ -704,7 +751,7 @@ callsites: action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/french_rev.md:111-113" filename: "dette.xlsx" idiom: "Z-downstream-read" @@ -712,7 +759,7 @@ callsites: action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/french_rev.md:155-156" filename: "dette.xlsx" idiom: "Z-downstream-read" @@ -720,7 +767,7 @@ callsites: action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/french_rev.md:233-234" filename: "dette.xlsx" idiom: "Z-downstream-read" @@ -728,7 +775,7 @@ callsites: action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/french_rev.md:235-236" filename: "dette.xlsx" idiom: "Z-downstream-read" @@ -736,7 +783,7 @@ callsites: action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/french_rev.md:276-277" filename: "fig_3.xlsx" idiom: "Z-downstream-read" @@ -744,7 +791,7 @@ callsites: action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/french_rev.md:438-439" filename: "dette.xlsx" idiom: "Z-downstream-read" @@ -752,7 +799,7 @@ callsites: action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/french_rev.md:475-476" filename: "assignat.xlsx" idiom: "Z-downstream-read" @@ -760,7 +807,7 @@ callsites: action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/french_rev.md:522-523" filename: "assignat.xlsx" idiom: "Z-downstream-read" @@ -768,7 +815,7 @@ callsites: action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/french_rev.md:586-587" filename: "assignat.xlsx" idiom: "Z-downstream-read" @@ -776,7 +823,7 @@ callsites: action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/french_rev.md:588-589" filename: "assignat.xlsx" idiom: "Z-downstream-read" @@ -784,7 +831,7 @@ callsites: action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/french_rev.md:656-657" filename: "assignat.xlsx" idiom: "Z-downstream-read" @@ -792,7 +839,7 @@ callsites: action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/french_rev.md:658-659" filename: "assignat.xlsx" idiom: "Z-downstream-read" @@ -800,75 +847,35 @@ callsites: action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" - lecture: "lectures/simple_linear_regression.md:444" - filename: "life-expectancy-vs-gdp-per-capita.csv" - idiom: "Z-downstream-read" - verdict: "same" - action: "downstream" - win: "none" - note: "read of a variable assigned elsewhere" - - repo: "" - lecture: "lectures/prob_dist.md:74-77" - filename: "us_adult_heights.csv" - idiom: "Z-downstream-read" - verdict: "simpler" - action: "downstream" - win: "none" - note: "read of a variable assigned elsewhere" - - repo: "" - lecture: "lectures/prob_dist.md:152-159" - filename: "japan_population_by_age.xlsx (data-lectures key: japan_population_by_age.csv)" - idiom: "Z-downstream-read" - verdict: "simpler" - action: "downstream" - win: "none" - note: "read of a variable assigned elsewhere" - - repo: "" - lecture: "lectures/long_run_growth.md:486" - filename: "mpd2020.xlsx (reuse)" + - repo: "QuantEcon/lecture-wasm" + lecture: "lectures/inflation_history.md:374-375" + filename: "chapter_3.xlsx" idiom: "Z-downstream-read" verdict: "same" action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "" - lecture: "lectures/long_run_growth.md:589-592" - filename: "mpd2020.xlsx (reuse)" + - repo: "QuantEcon/lecture-wasm" + lecture: "lectures/long_run_growth.md:488" + filename: "mpd2020.xlsx" idiom: "Z-downstream-read" verdict: "same" action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "" - lecture: "lectures/french_rev.md:110,115,158,231,233,413,447,492,555,557,615,617" - filename: "dette.xlsx / assignat.xlsx / fig_3.xlsx (12 reuses)" + - repo: "QuantEcon/lecture-wasm" + lecture: "lectures/long_run_growth.md:593-596" + filename: "mpd2020.xlsx" idiom: "Z-downstream-read" verdict: "same" action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "" + - repo: "QuantEcon/lecture-wasm" lecture: "lectures/simple_linear_regression.md:444" - filename: "life-expectancy-vs-gdp-per-capita.csv (reuse)" + filename: "life-expectancy-vs-gdp-per-capita.csv" idiom: "Z-downstream-read" verdict: "same" action: "downstream" win: "none" note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" - lecture: "lectures/french_rev.md:76" - filename: "dette.xlsx" - idiom: "Z-downstream-read" - verdict: "simpler" - action: "downstream" - win: "none" - note: "read of a variable assigned elsewhere" - - repo: "QuantEcon" - lecture: "lectures/french_rev.md:77" - filename: "assignat.xlsx" - idiom: "Z-downstream-read" - verdict: "simpler" - action: "downstream" - win: "none" - note: "read of a variable assigned elsewhere" diff --git a/PLAN-QELD-PACKAGE.md b/PLAN-QELD-PACKAGE.md index 4bb7cef..0c5fbb8 100644 --- a/PLAN-QELD-PACKAGE.md +++ b/PLAN-QELD-PACKAGE.md @@ -289,7 +289,7 @@ or every migrated read classifies `local-path` and the dashboard inverts. | phase | work | gate | |---|---|---| | **Q1 — Audit first** | `build_audit.py` learns `qeld.url('X')` → pattern `qeld`, counted migrated **and terminal**. For `pattern == 'qeld'`, assert the key exists in `lectures/` and is not deprecated — otherwise the qeld path loses every assertion #55/#48/#47 added. `migration.yml`: `final` := canonical-host *or* qeld | `audit.json` `stats` and `problems` unchanged on today's repos (**not** "byte-identical" — the audit stamps `date.today()`) | -| **Q2 — Schema hygiene** | Document `read_as` (used in 6 manifests) and `sheets` (8) in `manifest-schema.yml` — both are in use and neither appears in the file `AGENTS.md` calls "the authoritative, commented field reference". Add `deprecated:` (new, used nowhere yet) since §3.3 warns on it. `shape` is already documented. Delete `then: "iloc[1:]"` from `longprices.xls.yml:70` by moving `iloc[1:]` into the lecture — a post-read transform encoded as a string to evaluate is exactly what D5 excludes | `manifest-schema.yml` covers every field any manifest uses. Needs none of #14's decisions — do not block on it | +| **Q2 — Schema hygiene** | Document `read_as` (used in 6 manifests) and `sheets` (5) in `manifest-schema.yml` — both are in use and neither appears in the file `AGENTS.md` calls "the authoritative, commented field reference". Add `deprecated:` (new, used nowhere yet) since §3.3 warns on it. `shape` is already documented. Delete `then: "iloc[1:]"` from `longprices.xls.yml:70` by moving `iloc[1:]` into the lecture — a post-read transform encoded as a string to evaluate is exactly what D5 excludes | `manifest-schema.yml` covers every field any manifest uses. Needs none of #14's decisions — do not block on it | | **Q3 — Package** | `packages/qeld/`: `url()`, `info()`, context detection, advisory catalog. Catalog compiler shares a freshness gate with `CATALOG.md`. Format tier-1 assertion. First release to PyPI via trusted publishing | Offline suite green on every PR: catalog compiles and is fresh; unknown key warns and still returns a URL; URL form correct per detected context; suffix fidelity incl. `.csv.gz`; `info()` fields present. CPython matrix | | **Q4 — Live leg** | Post-merge + scheduled job: fetch each served URL, compare to the manifest hash, open an issue on failure | Green on `main`; an induced failure opens an issue | | **Q5 — Browser session** | `%pip install qeld==` in a real `lecture-wasm` page (**`%pip` routes through piplite, not micropip** — a console `micropip.install` is a false pass); `pd.read_excel(qeld.url('mpd2020.xlsx'), sheet_name='Regional data', header=[0,1,2], index_col=0)`; a `.csv.gz` read; record observed Pyodide and pyodide-kernel versions | Written pass/fail. Fail ⇒ wasm keeps URLs and the plan proceeds for the CPython repos |