From c061af456587562552680242e22a8c86da950ef8 Mon Sep 17 00:00:00 2001 From: DaanVanHauwermeiren Date: Wed, 5 Aug 2026 11:51:27 +0200 Subject: [PATCH 1/4] build: pin Pluto exactly so Pkg.update cannot nuke the cache MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit _cache/*.plutostate filenames embed the Pluto version, so any Pluto bump invalidates all 46 entries at once and the next build cold-runs every notebook, which exceeds the 6h job limit. Project.toml had no [compat] section at all, so a routine Pkg.update() could trigger this silently. Pin Pluto and PlutoSliderServer with "=" to make the bump a deliberate, reviewable change, and force-track the deployment Manifest, which is the only thing recording which version the cache was built with. Two edits. [compat] added to pluto-deployment-environment/Project.toml (verified it still parses; 38 deps intact), and a trailing !pluto-deployment-environment/Manifest.toml in .gitignore. Placement matters — git applies the last matching rule and there are four Manifest.toml lines, the last at 693, so the negation has to sit at the end of the file. Verified both directions: git check-ignore pluto-deployment-environment/Manifest.toml -> exit 1 (tracked) ✅ git check-ignore exercises/solved_notebooks/Manifest.toml -> .gitignore:693 ✅ --- .gitignore | 5 +++++ pluto-deployment-environment/Project.toml | 9 +++++++++ 2 files changed, 14 insertions(+) diff --git a/.gitignore b/.gitignore index 5976268b..ea0d29da 100644 --- a/.gitignore +++ b/.gitignore @@ -697,3 +697,8 @@ Manifest.toml *.jpg *.svg # _cache is on purpose omitted to avoid again cold run of github actions that results in timeout + +# ...but the deployment Manifest must stay tracked: it is the only thing that +# pins Pluto to the version baked into the _cache/*.plutostate filenames. +# Keep this last - .gitignore applies the last matching rule. +!pluto-deployment-environment/Manifest.toml diff --git a/pluto-deployment-environment/Project.toml b/pluto-deployment-environment/Project.toml index dffa0591..789c1cb3 100644 --- a/pluto-deployment-environment/Project.toml +++ b/pluto-deployment-environment/Project.toml @@ -37,3 +37,12 @@ URIs = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4" Unicode = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5" Unitful = "1986cc42-f94f-5a68-af5c-568840ba703d" YAML = "ddb6d928-2868-570f-bddf-ab3f9cf99eb6" + +[compat] +# EXACT pins, on purpose. The cached notebook states in _cache/ are named +# ".plutostate" (see PlutoSliderServer's +# Export.jl), so bumping Pluto invalidates all of them at once and the next +# build cold-runs every notebook - which exceeds GitHub's 6h job limit. +# Bumping these is a deliberate act: see website_maintenance.md first. +Pluto = "=0.20.13" +PlutoSliderServer = "=1.4.0" From 506e1adda35c3ae2af62306674fb7bf4cea7077b Mon Sep 17 00:00:00 2001 From: DaanVanHauwermeiren Date: Wed, 5 Aug 2026 11:52:37 +0200 Subject: [PATCH 2/4] ci: fail fast when the notebook cache cannot serve the build MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A stale cache was only discovered by a job that ran for six hours and then got killed, leaving no usable cache behind, so the next push failed the same way. Add tools/check_cache.py, which reproduces PlutoSliderServer's cache filename (pluto version + base64url(sha256(notebook))) in plain Python, and run it as a cache-preflight job gating build-and-deploy. Hard-fails on total invalidation (Pluto bump, empty _cache); only warns for edited notebooks and orphans, which are normal and cheap. Runs in ~2s, no Julia needed. The guard detects notebooks by the ### A Pluto.jl notebook ### header rather than by path — same test PlutoPages uses. That returns exactly 46, and stays right if someone adds a plain .jl helper under src/. It also honours ignore_cache from PlutoDeployment.toml. --- .github/workflows/ExportNotebooks.yml | 14 +++ tools/check_cache.py | 133 ++++++++++++++++++++++++++ 2 files changed, 147 insertions(+) create mode 100755 tools/check_cache.py diff --git a/.github/workflows/ExportNotebooks.yml b/.github/workflows/ExportNotebooks.yml index 73f29192..cd15a0b5 100644 --- a/.github/workflows/ExportNotebooks.yml +++ b/.github/workflows/ExportNotebooks.yml @@ -12,7 +12,21 @@ concurrency: cancel-in-progress: true jobs: + # Cheap gate on an expensive job: a cache that cannot serve this build means + # cold-running every notebook, which does not fit in the 6h job limit. Find + # out in seconds rather than after six hours of runner time. + cache-preflight: + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - name: Checkout source + uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + + - name: 🔎 Check notebook cache against src/ and the pinned Pluto version + run: python3 tools/check_cache.py + build-and-deploy: + needs: cache-preflight runs-on: ubuntu-latest timeout-minutes: 360 steps: diff --git a/tools/check_cache.py b/tools/check_cache.py new file mode 100755 index 00000000..fbe19384 --- /dev/null +++ b/tools/check_cache.py @@ -0,0 +1,133 @@ +#!/usr/bin/env python3 +"""Check that _cache/ matches the notebooks in src/ and the pinned Pluto version. + +Run before the expensive export job so a stale cache fails in seconds instead +of after a six-hour build that gets killed by the GitHub job limit. + +The cached notebook states are named by PlutoSliderServer as + + .plutostate e.g. 0_20_13AbC...xyz.plutostate + + src/Export.jl:56 escapeuri(string(pluto_version, hash)), "." => "_" + src/PlutoHash.jl plutohash = base64urlencode . sha256 (of the file bytes) + +Both halves matter: + + * content hash -- edit a notebook and only that notebook re-runs. Normal, cheap. + * pluto version -- bump Pluto and EVERY entry is invalidated at once, so the + next build cold-runs all notebooks. That does not fit in 6h. This is the + case the script hard-fails on. + +Deliberately plain Python: no Julia, no Pkg.instantiate, runs in ~2 seconds. +Keep it in sync with PlutoSliderServer if that naming ever changes. +""" + +import base64 +import fnmatch +import hashlib +import pathlib +import re +import sys + +try: + import tomllib # stdlib since Python 3.11; ubuntu-latest runners ship 3.12+ +except ModuleNotFoundError: + sys.exit(f"FAIL: needs Python 3.11+ for tomllib, got {sys.version.split()[0]}") + +ROOT = pathlib.Path(__file__).resolve().parent.parent +MANIFEST = ROOT / "pluto-deployment-environment" / "Manifest.toml" +DEPLOY_TOML = ROOT / "pluto-deployment-environment" / "PlutoDeployment.toml" +SRC = ROOT / "src" +CACHE = ROOT / "_cache" + +PLUTO_HEADER = b"### A Pluto.jl notebook ###" + + +def pluto_version() -> str: + """Read the exact Pluto version the Manifest pins.""" + text = MANIFEST.read_text() + m = re.search(r'^\[\[deps\.Pluto\]\]$(?:(?!^\[\[).)*?^version = "([^"]+)"', + text, re.S | re.M) + if not m: + sys.exit(f"FAIL: no Pluto version found in {MANIFEST.relative_to(ROOT)}") + return m.group(1) + + +def ignored_globs() -> list[str]: + """Notebooks PlutoDeployment.toml tells PlutoSliderServer not to cache.""" + if not DEPLOY_TOML.is_file(): + return [] + with DEPLOY_TOML.open("rb") as f: + return tomllib.load(f).get("Export", {}).get("ignore_cache", []) + + +def is_notebook(path: pathlib.Path) -> bool: + """Same test PlutoPages uses: Pluto.is_pluto_notebook, i.e. the header.""" + with path.open("rb") as f: + return f.read(len(PLUTO_HEADER)) == PLUTO_HEADER + + +def statefile(path: pathlib.Path, prefix: str) -> str: + digest = hashlib.sha256(path.read_bytes()).digest() + h = base64.urlsafe_b64encode(digest).decode().rstrip("=") + return f"{prefix}{h}.plutostate" + + +def main() -> int: + version = pluto_version() + prefix = version.replace(".", "_") + skip = ignored_globs() + + notebooks = [ + p for p in sorted(SRC.rglob("*.jl")) + if is_notebook(p) + and not any(fnmatch.fnmatch(str(p.relative_to(SRC)), g) + or fnmatch.fnmatch(p.name, g) for g in skip) + ] + cache = {p.name for p in CACHE.glob("*.plutostate")} if CACHE.is_dir() else set() + want = {statefile(p, prefix): p for p in notebooks} + + missing = {fn: p for fn, p in want.items() if fn not in cache} + orphans = sorted(cache - want.keys()) + wrong_version = [fn for fn in cache if not fn.startswith(prefix)] + + print(f"Pluto {version} | {len(notebooks)} notebooks | {len(cache)} cache entries") + + # Total invalidation: a Pluto bump, or an empty cache. Unrecoverable in CI. + if cache and len(wrong_version) == len(cache): + sys.exit( + f"\nFAIL: every cache entry was built by a different Pluto version.\n" + f" Manifest pins Pluto {version}, so entries must start with '{prefix}'\n" + f" but _cache/ holds e.g. {sorted(wrong_version)[0]}\n\n" + f" Cold-running all {len(notebooks)} notebooks exceeds the 6h job limit.\n" + f" Either revert the Pluto bump, or regenerate the cache locally:\n" + f" rm -rf _cache && julia --project=pluto-deployment-environment generate.jl\n" + f" and commit the result. See website_maintenance.md." + ) + if not cache: + sys.exit( + f"\nFAIL: _cache/ is empty or missing.\n" + f" Cold-running all {len(notebooks)} notebooks exceeds the 6h job limit.\n" + f" See website_maintenance.md for how to regenerate it." + ) + + for fn in orphans: + print(f" orphan {fn}") + for fn, p in sorted(missing.items(), key=lambda kv: str(kv[1])): + print(f" cold {p.relative_to(ROOT)}") + + if orphans: + print(f"\n{len(orphans)} orphaned entr{'y' if len(orphans) == 1 else 'ies'} " + f"(notebook edited or deleted). Safe to delete; do it with the next commit " + f"so _cache/ does not grow without bound.") + if missing: + print(f"\n{len(missing)} notebook(s) will run cold this build. That is expected " + f"after editing a notebook. Commit the regenerated _cache/ entries to keep " + f"later builds fast.") + if not missing and not orphans: + print("\nCache is complete and current.") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From 8052486ca950d1dc9ced4ed197058caac82977c5 Mon Sep 17 00:00:00 2001 From: DaanVanHauwermeiren Date: Wed, 5 Aug 2026 11:53:22 +0200 Subject: [PATCH 3/4] docs: document the _cache maintenance contract The existing note said outputs are cached "by the file hash" and omitted the Pluto version, which is the half that can break the build unrecoverably. Spell out both failure modes: bumping Pluto invalidates everything at once (with the exact recovery procedure), while bumping any other package invalidates nothing, so the site serves output from old package versions until someone forces a cold rebuild. --- website_maintenance.md | 69 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 68 insertions(+), 1 deletion(-) diff --git a/website_maintenance.md b/website_maintenance.md index cb3ddadf..f919fedb 100644 --- a/website_maintenance.md +++ b/website_maintenance.md @@ -36,7 +36,74 @@ Pluto notebooks will be rendered to HTML and included in the page. What you see On a separate system, we are running a PlutoSliderServer that is synchronized to the `Fall23` brach. This makes our notebooks interactive! -Notebook outputs are **cached** (for a long time) by the file hash. This means that a notebook file will only ever run once, which makes it much faster to work on the website. If you need to re-run your notebook, add a space somewhere in the code :) +Notebook outputs are **cached** (for a long time) by the file hash *and the Pluto version*. This means that a notebook file will only ever run once, which makes it much faster to work on the website. If you need to re-run your notebook, add a space somewhere in the code :) + +See [The notebook cache](#the-notebook-cache-_cache) below before you touch the Pluto version. + +## The notebook cache (`_cache/`) + +Running all notebooks from scratch takes well over six hours, which is more than a +GitHub Actions job is allowed to run. The build only finishes because `_cache/` is +committed to the repository, so every checkout starts warm. Treat that directory as +build infrastructure, not as generated junk. + +Each file is named by PlutoSliderServer as `.plutostate`, +for example `0_20_13AbC...xyz.plutostate`. Both halves matter, and they fail very +differently. + +### Editing a notebook: nothing to worry about + +The content hash changes, so CI re-runs that one notebook (a few minutes) and +everything else is served from cache. The old `.plutostate` is left behind as an +orphan. Regenerate locally and commit the new cache entries, deleting the orphans, so +the directory does not grow without bound: + +``` +julia --project=pluto-deployment-environment generate.jl +python3 tools/check_cache.py # lists orphans and notebooks that would run cold +``` + +### ⚠️ Bumping Pluto: read this first + +A new Pluto version invalidates **every** entry at once, because the version is part +of every filename. The next build then cold-runs all notebooks, exceeds the job +limit, and gets killed. That is how `main` broke before PR #81, and it does not +recover on its own. + +`Pluto` and `PlutoSliderServer` are therefore pinned with `=` in +`pluto-deployment-environment/Project.toml`, so `Pkg.update()` cannot do this by +accident. To bump on purpose: + +1. Edit the `[compat]` pins, then `Pkg.update()` in that environment. +2. Locally: `rm -rf _cache && julia --project=pluto-deployment-environment generate.jl`. + Budget several hours. +3. Commit the regenerated `_cache/` **in the same commit** as the `Manifest.toml` + change. The two must never be out of step. +4. `python3 tools/check_cache.py` must pass before you push. + +`pluto-deployment-environment/Manifest.toml` is the only thing that pins Pluto, so it +is force-tracked in `.gitignore`. Do not untrack it. + +### The inverse problem: stale output + +Because the cache key is the notebook's *content*, bumping any other package +(ModelingToolkit, Turing, ...) does **not** invalidate anything. The site keeps +serving output produced by the old versions until a notebook is edited. After a +significant SciML upgrade, force a full refresh by hand: + +``` +rm -rf _cache && julia --project=pluto-deployment-environment generate.jl +``` + +This also proves every notebook still runs, which a warm build never checks. Worth +doing once a term. + +### The preflight check + +`ExportNotebooks.yml` runs `tools/check_cache.py` as a separate `cache-preflight` +job before the expensive one. It fails in seconds when the cache cannot serve the +build (Pluto bump, empty `_cache/`) instead of after six hours of runner time. It +only warns for the normal cases, edited notebooks and orphans. Run it locally too. ## `.css`, `.html`, `.gif`, etc From 464840920df31f2dcbbfb4487eda7e07040a518b Mon Sep 17 00:00:00 2001 From: DaanVanHauwermeiren Date: Wed, 5 Aug 2026 12:02:36 +0200 Subject: [PATCH 4/4] build: pin .jl to LF so the cache survives a Windows checkout _cache/ entries are keyed by sha256 of the notebook's exact bytes. With only "* text=auto", .jl files check out as CRLF on Windows (eol falls back to core.eol=native), so every hash differs and all 46 entries miss. A cache regenerated there and committed would then miss on CI, reintroducing the 6h timeout. requirements.txt is already committed with CRLF, so this is a real configuration in use, not a hypothetical. All 209 tracked .jl files are already LF in the index: renormalizes nothing, invalidates no cache entry. --- .gitattributes | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/.gitattributes b/.gitattributes index bb05dd6c..19b79535 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,2 +1,16 @@ page/* linguist-vendored -* text=auto \ No newline at end of file +* text=auto + +# Notebooks are keyed into _cache/ by sha256 of their exact bytes, so a CRLF +# checkout (git's default on Windows, since `text=auto` above means the eol +# attribute falls back to core.eol=native) changes every hash and misses every +# cache entry, cold-running the whole site. Pin them to LF everywhere. +# All 209 tracked .jl files are already LF in the index, so this renormalizes +# nothing and invalidates no cache entry. +*.jl text eol=lf + +# Cached notebook states are binary. git already detects that from content +# today, but that is a heuristic over the first 8000 bytes: a mostly-ASCII +# state could flip to "text" and get line-ending mangled on checkout, silently +# corrupting the cache. Say it explicitly instead of relying on the guess. +*.plutostate binary