Guard the notebook cache: pin Pluto, fail fast on drift, document it - #82
Open
DaanVanHauwermeiren wants to merge 4 commits into
Open
Guard the notebook cache: pin Pluto, fail fast on drift, document it#82DaanVanHauwermeiren wants to merge 4 commits into
DaanVanHauwermeiren wants to merge 4 commits into
Conversation
_cache/*.plutostate filenames embed the Pluto version, so any Pluto bump invalidates all 46 entries at once and the next build cold-runs every notebook, which exceeds the 6h job limit. Project.toml had no [compat] section at all, so a routine Pkg.update() could trigger this silently. Pin Pluto and PlutoSliderServer with "=" to make the bump a deliberate, reviewable change, and force-track the deployment Manifest, which is the only thing recording which version the cache was built with. Two edits. [compat] added to pluto-deployment-environment/Project.toml (verified it still parses; 38 deps intact), and a trailing !pluto-deployment-environment/Manifest.toml in .gitignore. Placement matters — git applies the last matching rule and there are four Manifest.toml lines, the last at 693, so the negation has to sit at the end of the file. Verified both directions: git check-ignore pluto-deployment-environment/Manifest.toml -> exit 1 (tracked) ✅ git check-ignore exercises/solved_notebooks/Manifest.toml -> .gitignore:693 ✅
A stale cache was only discovered by a job that ran for six hours and then got killed, leaving no usable cache behind, so the next push failed the same way. Add tools/check_cache.py, which reproduces PlutoSliderServer's cache filename (pluto version + base64url(sha256(notebook))) in plain Python, and run it as a cache-preflight job gating build-and-deploy. Hard-fails on total invalidation (Pluto bump, empty _cache); only warns for edited notebooks and orphans, which are normal and cheap. Runs in ~2s, no Julia needed. The guard detects notebooks by the ### A Pluto.jl notebook ### header rather than by path — same test PlutoPages uses. That returns exactly 46, and stays right if someone adds a plain .jl helper under src/. It also honours ignore_cache from PlutoDeployment.toml.
The existing note said outputs are cached "by the file hash" and omitted the Pluto version, which is the half that can break the build unrecoverably. Spell out both failure modes: bumping Pluto invalidates everything at once (with the exact recovery procedure), while bumping any other package invalidates nothing, so the site serves output from old package versions until someone forces a cold rebuild.
|
Once the build has completed, you can preview your PR at this URL: https://Kermit-UGent.github.io/ModSim/previews/PR82/ |
_cache/ entries are keyed by sha256 of the notebook's exact bytes. With only "* text=auto", .jl files check out as CRLF on Windows (eol falls back to core.eol=native), so every hash differs and all 46 entries miss. A cache regenerated there and committed would then miss on CI, reintroducing the 6h timeout. requirements.txt is already committed with CRLF, so this is a real configuration in use, not a hypothetical. All 209 tracked .jl files are already LF in the index: renormalizes nothing, invalidates no cache entry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.