Refresh colab snapshot - #211
Merged
Merged
Conversation
Colab rebuilds its runtime image every week or two, and the notebooks' install cells are locked against a snapshot of its preinstalled versions. Once the snapshot is stale, the install cell downgrades the packages Colab has since bumped, which is slow and forces a runtime restart. refresh_colab_snapshot.py rewrites .github/colab-preinstalled.txt from googlecolab/backend-info and re-locks every CI-tested notebook. A new workflow runs it on Mondays and opens or updates one PR when the snapshot has drifted. lock_notebook.py now hands the notebook's current pins to uv as the previous lock, so a re-lock moves only what the snapshot or requirements.in forces. Packages Colab does not ship (dandi, pynwb, ...) stay at the versions the notebook was tested with. --upgrade restores the old behavior of resolving everything afresh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Colab's image has changed several times since the 2026-08-22 snapshot, most recently with the move to Ubuntu 24.04 on 2026-09-04, and 88 preinstalled packages now differ. Install cells were pulling click, cryptography, fsspec, pydantic, and others back to the August versions. This refreshes the snapshot and re-locks the 44 CI-tested notebooks. Every changed pin matches Colab's current version; no package outside the snapshot moved. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Contributor
|
Preview for this PR has been removed (PR closed). |
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Colab's default image has changed several times since the snapshot in
.github/colab-preinstalled.txtwas last refreshed on 2026-08-22, most recently with the move to Ubuntu 24.04 on 2026-09-04. 88 preinstalled packages now differ, so the install cells were pulling click, cryptography, fsspec, pydantic, and others back to the August versions on every run. In 001172/HigleyLab that was 11 downgrades.Snapshot Refresh and Re-Lock
The snapshot is refreshed from googlecolab/backend-info and the 44 CI-tested notebooks are re-locked against it. Every pin that changed matches Colab's current version, and no package outside the snapshot moved. Notebooks excluded from CI are left alone because their pins cannot be verified after a re-lock. Two of them (
validate_lev6and the cosyneadvanced_asset_search) holdclick==8.1.8for the dandi releases they pin, and re-locking them would have moved dandi to 0.80 and pynwb to 4.2.Re-Locks Keep Current Pins
lock_notebook.pynow hands the notebook's current pins to uv as the previous lock, so a re-lock moves only what the snapshot orrequirements.inforces. Before this change a re-lock also floated packages Colab does not ship (dandi 0.77 to 0.80, numcodecs, deno, blessed), which would have turned each snapshot refresh into an untested upgrade.--upgraderestores the old behavior of resolving everything afresh.Weekly Refresh Job
.github/workflows/refresh-colab-snapshot.ymlrunsrefresh_colab_snapshot.py --relockon Mondays at 05:00 UTC, an hour before the weekly test run, and on manual dispatch. When the snapshot has drifted it pushes tobot/refresh-colab-snapshotand opens or updates one PR that lists the changed packages.A PR opened with the default
GITHUB_TOKENdoes not trigger "Test changed notebooks". The workflow uses aCOLAB_REFRESH_TOKENsecret when one exists and falls back to the default token. Without the secret, the bot PR needs a close and reopen to get CI.The workflow has not run yet. I would dispatch it once after merge to confirm it reports no drift.
🤖 Generated with Claude Code