Skip to content

Refresh colab snapshot - #211

Merged
bendichter merged 2 commits into
masterfrom
refresh-colab-snapshot
Sep 21, 2026
Merged

bendichter merged 2 commits into
masterfrom
refresh-colab-snapshot

Conversation

@bendichter

@bendichter bendichter commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

Colab's default image has changed several times since the snapshot in .github/colab-preinstalled.txt was last refreshed on 2026-08-22, most recently with the move to Ubuntu 24.04 on 2026-09-04. 88 preinstalled packages now differ, so the install cells were pulling click, cryptography, fsspec, pydantic, and others back to the August versions on every run. In 001172/HigleyLab that was 11 downgrades.

Snapshot Refresh and Re-Lock

The snapshot is refreshed from googlecolab/backend-info and the 44 CI-tested notebooks are re-locked against it. Every pin that changed matches Colab's current version, and no package outside the snapshot moved. Notebooks excluded from CI are left alone because their pins cannot be verified after a re-lock. Two of them (validate_lev6 and the cosyne advanced_asset_search) hold click==8.1.8 for the dandi releases they pin, and re-locking them would have moved dandi to 0.80 and pynwb to 4.2.

Re-Locks Keep Current Pins

lock_notebook.py now hands the notebook's current pins to uv as the previous lock, so a re-lock moves only what the snapshot or requirements.in forces. Before this change a re-lock also floated packages Colab does not ship (dandi 0.77 to 0.80, numcodecs, deno, blessed), which would have turned each snapshot refresh into an untested upgrade. --upgrade restores the old behavior of resolving everything afresh.

Weekly Refresh Job

.github/workflows/refresh-colab-snapshot.yml runs refresh_colab_snapshot.py --relock on Mondays at 05:00 UTC, an hour before the weekly test run, and on manual dispatch. When the snapshot has drifted it pushes to bot/refresh-colab-snapshot and opens or updates one PR that lists the changed packages.

A PR opened with the default GITHUB_TOKEN does not trigger "Test changed notebooks". The workflow uses a COLAB_REFRESH_TOKEN secret when one exists and falls back to the default token. Without the secret, the bot PR needs a close and reopen to get CI.

The workflow has not run yet. I would dispatch it once after merge to confirm it reports no drift.

🤖 Generated with Claude Code

bendichter and others added 2 commits September 21, 2026 15:14
Colab rebuilds its runtime image every week or two, and the notebooks'
install cells are locked against a snapshot of its preinstalled versions.
Once the snapshot is stale, the install cell downgrades the packages Colab
has since bumped, which is slow and forces a runtime restart.

refresh_colab_snapshot.py rewrites .github/colab-preinstalled.txt from
googlecolab/backend-info and re-locks every CI-tested notebook. A new
workflow runs it on Mondays and opens or updates one PR when the snapshot
has drifted.

lock_notebook.py now hands the notebook's current pins to uv as the
previous lock, so a re-lock moves only what the snapshot or
requirements.in forces. Packages Colab does not ship (dandi, pynwb, ...)
stay at the versions the notebook was tested with. --upgrade restores the
old behavior of resolving everything afresh.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Colab's image has changed several times since the 2026-08-22 snapshot,
most recently with the move to Ubuntu 24.04 on 2026-09-04, and 88
preinstalled packages now differ. Install cells were pulling click,
cryptography, fsspec, pydantic, and others back to the August versions.

This refreshes the snapshot and re-locks the 44 CI-tested notebooks.
Every changed pin matches Colab's current version; no package outside
the snapshot moved.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

github-actions Bot added a commit that referenced this pull request Sep 21, 2026
@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Preview for this PR has been removed (PR closed).

github-actions Bot added a commit that referenced this pull request Sep 21, 2026
@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Executed notebooks

CI executes the notebooks changed in this PR through a Jupyter kernel and publishes each one, with its outputs, as soon as it finishes. Links appear below as notebooks complete; an hourglass means the notebook is still running.

44 of 44 finished. Commit 2e8dc6c. The executed copies live only in the preview and are removed when the PR closes; nothing is committed to the branch.

github-actions Bot added a commit that referenced this pull request Sep 21, 2026
github-actions Bot added a commit that referenced this pull request Sep 21, 2026
github-actions Bot added a commit that referenced this pull request Sep 21, 2026
github-actions Bot added a commit that referenced this pull request Sep 21, 2026
github-actions Bot added a commit that referenced this pull request Sep 21, 2026
github-actions Bot added a commit that referenced this pull request Sep 21, 2026
@bendichter
bendichter merged commit 96dc9a6 into master Sep 21, 2026
48 checks passed
github-actions Bot added a commit that referenced this pull request Sep 21, 2026
github-actions Bot added a commit that referenced this pull request Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant