Skip to content

Commit 74c40c1

Browse files
committed
Simplify workflows and clean up after diff-check removal
execute.yaml: - Drop fetch-depth: 0 (no longer needed; tj-actions/changed-files fetches the right base internally). - Add pip cache via setup-python (saves >1 min per PR run on the squidpy + scanpy install). - Stable pooch cache key (was hashFiles('**/*.ipynb') which invalidated the dataset cache on every notebook edit; bump v* suffix when adding a new dataset URL). - Replace hand-rolled `git diff | grep` notebook detection with tj-actions/changed-files. Cuts ~15 lines of brittle bash and handles rename/move events correctly. - Drop the trailing "we do NOT diff" comment block; the rationale lives in commit history, not in the YAML. preview.yaml: drop fetch-depth: 0 on the lib clone (only HEAD is built). strip_widget_metadata.py: docstring referenced the removed diff check in execute.yaml; rewrite to the actual remaining justification (small committed notebooks, clean PR diffs). .pre-commit-config.yaml: drop redundant `require_serial: false` (default value).
1 parent 744e83c commit 74c40c1

4 files changed

Lines changed: 23 additions & 25 deletions

File tree

.github/workflows/execute.yaml

Lines changed: 19 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ name: Execute notebooks
22

33
on:
44
schedule:
5-
# Weekly: Mondays 04:00 UTC re-executes ALL notebooks against latest releases.
5+
# Weekly: Mondays 04:00 UTC; re-executes ALL notebooks against latest releases.
66
- cron: "0 4 * * 1"
77
pull_request:
88
branches: [main]
@@ -22,19 +22,21 @@ jobs:
2222
timeout-minutes: 60
2323
steps:
2424
- uses: actions/checkout@v5
25-
with:
26-
# Need full history so we can diff PR HEAD against the merge base.
27-
fetch-depth: 0
2825

2926
- uses: actions/setup-python@v5
3027
with:
3128
python-version: "3.13"
29+
cache: pip
30+
cache-dependency-path: pyproject.toml
3231

3332
- name: Cache pooch datasets
3433
uses: actions/cache@v4
3534
with:
3635
path: ~/.cache/pooch
37-
key: pooch-${{ runner.os }}-${{ hashFiles('**/*.ipynb') }}
36+
# Datasets are pinned by URL inside notebook code, not by the
37+
# notebook's surrounding markdown — bump the v* suffix when an
38+
# actually-new dataset URL lands.
39+
key: pooch-${{ runner.os }}-v1
3840
restore-keys: |
3941
pooch-${{ runner.os }}-
4042
@@ -43,15 +45,22 @@ jobs:
4345
pip install --upgrade pip
4446
pip install -e ".[exec]"
4547
48+
- name: Detect changed notebooks (PR only)
49+
if: github.event_name == 'pull_request'
50+
id: changed
51+
uses: tj-actions/changed-files@v45
52+
with:
53+
files: |
54+
tutorials/**/*.ipynb
55+
examples/**/*.ipynb
56+
4657
- name: Determine notebooks to execute
4758
id: pick
4859
run: |
49-
# On the weekly schedule and manual dispatch, run all notebooks.
50-
# On PR, run only notebooks the PR touched, to keep CI fast as the
51-
# gallery grows. (sklearn does the same.)
5260
if [ "${{ github.event_name }}" = "pull_request" ]; then
53-
base="${{ github.event.pull_request.base.sha }}"
54-
nbs=$(git diff --name-only --diff-filter=AMR "$base"...HEAD -- '*.ipynb' | grep -E '^(tutorials|examples)/' || true)
61+
nbs="${{ steps.changed.outputs.all_changed_files }}"
62+
# tj-actions emits space-separated; one-per-line for the loop below.
63+
nbs=$(printf '%s\n' $nbs)
5564
else
5665
nbs=$(find tutorials examples -name "*.ipynb" -not -path "*/.ipynb_checkpoints/*" 2>/dev/null || true)
5766
fi
@@ -75,10 +84,3 @@ jobs:
7584
echo "Executing $nb"
7685
jupyter nbconvert --to notebook --execute --inplace "$nb"
7786
done <<< "${{ steps.pick.outputs.files }}"
78-
79-
# Note: we do NOT diff re-executed outputs against committed bytes.
80-
# Matplotlib PNG bytes vary across machines (font hinting, dpi),
81-
# cell execution timestamps always differ, and Python micro-version
82-
# leaks into metadata.language_info — none of which are meaningful
83-
# regressions. The real test is `nbconvert --execute` succeeding
84-
# without raising on any cell, which the step above enforces.

.github/workflows/preview.yaml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,6 @@ jobs:
3333
with:
3434
repository: scverse/spatialdata-plot
3535
path: lib
36-
fetch-depth: 0
3736

3837
- name: Mount PR notebooks into lib's submodule path
3938
run: |

.pre-commit-config.yaml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,4 +59,3 @@ repos:
5959
entry: python scripts/strip_widget_metadata.py
6060
language: system
6161
files: \.ipynb$
62-
require_serial: false

scripts/strip_widget_metadata.py

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,10 @@
11
#!/usr/bin/env python3
22
"""Strip Jupyter widget metadata + outputs from .ipynb files.
33
4-
Widget UUIDs are regenerated on every execution, so any notebook that runs
5-
something using `tqdm.notebook` (e.g. `pooch` downloads, `scanpy` progress
6-
bars in a Jupyter kernel) drifts on every re-execution. Stripping widgets
7-
makes the diff-against-committed check in `execute.yaml` deterministic, and
8-
keeps committed notebooks reproducible without losing the visible outputs
9-
(figures, repr cells, prints).
4+
Anything using `tqdm.notebook` (pooch downloads, scanpy progress bars in a
5+
Jupyter kernel) emits widget output blobs whose UUIDs regenerate on every
6+
execution. Stripping them keeps committed notebooks small and produces
7+
clean PR diffs that reflect real source changes only.
108
119
Usage: strip_widget_metadata.py <notebook> [<notebook> ...]
1210
Exits 0 if no changes, 1 if files were modified (so pre-commit reports the

0 commit comments

Comments
 (0)