Skip to content

Build datasets in notebooks without ipywidgets; fix datasets overview example - #1260

Open
solarsys wants to merge 1 commit into
sunlabuiuc:masterfrom
solarsys:fix/dask-progress-without-ipywidgets-pr
Open

solarsys wants to merge 1 commit into
sunlabuiuc:masterfrom
solarsys:fix/dask-progress-without-ipywidgets-pr

Conversation

@solarsys

@solarsys solarsys commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator

Problem

When a dataset builds its event cache in a Jupyter kernel, BaseDataset calls dask's progress(). In a kernel, dask draws an ipywidgets progress bar, but ipywidgets is not a PyHealth dependency. In a notebook without it, such as a plain venv kernel in VS Code or Cursor, building any dataset fails:

ModuleNotFoundError: No module named 'ipywidgets'
Error during caching, removing incomplete file …/global_event_df.parquet

The same unguarded call is in the 2.0.2 release.

Changes

  • base_dataset.py:
    • When ipywidgets isn't importable, pass notebook=False to dask's progress(), so dask falls back to its text bar. With ipywidgets installed, behaviour is unchanged.
    • In notebooks, where dask uses threaded workers, wait 0.3 s after the compute before closing the cluster. Without this, closing the cluster races dask's 100 ms progress feed and logs a spurious CancelledError traceback, even though the data was written correctly. It's a pragmatic workaround and only runs in notebooks; happy to replace it if there's a cleaner hook.
  • tests/core/test_base_dataset.py: simulates a kernel without ipywidgets, as in Jupyter (threaded workers, kernel detection on). It asserts that the build succeeds and that the dask scheduler logs no errors. The test fails on master with the ModuleNotFoundError above and passes with this change.
  • examples/datasets_overview.ipynb: the synthetic-samples cell called SampleDataset(samples=…). That constructor no longer exists; the committed output already showed an error, and running it today raises TypeError: … missing … 'path'. It now uses create_sample_dataset(...), and its stale error output is cleared. The new cell runs and prints Loaded 2 synthetic samples. No other cell changed.
  • docs/install.rst: a note on notebooks and the optional ipywidgets, under Platform-Specific Notes.

Verification

🤖 Generated with Claude Code

… example

In a Jupyter kernel, dask's progress() draws an ipywidgets bar. ipywidgets
is not a PyHealth dependency, so building any dataset's event cache in a
notebook without it (e.g. a plain venv kernel in VS Code or Cursor) failed
with ModuleNotFoundError and deleted the half-built cache.

- base_dataset: pass notebook=False to dask's progress() when ipywidgets
  is not importable, so dask falls back to its text bar.
- In notebooks (threaded workers), wait 0.3 s after the compute so closing
  the cluster does not race dask's 100 ms progress feed and log a
  spurious CancelledError traceback.
- tests: simulate a kernel without ipywidgets; the build succeeds and the
  scheduler logs no errors (fails on master with ModuleNotFoundError).
- examples/datasets_overview.ipynb: the synthetic-samples cell called
  SampleDataset(samples=...), which no longer exists; use
  create_sample_dataset. Its stale error output is cleared.
- docs/install.rst: note on notebooks and the optional ipywidgets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@solarsys
solarsys requested a review from jhnwu3 October 2, 2026 12:54

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant