Execute a deterministic public-notebook smoke suite across supported Python versions

[Alek99]

Summary

The repository publishes 24 notebooks, including six README-linked multi-million-row examples, but CI never executes a notebook top to bottom. PDSH tests validate only JSON/paired-cell structure, and Binder CI only builds an image without launching it. This leaves notebook API drift, missing runtime dependencies, stale outputs, remote-source changes, and supported-Python regressions unchecked.

Evidence

• README directly promotes the six real-world notebooks:

  xy/README.md

  Lines 258 to 269 in 99eda6d

  	## Examples
  	Each notebook fetches its rows from the linked public source; no raw datasets
  	are stored in this repository. Counts describe the featured chart, and the
  	notebooks scale further. See the
  	[example guide](https://github.com/reflex-dev/xy/blob/main/examples/real_world/README.md) for sources, workload controls,
  	and setup.
  	| | | |
  	| :---: | :---: | :---: |
  	| **Gaia DR3 · HR diagram**<br><sub>250,000 plotted stars</sub><br><br><br><br>[Open notebook](https://github.com/reflex-dev/xy/blob/main/examples/real_world/01_gaia_hr_diagram.ipynb) | **gnomAD v4.1 · allele frequency**<br><sub>164,000 plotted variants</sub><br><br><br><br>[Open notebook](https://github.com/reflex-dev/xy/blob/main/examples/real_world/02_gnomad_allele_frequency.ipynb) | **Pan-UKBB · Manhattan plot**<br><sub>814,294 plotted variants</sub><br><br><br><br>[Open notebook](https://github.com/reflex-dev/xy/blob/main/examples/real_world/03_pan_ukbb_manhattan.ipynb) |
  	| **Dukascopy · EUR/USD ticks**<br><sub>101,427 plotted ticks</sub><br><br><br><br>[Open notebook](https://github.com/reflex-dev/xy/blob/main/examples/real_world/04_dukascopy_fx_ticks.ipynb) | **LIGO · GW150914 strain**<br><sub>16,777,216 raw · 3,441 shown</sub><br><br><br><br>[Open notebook](https://github.com/reflex-dev/xy/blob/main/examples/real_world/05_ligo_gw150914_strain.ipynb) | **NYC TLC · taxi pickup density**<br><sub>300,000 pickup records</sub><br><br><br><br>[Open notebook](https://github.com/reflex-dev/xy/blob/main/examples/real_world/06_nyc_taxi_density.ipynb) |

• Their guide says each notebook downloads public data, with defaults up to millions of rows/16.8M samples:

  xy/examples/real_world/README.md

  Lines 1 to 20 in 99eda6d

  	# Real-world large-data notebooks
  	These six notebooks use public datasets large enough to exercise XY's density,
  	decimation, and faceting paths with recognizable scientific and operational
  	data:
  	| Notebook | Dataset | Default workload | XY path |
  	| --- | --- | ---: | --- |
  	| `01_gaia_hr_diagram.ipynb` | [ESA Gaia DR3](https://www.cosmos.esa.int/web/gaia-users/archive/programmatic-access) | 1M stars | scatter density |
  	| `02_gnomad_allele_frequency.ipynb` | [gnomAD v4.1 genomes](https://gnomad.broadinstitute.org/news/2024-04-gnomad-v4-1/) | up to 3.52M variants | scatter density + log axis |
  	| `03_pan_ukbb_manhattan.ipynb` | [Pan-UKBB standing-height GWAS](https://pan.ukbb.broadinstitute.org/downloads/index.html) | up to 2.64M variants | scatter density |
  	| `04_dukascopy_fx_ticks.ipynb` | [Dukascopy EUR/USD ticks](https://www.dukascopy.com/swiss/english/marketwatch/historical/) | five calendar days | M4 line decimation |
  	| `05_ligo_gw150914_strain.ipynb` | [GWOSC GW150914 strain](https://gwosc.org/events/GW150914/) | 16.8M samples | M4 line decimation |
  	| `06_nyc_taxi_density.ipynb` | [NYC TLC yellow-taxi pickups](https://data.cityofnewyork.us/d/2yzn-sicd) | 1M trips | density, hexbin, 24 facets |
  	## Screenshots
  	These are rendered chart outputs, not copies of the source datasets. The PNGs
  	below are the only data-derived artifacts checked in; each notebook downloads
  	its working rows from the linked public source into the git-ignored cache.

  and

  xy/examples/real_world/README.md

  Lines 65 to 92 in 99eda6d

  	## Setup
  	From a Python 3.11+ environment:
  	```bash
  	python -m pip install xy jupyter numpy requests pysam h5py gwosc
  	jupyter lab
  	```
  	Open this directory in Jupyter and run a notebook from top to bottom. Each
  	notebook documents its official source, attribution notes, expected download,
  	and environment variables for scaling the workload.
  	Downloads and indexes are cached under `data/` by default. Set
  	`XY_REAL_WORLD_DATA=/absolute/path` to use a shared cache outside the checkout.
  	The directory is git-ignored.
  	Start with a smaller remote sample when checking connectivity:
  	```bash
  	GNOMAD_CHROMOSOMES=22 GNOMAD_WINDOWS=2 jupyter lab
  	PANUKBB_WINDOWS=2 PANUKBB_VARIANTS_PER_WINDOW=2000 jupyter lab
  	TLC_MAX_ROWS=1000000 jupyter lab
  	```
  	The notebooks intentionally keep acquisition separate from visualization.
  	Once a dataset is cached, chart cells can be rerun and restyled without another
  	download.

• The only notebook-specific test checks paired cell metadata/source shape; it never executes cells:

  xy/tests/test_pdsh_dual_engine_notebooks.py

  Lines 12 to 50 in 99eda6d

  	def test_pdsh_notebooks_contain_fresh_matplotlib_and_xy_sections() -> None:
  	subprocess.run(
  	[sys.executable, str(PDSH / "sync_dual_engine_notebooks.py"), "--check"],
  	cwd=ROOT,
  	check=True,
  	)
  	paths = sorted(PDSH.glob("pdsh_*.ipynb"))
  	assert len(paths) == 14
  	for path in paths:
  	notebook = json.loads(path.read_text(encoding="utf-8"))
  	code_cells = [cell for cell in notebook["cells"] if cell.get("cell_type") == "code"]
  	engines = [cell.get("metadata", {}).get("xy_pdsh_engine") for cell in code_cells]
  	assert "matplotlib" in engines and "xy" in engines
  	assert len(code_cells) % 2 == 0
  	assert all(
  	engines[index : index + 2] == ["matplotlib", "xy"]
  	for index in range(0, len(engines), 2)
  	)
  	assert all(
  	code_cells[index]["metadata"]["xy_pdsh_source_id"]
  	== code_cells[index + 1]["metadata"]["xy_pdsh_source_id"]
  	for index in range(0, len(code_cells), 2)
  	)
  	mpl_code = "\n".join(
  	"".join(cell.get("source", []))
  	for cell in notebook["cells"]
  	if cell.get("metadata", {}).get("xy_pdsh_engine") == "matplotlib"
  	and cell.get("cell_type") == "code"
  	)
  	xy_code = "\n".join(
  	"".join(cell.get("source", []))
  	for cell in notebook["cells"]
  	if cell.get("metadata", {}).get("xy_pdsh_engine") == "xy"
  	and cell.get("cell_type") == "code"
  	)
  	assert "xy.pyplot" not in mpl_code
  	assert "matplotlib" in mpl_code
  	assert "xy.pyplot" in xy_code
  	assert "xy_plt.show()" not in xy_code

• Binder CI explicitly builds with --no-run:

  xy/.github/workflows/binder.yml

  Lines 50 to 56 in 99eda6d

  	# Build the image exactly as mybinder.org would — conda solve of
  	# .binder/environment.yml, then the postBuild source build of the
  	# native core and render client — without launching it. The tool is
  	# pinned so the check stays deterministic across PRs; bump it
  	# deliberately when mybinder's deployment moves.
  	- name: Build the Binder image (no launch)
  	run: uvx --from jupyter-repo2docker==2026.4.0 jupyter-repo2docker --no-run --image-name xy-binder-ci .

• The package claims Python >=3.11, but the only explicitly versioned full library suite is the 3.11 floor job rather than a declared supported-version matrix:

  xy/pyproject.toml

  Lines 5 to 15 in 99eda6d

  	[project]
  	name = "xy"
  	dynamic = ["version"]
  	description = "Experimental Python charting engine with a native Rust core, binary columnar transport, and a GPU render client"
  	readme = "README.md"
  	requires-python = ">=3.11"
  	license = { text = "Apache-2.0" }
  	dependencies = [
  	"numpy>=1.24",
  	"anywidget>=0.9",
  	]

  and

  xy/.github/workflows/ci.yml

  Lines 275 to 320 in 99eda6d

  	python_floor:
  	name: Python 3.11 floor
  	runs-on: ubuntu-latest
  	steps:
  	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
  	with:
  	# Full history and tags: the distribution version is derived
  	# from the latest `v*` tag, and a shallow clone has none.
  	fetch-depth: 0
  	- uses: dtolnay/rust-toolchain@4be7066ada62dd38de10e7b70166bc74ed198c30 # stable
  	- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
  	with:
  	python-version: "3.11"
  	- uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
  	with:
  	enable-cache: false
  	- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
  	with:
  	node-version: "22"
  	- name: Verify syntax/annotation floor
  	run: python scripts/check_python_floor.py
  	- name: Build native core
  	run: cargo build --release
  	# The render client is a generated artifact (§33): build it so the full
  	# test suite (which reads python/xy/static/*.js) has it on disk.
  	- name: Build render client
  	run: |
  	npm ci
  	node js/build.mjs
  	- name: Install package for Python 3.11
  	run: |
  	python -VV
  	# Absolute path: newer uv resolves a bare name as a directory, not
  	# the interpreter setup-python put on PATH.
  	uv venv .venv --python "$(command -v python)"
  	uv pip install -p .venv/bin/python -e . "pytest>=8,<9"
  	- name: Public API coherence
  	run: .venv/bin/python scripts/check_public_api.py
  	- name: Python 3.11 tests (native core)
  	run: .venv/bin/python -m pytest -q

Acceptance criteria

• Define a CI notebook profile with reduced/offline fixtures, deterministic seeds, bounded time/memory, and no dependence on mutable remote data.
• Execute a representative public set (basic XY, matplotlib shim/PDSH, and one reduced real-world path) top to bottom on the Python floor and newest supported stable version on every PR.
• Run the full supported-Python/notebook matrix on a scheduled or release gate, with a separately labeled network-backed acquisition check where needed.
• Fail on cell errors, missing dependencies, stale/incompatible kernelspecs, or unintended output drift.
• Binder validation launches the built image and executes at least the starter notebook, and workflow path filters include notebook/dependency/config changes.