Build and test conda-recipe-cf for Intel Python CI by jharlow-intel · Pull Request #204 · IntelPython/mkl_random

jharlow-intel · 2026-10-23T20:23:40Z

Description

Adds what Intel Python CI (gh-ci) needs from mkl_random: a ci/ directory and a new workflow, .github/workflows/intel-ci.yml. The workflow builds conda-recipe-cf for every platform and Python version the recipe declares, and tests each package in a clean conda-forge environment with ci/test.py. Intel's Jenkins then waits for that run, tests each package again on its own machines, scans it, and for a release tag publishes the packages it tested. Jenkins isn't switched on for mkl_random yet, so until it is, this only adds a GitHub Actions workflow.

ci/matrix.py, ci/build.py, ci/hooks.py: the shared gh-ci scripts, standard library only. Here they look for conda-recipe-cf/ first and build Linux and Windows on GitHub-hosted runners. hooks.py installs the test requirements from ci/config.yml, the way Jenkins does.
ci/test.py: runs pytest --pyargs mkl_random against the installed package and writes a JUnit report.
ci/benchmark.py: because benchmarks/ exists, Jenkins runs it once per pull request or tag, with the newest regular Linux package, on a bare-metal benchmarking machine. It runs the asv suite against the installed package: one quick sample of each benchmark on a pull request, a full measurement on a tag. A failing benchmark fails it; nothing compares the timings with a baseline yet.
ci/config.yml: the workflow, the artifact names (mkl_random {platform} Python {python}, as conda-package-cf.yml names them) and the test requirements, pytest, and the benchmark requirements, psutil, with asv from PyPI, as conda-forge has no asv for Python 3.15 yet.
conda-recipe-cf/conda_build_config.yaml: now lists the Python versions, the ones conda-package-cf.yml builds today, for ci/matrix.py, and numpy: '2'. Without a NumPy variant, conda-build falls back to NumPy 1.26, which has no builds for Python 3.13 and newer. conda-package-cf.yml passes --python and --numpy itself, so it is unaffected.
ci/README.md and AGENTS.md notes.

A pull request builds the oldest and the newest regular Python versions on Linux and Windows; pushes to master and tags build them all.

Miniforge is pinned to 26.7.2-0. Miniforge 26.9.4-0, released on 22 October, makes setup-miniconda's conda init fail on Windows runners, and the existing conda-package*.yml workflows, which install latest, hit it too on this pull request.

Overlap with #202. It also sets numpy: '2' in conda_build_config.yaml and drops Python 3.10, so whichever lands second needs a small rebase, and if 3.10 goes it should leave the python: list here too. conda-package-cf.yml and intel-ci.yml now both build conda-recipe-cf; retiring one can be a follow-up.

Verification

Tests: this pull request's Intel Python CI run built Linux and Windows for Python 3.10 and 3.15, the oldest and the newest, and ran ci/test.py against each package: 380 passed on each platform and version.
Lint: pre-commit run on every changed file, all hooks passing, and zizmor on intel-ci.yml, no findings.
python ci/matrix.py lists 14 builds, Linux and Windows for 3.10 to 3.15 and 3.14t; with --event pull_request it lists 3.10 and 3.15 on both.
ci/config.yml loads with gh-ci's own parser.

Not verified

The builds a tag or a push to the default branch adds, Python 3.11 to 3.14 and 3.14t; they use the same jobs.
The Jenkins side, which isn't enabled for mkl_random yet, the benchmark cell included.
ci/benchmark.py against mkl_random's suite. The same script ran mkl_fft's.

Adds what Intel Python CI (gh-ci) needs from this repository: ci/ with the shared matrix, build and test-hook scripts, ci/test.py and ci/config.yml, and .github/workflows/intel-ci.yml, which builds conda-recipe-cf for every platform and Python version the recipe declares and tests each package in a clean conda-forge environment, as Jenkins later does. conda-recipe-cf/conda_build_config.yaml now declares the Python versions conda-package-cf.yml builds, for ci/matrix.py, and NumPy 2: without a numpy variant conda-build falls back to NumPy 1.26, which has no builds for Python 3.13 and newer. conda-package-cf.yml passes --python and --numpy itself, so it is unaffected.

…ws runners

antonwolfy · 2026-10-26T21:29:32Z

+
+concurrency:
+  group: intel-ci-${{ github.ref }}
+  cancel-in-progress: true


With cancel-in-progress: true keyed on github.ref, two quick merges cancel the first commit's whole run. Jenkins, which waits for that run, gets no packages for it.

Suggested change

cancel-in-progress: true

cancel-in-progress: ${{ github.event_name == 'pull_request' }}

Applied in 6006c16, so a push to master no longer stops the run before it. For what it is worth, Jenkins only waits for pull request and tag runs, and every tag is its own ref, so it never lost packages this way.

antonwolfy · 2026-10-26T21:37:08Z

+numpy:
+    - '2'


That only will work when ##202 is merged

Agreed, this should land after #202. With the current pin_compatible('numpy', min_pin="x.x"), the 3.15 package built here requires numpy >=2.5,<3.0a0 at run time, which #202 fixes. Once #202 merges I will rebase: numpy: '2' will come from #202, and 3.10 will leave the python: list along with its matrix.

antonwolfy · 2026-10-26T21:45:27Z

@@ -1,3 +1,16 @@
+# The Python versions to build, one per line and without selectors: ci/matrix.py
+# reads them for .github/workflows/intel-ci.yml. NumPy 2, as in conda-forge's
+# global pinning; conda-package-cf.yml passes its own with --python and --numpy.


--numpy is dropping in #202

Removed from the comment in 6006c16, so it stays true after #202.

antonwolfy · 2026-10-26T21:47:12Z

+        "--artifacts", required=True, type=Path, help="the downloaded artifact"
+    )
+    parser.add_argument(
+        "--work", required=True, type=Path, help="an empty scratch directory"


hooks.py doesn't check that --work is empty, and may install conda-index into the user's base environment.

Fixed in b758764. hooks.py now refuses a --work that is not an empty directory, and it writes the local channel's repodata.json itself, as Jenkins does, so it installs nothing into base.

antonwolfy · 2026-10-26T21:48:47Z

+
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.12"


Is there any issue to use the newer python?

No issue: the job only runs ci/matrix.py, which needs nothing beyond the standard library. Moved to 3.14 in 6006c16.

antonwolfy · 2026-10-26T21:53:59Z

+          activate-environment: ""
+          conda-remove-defaults: "true"
+
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0


Alternatively we can drop setup-python and run with shell: bash -el {0} with base activated.

bash is available on Windows runners too, and setup-miniconda supports this shell there. The step bodies become plain python ci/build.py … and python ci/hooks.py …. The $CONDA path code and hooks.py's re-run of itself would then only matter for local use.

Done in 6006c16: build and test run in bash -el {0} with base active, without setup-python. On Windows the new steps work: conda-build installs, the package builds and the imports pass. This run is still red on Windows because conda-forge's pytest 9.2.0, published today, dropped colorama again, so the recipe's pip check fails (conda-forge/pytest-feedstock#208). I will re-run once a fixed build is out.

Re-ran after conda-forge published the fixed pytest build: the run is green on Linux and Windows, test jobs included.

antonwolfy · 2026-10-26T22:00:21Z

@@ -0,0 +1,135 @@
+# Intel Python CI (gh-ci) for mkl_random: builds the conda-forge recipe, conda-recipe-cf, for every


The same PR is open in three repos at once.
Have you considered moving common things (scripts, workflow) to some common repo like https://github.com/IntelPython/devops-tools ?
That would help to reduce the duplication.

Yes, we considered it. Duplicated code is a known consequence of the new CI's design, where each repository carries everything its CI needs, and the design's other benefits make up for it.

A pull request now builds and tests the oldest and the newest regular Python versions the recipe declares, instead of the two newest, so the oldest supported Python is tested before a change merges rather than first on a release tag. Intel's Jenkins tests the same two.

Intel's Jenkins now runs one benchmark cell for a repository with a benchmarks/ directory, with the newest regular Linux Python's package, on a bare-metal benchmarking machine, and requires ci/benchmark.py to say how. It runs the asv suite against the installed package and keeps the results with the build: one quick sample of each benchmark on a pull request, a full measurement on a tag. asv comes from PyPI, as conda-forge has no build of it for Python 3.15 yet.

Build and test steps run in a bash login shell with conda base active, so they call ci/build.py and ci/hooks.py directly and need no setup-python. Pushes to master no longer cancel each other.

…pty --work Write repodata.json the way Jenkins' gh_ci_run does instead of installing conda-index into base, so the script never changes the base environment.

A key with a selector right after the list, such as c_compiler: # [linux], was refused as a selector in the python list.

Build and test steps run in a bash login shell with conda base active, so they call ci/build.py and ci/hooks.py directly and need no setup-python. Pushes to the default branch no longer cancel each other. Same as IntelPython/mkl_random#204.

jharlow-intel added 2 commits October 23, 2026 14:22

Pin Miniforge in Intel Python CI: 26.9.4-0 breaks conda init on Windo…

5d87148

…ws runners

jharlow-intel marked this pull request as ready for review October 23, 2026 20:42

jharlow-intel requested review from antonwolfy, ndgrigorian, vlad-perevezentsev and xaleryb as code owners October 23, 2026 20:42

antonwolfy added this to the 1.7.0 release milestone Oct 26, 2026

antonwolfy reviewed Oct 26, 2026

View reviewed changes

jharlow-intel added 6 commits October 27, 2026 10:46

Run Intel Python CI steps in conda base with bash, cancel only PR runs

6006c16

Build and test steps run in a bash login shell with conda base active, so they call ci/build.py and ci/hooks.py directly and need no setup-python. Pushes to master no longer cancel each other.

hooks.py: write the local channel's repodata itself and require an em…

b758764

…pty --work Write repodata.json the way Jenkins' gh_ci_run does instead of installing conda-index into base, so the script never changes the base environment.

Merge remote-tracking branch 'origin/master' into ci/intel-python-ci

3635be7

matrix.py: find the end of the python list before checking for selectors

da0b913

A key with a selector right after the list, such as c_compiler: # [linux], was refused as a selector in the python list.

jharlow-intel self-assigned this Oct 28, 2026

jharlow-intel requested a review from antonwolfy October 28, 2026 18:31

jharlow-intel mentioned this pull request Oct 28, 2026

Build and test conda-recipe-cf for Intel Python CI IntelPython/mkl-service#265

Open

	cancel-in-progress: true
	cancel-in-progress: ${{ github.event_name == 'pull_request' }}

		@@ -0,0 +1,135 @@
		# Intel Python CI (gh-ci) for mkl_random: builds the conda-forge recipe, conda-recipe-cf, for every

		numpy:
		- '2'

Conversation

jharlow-intel commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Verification

Not verified

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jharlow-intel commented Oct 23, 2026 •

edited

Loading