You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
CI: enforce patched Python floor in setup-python and simplify harden-pip - #3259
The CPython releases 3.15.2, 3.14.10, 3.13.18, 3.12.16 and 3.11.18 (announcement, CVE-2026-115557) patch the ssl module to reject the compromised eMudhra/emSign intermediate keys. With the runner OS trust stores now confirmed fixed fleet-wide and pip 26.2.2 using truststore, the remaining exposure is any job that runs an unpatched interpreter: its ssl module silently keeps trusting the compromised keys, and nothing in CI notices today.
This PR closes that gap with three changes.
Enforce patched Python floors in setup-python
Every single-version setup-python step that feeds harden-pip now uses a tilde-range floor spec, so the tool cache can never resolve to a pre-fix patch release. Jobs that previously ran harden-pip on the system interpreter get a floored setup-python step:
Workflow (job)
Before
After
pre-commit.yml
3.14
~3.14.10
coverage.yml (4 jobs)
3.14
~3.14.10 via new PY_VER_FLOOR
release.yml (verify-testpypi, verify-pypi)
system python3
new step, ~3.12.16
ci-nightly.yml (test-ci-tools-for-release)
system python
new step, ~3.12.16
test-sdist-linux.yml
3.12
~3.12.16
test-sdist-windows.yml
3.12
~3.12.16
build-wheel.yml (cibuildwheel host)
3.12
~3.12.16
release.yml's check-release-notes job does not call harden-pip, but its setup-python is floored to ~3.12.16 as well for consistency.
setup-python with a tilde-range floor falls back to the manifest when the cached patch does not satisfy the floor, so no check-latest override is needed. In pre-commit.yml, PYTHON_VERSION: "3.14" stays minor-only because it is the cache restore-key prefix; in coverage.yml, PY_VER: "3.14" stays minor-only because it is also passed to ci/tools/env-vars. The floor is supplied to python-version separately in both. The release.yml verify jobs share a YAML anchor, so one new step covers both, and python3 -m venv builds the verification venv from the floored interpreter because setup-python puts its shim first on PATH.
Resolve the latest patch in the matrix jobs
The matrix jobs in test-wheel-linux.yml, test-wheel-windows.yml and build-wheel.yml run setup-python with matrix.PY_VER across several minor series (3.11 through 3.15t), each with a different first-fixed patch, so a single floor spec cannot cover them. These steps set check-latest: true instead, so setup-python resolves the newest patch from the manifest rather than a stale tool-cache entry. Every series in the matrix has a fixed release, so the newest patch always carries the fix.
Simplify harden-pip
Now that setup-python enforces the floor, the hash-pinned pip bootstrap is no longer needed: a plain pip install -U pip runs on an ssl module that rejects any forged certificate, so the download is safe, and it always picks up the latest pip (which uses truststore by default). The action is reduced to that single step, and its description documents the floor-spec convention that call sites must follow.
Checklist
New or existing tests cover these changes. (CI exercises every floored setup-python and harden-pip step; the floor spec itself is the enforcement mechanism.)
The documentation is up to date with these changes.
Print sys.version in the harden-pip action and fail when the interpreter
is older than the first release of its minor series that rejects the
compromised eMudhra/emSign intermediate CA keys.
Now that check_python_patched.py confirms the interpreter carries the
ssl-module fix (CVE-2026-115557), the hash-pinned pip download is no
longer needed for security: Python's own ssl rejects any MITM'd cert,
so a plain 'pip install -U pip' is safe.
Replace the pinned-URL step with 'pip install -U pip' and update the
action description accordingly. The python input and all 13 call sites
are unchanged.
juenglin
changed the title
CI: fail jobs that run an unpatched CPython interpreter
CI: guard against unpatched CPython and simplify harden-pip
Oct 28, 2026
The harden-pip interpreter guard rejected the pre-commit job's Python
3.14.9: with a minor-only spec and the default check-latest: false,
setup-python reused the hosted image's cached 3.14.9, which predates the
eMudhra ssl fix (3.14.10). Set check-latest: true so it resolves the
patched patch release from the manifest. Cache-key prefix logic is
unchanged (still keyed on the resolved version / PYTHON_VERSION minor).
With the setup-python call sites floored to the first patched release of
each minor series, the ssl module is guaranteed clean before harden-pip
runs, making the inline runtime check (check_python_patched.py) and its
unit test redundant. Drop both, revert the ruff.toml T201 exception that
accompanied them, and replace the check-latest workaround with explicit
tilde-range floor specs.
pre-commit 3.14 -> ~3.14.10
release 3.12 -> ~3.12.16
test-sdist-linux 3.12 -> ~3.12.16
test-sdist-win 3.12 -> ~3.12.16
build-wheel:112 3.12 -> ~3.12.16 (cibuildwheel host python)
setup-python resolves a tilde-range against the manifest even when a
stale cached patch does not satisfy the floor, so no check-latest is
needed. CVE-2026-115557.
juenglin
changed the title
CI: guard against unpatched CPython and simplify harden-pip
CI: enforce patched Python floor in setup-python and simplify harden-pip
Oct 28, 2026
Cover the sites the tilde-range sweep missed:
- release.yml verify jobs (shared anchor): add setup-python ~3.12.16
before the venv is created, so python3 -m venv inherits a patched ssl
module. These jobs have no checkout and build the venv from the system
python3 otherwise.
- ci-nightly.yml test-ci-tools-for-release: add setup-python ~3.12.16
before harden-pip (the job previously ran on the system interpreter).
- coverage.yml: add PY_VER_FLOOR (~3.14.10) and use it as the
setup-python spec in all four jobs; PY_VER stays minor-only for
env-vars/naming.
CVE-2026-115557.
The matrix jobs in test-wheel-linux, test-wheel-windows and build-wheel
run setup-python with matrix.PY_VER across 3.11 through 3.15t, each with
a different first-fixed patch, so a single tilde-range floor cannot
cover them. Set check-latest: true so setup-python resolves the newest
patch from the manifest instead of a stale tool-cache entry; every
series in the matrix has a fixed release, so the newest patch always
carries the eMudhra ssl fix (CVE-2026-115557).
~3.12.16 resolves correctly on Linux but fails on Windows because the
Windows python-versions manifest lags behind Linux when a security batch
ships. Switch build-wheel (cibuildwheel host) and test-sdist-windows to
check-latest: true with the minor-only spec so they resolve the newest
available patch on any OS, consistent with the matrix setup-python steps.
Updated build and test workflows to select the latest available Python patch releases, with minimum version floors where required for the SSL certificate fix.
Updated pip hardening to require a Python interpreter with that SSL fix and install the latest pip instead of a pinned version.
Updated release, coverage, pre-commit, and source-distribution workflows to use compatible Python versions.
The harden-pip action checks Python versions for the stated SSL fix before upgrading pip. CI workflows now select explicit Python version floors or check for the latest patch release.
The action checks the configured interpreter against listed Python patch thresholds. Its pip step now upgrades pip without a version or wheel hash pin.
Explicit Python version floors .github/workflows/ci-nightly.yml, .github/workflows/coverage.yml, .github/workflows/pre-commit.yml, .github/workflows/release.yml, .github/workflows/test-sdist-linux.yml
These workflows now select Python versions at or above the stated SSL-fix floors.
These workflows now check for the latest patch matching the requested Python version.
Possibly related PRs
NVIDIA/cuda-python#3197: Adds the harden-pip action whose pip installation behavior and interpreter requirement change here.
NVIDIA/cuda-python#3201: Changes the Pathfinder verification workflow that now sets up Python at the stated floor.
NVIDIA/cuda-python#3176: Adds harden-pip to coverage and changes wheel-test setup paths updated here.
Priority: Normal
Change: Other
Merge Risk: Moderate · up to dd043
Windows wheel-build and sdist CI jobs will fail at the harden-pip step until patched Windows interpreters are available or a secure bootstrap is retained. This should be resolved or explicitly accepted before merge. Some workflow jobs also still inherit default token permissions.
- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Comment @coderabbitai help to get the list of available commands.
important: Add an explicit least-privilege token scope to this job.
test-ci-tools-for-release has no permissions block. It checks out the repository and runs pip-installed and repository test code, so its GITHUB_TOKEN scope depends on repository defaults. Add permissions: contents: read to this job. GitHub applies repository defaults when the workflow or job does not override them. (raw.githubusercontent.com)
Sources: Path instructions, Linters/SAST tools
Minor · important: Set a least-privilege default for the coverage workflow. · coverage.yml:20-26
important: Set a least-privilege default for the coverage workflow.
coverage-vars has no permissions block and only checks out the repository to read ci/versions.yml. Add a restrictive workflow default and grant this job contents: read explicitly. Keep write and OIDC permissions limited to jobs that require them. GitHub derives initial permissions from repository or organization defaults unless the workflow or job overrides them. (raw.githubusercontent.com)
Sources: Path instructions, Linters/SAST tools
Minor · important: Limit the release-notes job token to read access. · release.yml:168-170
important: Limit the release-notes job token to read access.
check-release-notes has no job-level permissions block. It checks out inputs.git-tag and executes ci/tools/check_release_notes.py, so a repository default with write access would expose that token to code from the selected tag. Add permissions: contents: read here and define a restrictive workflow default. Keep write permissions only on publishing jobs. (raw.githubusercontent.com)
Reviewing files that changed from the base of the PR and between 94625c1 and cb8baed.
Files selected for processing: 10
.github/actions/harden-pip/action.yml
.github/workflows/build-wheel.yml
.github/workflows/ci-nightly.yml
.github/workflows/coverage.yml
.github/workflows/pre-commit.yml
.github/workflows/release.yml
.github/workflows/test-sdist-linux.yml
.github/workflows/test-sdist-windows.yml
.github/workflows/test-wheel-linux.yml
.github/workflows/test-wheel-windows.yml
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.
The reason will be displayed to describe this comment to others. Learn more.
Critical · Security & Privacy · Quick win
important: Keep the verified bootstrap for unpatched Python..github/workflows/build-wheel.yml (Line 106) and .github/workflows/test-sdist-windows.yml (Line 58) request 3.12 without a floor. The current manifest resolves Windows to 3.12.10, below the 3.12.16 fix. (raw.githubusercontent.com)
With an older bundled pip, a network attacker can use a compromised CA key to replace the unpinned download and execute code in CI. Restore the hash-pinned bootstrap and ensure callers use a patched interpreter or fail closed. (discuss.python.org)
As per path instructions: “focus on … security boundaries.”
The reason will be displayed to describe this comment to others. Learn more.
Major · Security & Privacy · Heavy lift
important: Keep an explicit patched-version floor in these setup steps. check-latest: true selects the newest version available in the actions/python-versions manifest. It does not require 3.12.16 or the corresponding floor for each matrix series. If the Windows manifest lags during a security release, setup-python can install the previous patch, and harden-pip then runs with the pre-fix ssl module. Retain a floor range such as ~3.12.16 for fixed-version jobs, and add an offline version-floor assertion before harden-pip for matrix jobs and the other affected workflows. (github.com)
As per path instructions, this CI change must preserve the security boundary before networked package installation.
The reason will be displayed to describe this comment to others. Learn more.
codex GPT-6.1-Sol ultra:
[P1] check-latest does not enforce the Windows security floor — build-wheel.yml:112–115. The manifest still selects Windows 3.11.9, 3.12.10, and 3.13.17, below the required fixes. This PR’s CI confirms the older versions execute. Removing the verified pip bootstrap therefore lacks its stated prerequisite. Retain that bootstrap until patched interpreters are available, or obtain patched builds and enforce explicit minimums.
[P1] Nested Lychee callers still use unpatched Python — harden-pip/action.yml:28. lychee-links invokes this action, but lychee.yml:73 and build-docs.yml:310 still request unconstrained 3.14. The docs job selects 3.14.9 before running the simplified upgrade. Floor both callers to ~3.14.10, or preserve the verified bootstrap.
Re-add the version check dropped in the earlier simplification commit,
this time inlined in action.yml rather than in a separate .py file, so
no additional source file or test is needed. The guard fails the step
(and the job) when the interpreter predates the first release of its
minor series that rejects the compromised eMudhra CA keys
(CVE-2026-115557), emitting a ::error:: workflow annotation.
This surfaces the Windows python-versions manifest lag explicitly
(e.g. 3.12.10 instead of 3.12.16) so it shows as a red job rather
than a silent unpatched pip upgrade.
Reviewing files that changed from the base of the PR and between cb8baed and dd043cd.
Files selected for processing: 1
.github/actions/harden-pip/action.yml
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
print("::error::Python "+p+" is older than "+fl+", the first "+str(v.major)+"."+str(v.minor)+" release that rejects the compromised eMudhra CA keys (CVE-2026-115557).")
set -u
printf'%s\n''--- relevant commit messages ---'
git log -8 --format='%H%n%s%n%b' -- .github/actions/harden-pip/action.yml .github/workflows/build-wheel.yml .github/workflows/test-sdist-windows.yml
printf'%s\n''--- actions/python-versions manifest: Windows 3.12 entries ---'
python3 - <<'PY'import jsonimport urllib.requesturl = "https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json"try: with urllib.request.urlopen(url, timeout=15) as r: data = json.load(r)except Exception as exc: print(f"manifest fetch failed: {exc}")else: found = [] for item in data: version = str(item.get("version", "")) if version.startswith("3.12"): files = item.get("files", []) windows = [f for f in files if str(f.get("platform", "")).lower().startswith("win")] if windows: found.append((version, windows)) print(json.dumps(found, indent=2))PY
Repository: NVIDIA/cuda-python
Length of output: 23827
important: Restore the Windows CI path.
Both Windows workflows request Python 3.12 with check-latest: true and run harden-pip. The Windows manifest resolves that request to Python 3.12.10, but the guard rejects every 3.12 interpreter below 3.12.16. These jobs therefore fail before their build or test steps. The fail-closed behavior is intentional, but it remains a mergeability problem.
Provide patched Windows interpreters for the required minor versions, or retain a secure hash-pinned bootstrap for affected versions.
Both workflows pass their Python interpreter to lychee-links, which
calls harden-pip. Neither had a patched-version floor, and the docs
CI run (job 123173870397) confirmed 3.14.9 was resolved before the
simplified pip upgrade ran (no hash-pinned bootstrap to compensate).
Apply the same ~3.14.10 tilde-range floor used in pre-commit.yml:
- lychee.yml: inline the floor in the python-version field; PYTHON_VERSION
stays minor-only ("3.14") for the cache restore-key prefix.
- build-docs.yml: direct literal, no cache interaction to preserve.
3.14.10 is present in the manifest for all runner platforms, so the
tilde-range resolves cleanly. Raised by rwgk in review.
All 14 Windows build rows first select Python 3.12.10, then fail the new 3.12.16 minimum before building. Current x64 and ARM64 logs confirm this. Provision patched Windows interpreters for both host and legacy matrix jobs, or provide an explicit legacy path retaining verified bootstrapping. Simply changing the requested floor cannot supply unavailable Windows binaries.
The earlier unchecked-upgrade paths now fail closed, and both missed Lychee callers are fixed. No additional findings.
Reviewed 680f0dc. All 38 offline guard cases pass; pre-commit and docs CI pass. Full CI remains red from the Windows failures.
CI: install eMudhra-fixed Pythons with a pinned uv (#3275)
Replaces `actions/setup-python` + `harden-pip` everywhere with a new
`.github/actions/setup-hardened-python` composite action.
The action uses `astral-sh/setup-uv` (SHA-pinned, `version: "latest-known"`)
to install CPython via python-build-standalone, whose archive integrity is
verified against embedded checksums rather than TLS. This unblocks Windows:
python.org no longer publishes installers for the 3.11–3.13 maintenance
series, so `setup-python` can never deliver 3.11.18/3.12.16/3.13.18 there.
A `shell: python` guard step fails the job if the `ssl` module predates the
eMudhra fix (CVE-2026-115557) or if the series has no known fixed release.
Notable call-site changes: build-wheel passes the host interpreter to
cibuildwheel via `python-path`; ci-nightly and release verify jobs now
get a hardened interpreter instead of the system Python; pre-commit and
lychee cache keys gain a `uv` component; `harden-pip` is removed.
Refs #3195. Supersedes #3259.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
juenglin
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
rwgk
left a comment
Description
Refs #3195
The CPython releases 3.15.2, 3.14.10, 3.13.18, 3.12.16 and 3.11.18 (announcement, CVE-2026-115557) patch the
sslmodule to reject the compromised eMudhra/emSign intermediate keys. With the runner OS trust stores now confirmed fixed fleet-wide and pip 26.2.2 usingtruststore, the remaining exposure is any job that runs an unpatched interpreter: itssslmodule silently keeps trusting the compromised keys, and nothing in CI notices today.This PR closes that gap with three changes.
Enforce patched Python floors in setup-python
Every single-version
setup-pythonstep that feedsharden-pipnow uses a tilde-range floor spec, so the tool cache can never resolve to a pre-fix patch release. Jobs that previously ranharden-pipon the system interpreter get a flooredsetup-pythonstep:pre-commit.yml3.14~3.14.10coverage.yml(4 jobs)3.14~3.14.10via newPY_VER_FLOORrelease.yml(verify-testpypi,verify-pypi)python3~3.12.16ci-nightly.yml(test-ci-tools-for-release)python~3.12.16test-sdist-linux.yml3.12~3.12.16test-sdist-windows.yml3.12~3.12.16build-wheel.yml(cibuildwheel host)3.12~3.12.16release.yml'scheck-release-notesjob does not callharden-pip, but itssetup-pythonis floored to~3.12.16as well for consistency.setup-pythonwith a tilde-range floor falls back to the manifest when the cached patch does not satisfy the floor, so nocheck-latestoverride is needed. Inpre-commit.yml,PYTHON_VERSION: "3.14"stays minor-only because it is the cache restore-key prefix; incoverage.yml,PY_VER: "3.14"stays minor-only because it is also passed toci/tools/env-vars. The floor is supplied topython-versionseparately in both. Therelease.ymlverify jobs share a YAML anchor, so one new step covers both, andpython3 -m venvbuilds the verification venv from the floored interpreter becausesetup-pythonputs its shim first onPATH.Resolve the latest patch in the matrix jobs
The matrix jobs in
test-wheel-linux.yml,test-wheel-windows.ymlandbuild-wheel.ymlrunsetup-pythonwithmatrix.PY_VERacross several minor series (3.11 through 3.15t), each with a different first-fixed patch, so a single floor spec cannot cover them. These steps setcheck-latest: trueinstead, sosetup-pythonresolves the newest patch from the manifest rather than a stale tool-cache entry. Every series in the matrix has a fixed release, so the newest patch always carries the fix.Simplify harden-pip
Now that
setup-pythonenforces the floor, the hash-pinned pip bootstrap is no longer needed: a plainpip install -U pipruns on ansslmodule that rejects any forged certificate, so the download is safe, and it always picks up the latest pip (which usestruststoreby default). The action is reduced to that single step, and its description documents the floor-spec convention that call sites must follow.Checklist
setup-pythonandharden-pipstep; the floor spec itself is the enforcement mechanism.)