From 5bf09d4c5d769e895a341f668bf4d82d3092f5a1 Mon Sep 17 00:00:00 2001 From: Hans Johnson Date: Sat, 22 Aug 2026 14:35:00 -0500 Subject: [PATCH 1/4] Declare ruff's vendored-file excludes in pyproject.toml Ruff is the only formatting/linting tool whose exclusion of the two vendored/generated files is not declared in pyproject.toml. Black has it in [tool.black] exclude and pyrefly has it in [tool.pyrefly] project-excludes; ruff's copy lives in the callers instead: runtests.sh:602,604 --exclude versioneer.py --exclude monai/_version.py .pre-commit-config.yaml exclude: (?x)(^versioneer.py|^monai/_version.py) Two consequences follow from that placement. First, the exclusion only applies when ruff is reached through one of those two callers. A one-off developer or IDE invocation -- plainly "ruff check --fix ." at the repository root -- does not get it, and rewrites versioneer.py and monai/_version.py. Before this change that call reports 111 violations (84 UP031, 11 N806, 7 N801, 6 UP035, 2 N818, 1 B904), every one of them inside those two files, and --fix modifies both. After it, "ruff check ." reports "All checks passed!" and --fix is a no-op. Verified on ruff 0.14.14 (the version pyproject currently resolves to) and on 0.16.4 (latest); the whole 0.14.11-0.16.4 range behaves identically here. Second, the setting has to be restated once per caller, so each new way of invoking a tool adds another copy that can drift. #9061 shows the shape of this: it adds black and isort pre-commit hooks, and each one carries its own exclude block, with the accompanying note that "black will be given individual file names and so will ignore the excludes in pyproject.toml". Declaring the setting where the tool looks for it by default keeps one definition no matter how many routes reach the tool. This commit only adds the declaration; the now-redundant copies in runtests.sh and .pre-commit-config.yaml are left in place so this change is inert on its own and can be verified independently. They become removable once this has landed. No source files are changed. runtests.sh --codeformat passes on all four legs (copyright 1360 files, isort, black, ruff, pyrefly 0 errors) and pre-commit run --all-files passes. Signed-off-by: Hans Johnson --- pyproject.toml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/pyproject.toml b/pyproject.toml index 9c5f892283..650dc0b417 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -296,6 +296,8 @@ exclude = "monai/bundle/__main__.py" [tool.ruff] line-length = 120 target-version = "py310" +# Vendored/generated; matches [tool.black] exclude and [tool.pyrefly] project-excludes. +extend-exclude = ["versioneer.py", "monai/_version.py"] [tool.ruff.lint] select = [ From 2aa2d14a1612dcd96e84dd7ef4a7eefe23ccaa64 Mon Sep 17 00:00:00 2001 From: Hans Johnson Date: Sat, 22 Aug 2026 14:45:44 -0500 Subject: [PATCH 2/4] Resolve lint tools from the environment pyproject.toml defines Tool versions were declared twice and could not agree. pyproject.toml asked for "ruff>=0.14.11,<0.15" while .pre-commit-config.yaml pinned rev v0.15.20 -- disjoint ranges, so a developer following CONTRIBUTING.md and pre-commit.ci were guaranteed to run different linters. pycln was pinned only in the hook and never installed by runtests.sh at all. pyproject.toml now owns both the versions and the settings: - a "lint" optional-dependency group holds the tools that .pre-commit-config.yaml and runtests.sh both invoke, and "testing" pulls it in via monai[lint] so developers still have one install. The group excludes torch and the optional dependencies, so a lint-only environment can be built from it alone. - ruff is pinned to 0.16.4, the newest release that leaves this codebase unchanged. Every release from 0.14.11 to 0.16.4 was run against dev and they are indistinguishable here: same violations before, same files touched by --fix, and "All checks passed!" on each. - pycln is added at 2.6.0, the version its hook used. - [tool.black] switches from exclude to force-exclude with the same regex, because black ignores exclude for filenames given on the command line, which is how pre-commit invokes it. The ruff and pycln hooks become local hooks with language: system, and black and isort join them, so pre-commit runs the tools from that environment rather than building its own from a second set of pins. There is no longer a rev: to keep in sync. The hygiene hooks from pre-commit-hooks keep their rev:, as they have no counterpart in pyproject.toml and so are already single-definition. The flags on those entries make the pyproject settings apply to the explicit filenames pre-commit passes. Each is load-bearing; run against the two vendored files directly: black with force-exclude nothing to do / without: 2 would be reformatted ruff with --force-exclude no files found / without: 200 errors isort with --filter-files skipped 2 files / without: 2 sort errors Because these hooks need an environment pre-commit.ci does not build, they are listed under ci.skip; the pre-commit job added to cicd_tests.yml runs them instead. parse_dependencies() gains expansion of self-referential requirements. runtests.sh's install_deps writes its output to a requirements file and runs "pip install -r" on it, and the parser appended each group verbatim, so "monai[lint]" reached pip as a plain requirement with no local path. pip would have resolved it against the package index -- installing the published release over the checkout under test, and none of the five lint tools. Self-references are now replaced by the group they name, with a seen-set so a group that refers to itself terminates. Covered by new cases in tests/config/test_print_dependencies.py, including the cyclic one. No source file is reformatted; the only Python change is that parser. pre-commit run --all-files and runtests.sh --autofix both leave the tree untouched. Signed-off-by: Hans Johnson --- .pre-commit-config.yaml | 43 +++++++++++++++++-------- monai/config/print_dependencies.py | 42 ++++++++++++++++++++++++ pyproject.toml | 17 +++++++--- tests/config/test_print_dependencies.py | 10 ++++-- 4 files changed, 91 insertions(+), 21 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 2d78b08041..e3a27e079c 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -6,6 +6,9 @@ ci: autoupdate_commit_msg: '[pre-commit.ci] pre-commit suggestions' autoupdate_schedule: quarterly # submodules: true + # The hooks below run the tools from the environment pyproject.toml defines, which + # pre-commit.ci does not build; the pre-commit job in cicd_tests.yml runs them instead. + skip: [black, isort, pycln, ruff-check] repos: - repo: https://github.com/pre-commit/pre-commit-hooks @@ -26,19 +29,33 @@ repos: args: ['--autofix', '--no-sort-keys', '--indent=4'] - id: end-of-file-fixer - id: mixed-line-ending - - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.15.20 - hooks: - - id: ruff-check - args: ["--fix"] - exclude: | - (?x)( - ^versioneer.py| - ^monai/_version.py - ) - - repo: https://github.com/hadialqattan/pycln - rev: v2.6.0 + # Versions come from [project.optional-dependencies].lint and settings from the + # [tool.*] tables, so these hooks and runtests.sh cannot resolve a different tool. + # The --force-exclude/--filter-files flags apply those settings to the explicit + # filenames pre-commit passes. + - repo: local hooks: + - id: ruff-check + name: ruff check + entry: python -m ruff check --force-exclude --fix + language: system + types_or: [python, pyi] + + - id: isort + name: isort + entry: python -m isort --filter-files + language: system + types_or: [python, pyi] + + - id: black + name: black + entry: python -m black + language: system + types_or: [python, pyi] + - id: pycln - args: [--config=pyproject.toml] + name: pycln + entry: python -m pycln --config=pyproject.toml + language: system + types_or: [python, pyi] diff --git a/monai/config/print_dependencies.py b/monai/config/print_dependencies.py index a099949eca..7b8957983c 100644 --- a/monai/config/print_dependencies.py +++ b/monai/config/print_dependencies.py @@ -18,6 +18,7 @@ from __future__ import annotations +import re import sys from collections.abc import Collection @@ -25,10 +26,49 @@ PROJ_KEY = "project" OPTS_KEY = "optional-dependencies" DEP_KEY = "dependencies" +NAME_KEY = "name" REQ_KEY = "requires" TOML_FILE = "pyproject.toml" +def _expand_self_extras(dependencies: list[str], name: str, opts: dict) -> list[str]: + """ + Replace self-referential requirements such as ``monai[lint]`` with the contents of that group. + + pip resolves such a requirement against the package index, so leaving one in a generated + requirements file installs the published release instead of the checkout being worked on. + + Args: + dependencies: requirement strings, some of which may be self-references. + name: this project's name, the only one treated as a self-reference. + opts: the "optional-dependencies" table the groups are read from. + + Returns: + List of requirements with every self-reference replaced by the group it names. + + Raises: + KeyError: If a self-reference names a group absent from `opts`. + """ + pattern = re.compile(rf"^{re.escape(name)}\s*\[([^\]]+)\]$", re.IGNORECASE) + expanded: list[str] = [] + pending = list(dependencies) + seen: set[str] = set() + + while pending: + req = pending.pop(0) + match = pattern.match(req.strip()) + if match is None: + expanded.append(req) + continue + for group in (g.strip() for g in match.group(1).split(",")): + if group in seen: # a group already expanded, or a cycle + continue + seen.add(group) + pending.extend(opts[group]) + + return expanded + + def parse_dependencies(filename: str | None = None, sections: Collection[str] | None = None) -> list[str]: """ Parse the toml file given by `filename` and return the dependency sections selected by `sections`. @@ -68,6 +108,8 @@ def parse_dependencies(filename: str | None = None, sections: Collection[str] | for s in sections: dependencies += opts[s] + dependencies = _expand_self_extras(dependencies, proj[NAME_KEY], opts) + return sorted(set(dependencies)) diff --git a/pyproject.toml b/pyproject.toml index 650dc0b417..2e9c360030 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -157,20 +157,25 @@ torchvision = ["torchvision"] tqdm = ["tqdm>=4.47.0"] transformers = ["transformers>=4.53.0, <5.0"] # 5.x references torch.float8_e8m0fnu absent in older PyTorch builds zarr = ["zarr"] +# the tools .pre-commit-config.yaml and runtests.sh invoke; installable without torch +lint = [ + "black>=26.3.1", + "isort>=5.1,<6,!=6.0.0", + "pre-commit", + "pycln==2.6.0", + "ruff==0.16.4" +] # these dependencies are for testing/building only, they aren't needed for regular use so don't appear in "all" testing = [ - "black>=26.3.1", + "monai[lint]", "coverage>=5.5", - "isort>=5.1,<6,!=6.0.0", "mccabe", "packaging", "parameterized", "pep8-naming", - "pre-commit", "pycodestyle", "pyflakes", "pyrefly>=1.0.0", - "ruff>=0.14.11,<0.15", "tomli", # used in print_dependencies.py for Python<3.11 "typeguard<3", # https://github.com/microsoft/nni/issues/5457 "types-PyYAML", @@ -266,7 +271,9 @@ line-length = 120 target-version = ['py310'] skip-magic-trailing-comma = true include = '\.pyi?$' -exclude = ''' +# force-exclude, not exclude: black ignores exclude for filenames passed explicitly, +# which is how pre-commit invokes it. +force-exclude = ''' ( /( # exclude a few common directories in the root of the project diff --git a/tests/config/test_print_dependencies.py b/tests/config/test_print_dependencies.py index bbf8c4c7cd..628ff2cb45 100644 --- a/tests/config/test_print_dependencies.py +++ b/tests/config/test_print_dependencies.py @@ -32,14 +32,18 @@ [project.optional-dependencies] all = ["something", "another"] -testing = ["coverage", "black"] +lint = ["ruff", "black"] +testing = ["test[lint]", "coverage"] +cyclic = ["test[cyclic]", "spam"] """ PARSE_CASES = [ ([], ["numpy", "torch"]), - (["testing"], ["black", "coverage", "numpy", "torch"]), + # "test[lint]" expands rather than reaching pip as a requirement on the published package + (["testing"], ["black", "coverage", "numpy", "ruff", "torch"]), (["build-system"], ["numpy", "setuptools", "torch", "wheel"]), - (["*"], ["another", "black", "coverage", "numpy", "something", "torch"]), + (["*"], ["another", "black", "coverage", "numpy", "ruff", "something", "spam", "torch"]), + (["cyclic"], ["numpy", "spam", "torch"]), ] From 5a51a447492f93f8df9b5b22b7a74fb797c314a3 Mon Sep 17 00:00:00 2001 From: Hans Johnson Date: Sat, 22 Aug 2026 14:45:52 -0500 Subject: [PATCH 3/4] Invoke ruff through PY_EXE and drop its duplicated excludes runtests.sh called ruff as a bare executable on PATH while isort, black, pylint and pytype all go through "${PY_EXE}" -m. The guard above it, is_pip_installed ruff, tests importlib.util.find_spec using PY_EXE, so the check interrogated one environment and the invocation ran whatever ruff PATH happened to offer. In a clean virtualenv built per CONTRIBUTING.md this makes ./runtests.sh --codeformat fail outright: ruff ./runtests.sh: line 598: ruff: command not found Check failed! and where a system ruff does exist it silently wins over the pinned one. The --exclude versioneer.py --exclude monai/_version.py flags are dropped because [tool.ruff] extend-exclude now carries them, so they apply however ruff is reached rather than only through this script. --unsafe-fixes is left on the fix path as-is; making it symmetric with the check path is a behaviour change and belongs on its own. Signed-off-by: Hans Johnson --- runtests.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/runtests.sh b/runtests.sh index 73508a093b..d74abebf62 100755 --- a/runtests.sh +++ b/runtests.sh @@ -595,13 +595,13 @@ then then install_deps fi - ruff --version + ${cmdPrefix}"${PY_EXE}" -m ruff --version if [ $doRuffFix = true ] then - ruff check --fix --unsafe-fixes --exclude versioneer.py --exclude "monai/_version.py" "$homedir" + ${cmdPrefix}"${PY_EXE}" -m ruff check --fix --unsafe-fixes "$homedir" else - ruff check --exclude versioneer.py --exclude "monai/_version.py" "$homedir" + ${cmdPrefix}"${PY_EXE}" -m ruff check "$homedir" fi ruff_status=$? From 733c6ce73cd97d100f2a4c76e162f5537a01ef43 Mon Sep 17 00:00:00 2001 From: Hans Johnson Date: Sat, 22 Aug 2026 14:45:59 -0500 Subject: [PATCH 4/4] Run pre-commit in CI on a lint-only environment pre-commit has never run in GitHub Actions; only the external pre-commit.ci service ran it. Now that the formatting hooks are skipped there, because they need an environment that service does not build, this job runs them. It installs the "lint" extra alone, read out of pyproject.toml at run time so the versions are not restated in the workflow. That needs neither torch nor the optional dependencies, unlike static-checks, which installs .[all,testing] before running formatters. static-checks is left unchanged. Running both routes keeps the copyright and pyrefly coverage it provides, and makes CI fail if pre-commit and runtests.sh ever disagree about the same files. Signed-off-by: Hans Johnson --- .github/workflows/cicd_tests.yml | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/.github/workflows/cicd_tests.yml b/.github/workflows/cicd_tests.yml index e9b207951f..19f5c0d44c 100644 --- a/.github/workflows/cicd_tests.yml +++ b/.github/workflows/cicd_tests.yml @@ -52,6 +52,34 @@ env: # When support is dropped for a version it is important to update these as appropriate. jobs: + pre-commit: # Run the hooks pre-commit.ci skips, using the tools pyproject.toml pins + runs-on: ubuntu-latest + permissions: + contents: read + steps: + # This job executes the hooks named by a PR's own .pre-commit-config.yaml, so + # it must not leave a usable token in .git/config for those hooks to reach. + - uses: actions/checkout@v7 + with: + persist-credentials: false + # reads the pins below with tomllib, which needs 3.11+; the hooks take their + # target from pyproject.toml, so the interpreter version does not affect them + - name: Set up Python ${{ env.PYTHON_VER3 }} + uses: actions/setup-python@v6 + with: + python-version: ${{ env.PYTHON_VER3 }} + cache: 'pip' + - name: Install lint tools + run: | + # the lint extra alone, so this job needs neither torch nor the optional dependencies + python -m pip install --upgrade pip + python -c "import tomllib; print('\n'.join(tomllib.load(open('pyproject.toml','rb'))['project']['optional-dependencies']['lint']))" > lint-requirements.txt + cat lint-requirements.txt + python -m pip install -r lint-requirements.txt + rm lint-requirements.txt + - name: Run pre-commit + run: python -m pre_commit run --all-files --show-diff-on-failure + static-checks: # Perform static type and other checks using runtests.sh runs-on: ubuntu-latest strategy: