Skip to content

Materialize product() passed to parametrize (pytest 9.1 compat)#5244

Open
jaraco wants to merge 4 commits into
mainfrom
fix/parametrize-product-iterator
Open

Materialize product() passed to parametrize (pytest 9.1 compat)#5244
jaraco wants to merge 4 commits into
mainfrom
fix/parametrize-product-iterator

Conversation

@jaraco

@jaraco jaraco commented Jun 28, 2026

Copy link
Copy Markdown
Member

Problem

setuptools/tests/test_config_discovery.py::TestDiscoverPackagesAndPyModules::test_purposefully_empty passes an itertools.product (a one-shot iterator) directly as argvalues to @pytest.mark.parametrize.

pytest 9.1 deprecates non-Collection iterables for argvalues (PytestRemovedIn10Warning), which surfaces here as an error during collection — taking the whole test_config_discovery.py module down with it. The class-level decorator is exactly the hazard the deprecation targets: a bare iterator is exhausted after the first test method collects, silently skipping the rest.

See pytest-dev/pytest#13409 and the deprecation note.

Fix

Render the product eager via a small documented helper:

greedy_product = compose(list, product)  # jaraco.functools.compose

so argvalues is a re-iterable list. The rationale lives in the helper's docstring.

jaraco.functools is already a setuptools test dependency, so no new requirement.

Verification

  • test_purposefully_empty now collects and passes (30 params).
  • ruff check + ruff format --check clean.

Note: this only manifests with pytest >= 9.1; CI on an older pytest won't have flagged it yet, so this is a forward-compat fix.

🤖 Generated with Claude Code

jaraco and others added 2 commits June 28, 2026 15:24
A one-shot iterator (itertools.product) passed to @pytest.mark.parametrize
gets exhausted after the first of this class' test methods collects, so the
rest are silently skipped. pytest 9.1 deprecates this (pytest-dev/pytest#13409,
PytestRemovedIn10Warning), turning it into a collection error. Wrap it in
list() so the values can be reused.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the inline list(product(...)) with a module-level
greedy_product = compose(list, product), documenting the pytest 9.1
re-iterable-Collection requirement (pytest-dev/pytest#13409) as its docstring.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@mergify

mergify Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@jaraco

jaraco commented Jun 28, 2026

Copy link
Copy Markdown
Member Author

mypy checks are failing with:

[2](https://github.com/pypa/setuptools/actions/runs/28333781829/job/83936383357?pr=5244#step:11:63)
[gw0] darwin -- Python 3.14.6 /Users/runner/work/setuptools/setuptools/.tox/py/bin/python
setuptools/tests/test_config_discovery.py:24: error: Need type annotation for "greedy_product"  [var-annotated]
setuptools/tests/test_config_discovery.py:24: error: Argument 1 to "compose" has incompatible type "type[list[_T]]"; expected "Callable[[product[tuple[Never]]], list[Never]]"  [arg-type]
setuptools/tests/test_config_discovery.py:176: error: Too many arguments  [call-arg]

compose(list, product) can't be typed: product is an overloaded callable, and
passing it by value to compose's Callable[_P, _R] parameter collapses it to its
first overload (single iterable), so mypy types the result as a one-arg function
(var-annotated / arg-type / too-many-arguments errors).

Define greedy_product as a function that *calls* product(*iterables) instead, so
the variadic overload resolves normally and the helper types cleanly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jaraco

jaraco commented Jun 28, 2026

Copy link
Copy Markdown
Member Author

Claude says python/mypy#15737 and python/mypy#13540 are precisely relevant.

Revert greedy_product to the concise compose(list, product) form. mypy can't
type it (product is overloaded; passing it by value through compose's ParamSpec
collapses it to its first overload, python/mypy#13540), so suppress the
resulting var-annotated/arg-type/call-arg errors with type: ignore comments that
cite the upstream issue. warn_unused_ignores is disabled in mypy.ini, so these
are safe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant