Move selection before repeat

[CanYing0913]

TL;DR

fix #15 #21 #95

Motivation

pytest-repeat currently implements repetition by adding an indirect parameter in pytest_generate_tests. As a result, pytest creates original item count × repeat count items before collection filtering and other pytest_collection_modifyitems hooks run.

For large suites and high repeat counts, this multiplies collection work even for tests that will later be deselected. In real-world cases, collection can take more than an hour while the selected tests themselves only take a few minutes.

What changed

This PR moves repeat expansion until after collection modification and deselection have completed:

1. Collect the original test items once.
2. Apply -k, -m, and other collection filtering/reordering plugins to the original items.
3. Expand only the remaining items according to --count or @pytest.mark.repeat.
4. Pass the final repeated item list to the runner and pytest-xdist.

The implementation also:

• creates an independent item for each repetition;
• preserves the existing [current-total] node ID suffixes, including parameterized IDs such as [param-1-10];
• preserves function, class, module, and session repeat ordering;
• rebuilds fixture requests and copies per-item mutable state so repetitions do not share runtime state;
• keeps collection totals consistent with the final repeated item list.

pytest-xdist and filtering

The repeated items are created before pytest_collection_finish, so xdist workers report the final unique node IDs to the controller and each repetition can be scheduled independently.

For example:

pytest -n 4 -k important --count=10

collects the original tests, applies -k important, expands each selected test ten times, and then distributes those repeated items across workers.

Because filtering now happens before repeat expansion, -k and -m match the original test IDs and markers. Generated repeat suffixes such as [1-10] are not available as filtering targets. Final reports and xdist scheduling still use the suffixed node IDs.

Performance characteristics

Collection hooks such as pytest_itemcollected now run only for the original items, and ordinary pytest_collection_modifyitems implementations process the unexpanded list. The final runner and xdist item list still contains selected item count × repeat count entries, so execution scheduling and final item storage remain proportional to the number of repetitions.

Tests

Added coverage for:

• repeat expansion after -k deselection;
• collection plugins seeing only original items;
• unique final node IDs and --collect-only totals;
• fixture isolation between repetitions;
• pytest-xdist execution with repeated items.

The existing parameterization, repeat marker, step number, unittest, and repeat-scope tests continue to pass.

Locally verified with:

• pytest 6.2.5, 7.4.4, 8.4.2, and 9.0.2;
• pytest-xdist 3.8.0;
• flake8.

[CanYing0913]

Hi @okken Can you take a look? I feel this is important for large-scale testing. So much appreciate for your time :)

[RonnyPfannschmidt]

applying the pytest ai policy here - thisis a massive structural change that can only be described as letting a statistic monkey play with fire

[CanYing0913]

@RonnyPfannschmidt Thanks for the feedback.

I think it's worth discussing the underlying problem this PR is trying to solve rather than dismissing it just because of AI involvement. pytest-repeat works well for small test suites, but it scales poorly for very large ones. For tests focused on numerical correctness or system stability—especially GPU kernel tests—it is common to repeat a small subset hundreds or thousands of times, and those runs can take days to finish. In that workflow, the current implementation becomes a bottleneck before execution even starts. For example, repeating the FlashAttention-3 test suite 1000 times took over 5 hours just for test collection. With this change, collection drops to under 20 seconds because the selected tests are expanded after filtering rather than before.

I think that's a meaningful improvement for this use case, and I'd prefer to discuss the design itself rather than whether the PR should be dismissed under the AI policy.

If there are concerns with my proposed approach (selecting first and expanding afterward), I'd be happy to discuss those. From my perspective, expanding the entire workload before applying selection no longer makes sense for large-scale repeat workloads, and I'd like to better understand the reasons behind the current design.

AI is a tool, and I understand it has been abused to spam every repo of open-source. If the insight of this PR is good enough, I don't bother to use it to make my idea possible. Of course I don't know much about pytest as a whole - but we have many brilliant contributes like you to help me tailor the code. Again, I don't want to bury this PR just because it involves AI, just tell me how to make it formal and better. Thanks :)

B.R.
CY

[RonnyPfannschmidt]

we are currently at the pytest sprint and actively discussion reasonable solutions for this

the reason i buried specifically this pr is that it does very problematic things around copying nodes and mutating them and had no prior technical discussion

[CanYing0913]

it does very problematic things around copying nodes and mutating them and had no prior technical discussion

I agree. I am glad it draws discussions. Prior solutions to use pattern matching with pytest-repeat (#15 ) is reasonable but not so satisfying.