Under -n, a collection-time vacuity refusal loses the sentence that says what was wrong. The run still fails, so nothing is falsely certified — but the reader is handed a 35-line pluggy traceback instead of the reason, and the exit code changes from 4 to 1.
Found while checking that #924's fix behaves under xdist. It is not caused by that change; it is the behaviour of every refusal this hook raises, and it is on main today.
Measured
Same offending file, same tree, same runner (pytest 9.1.1, pytest-xdist 3.8.0 — the pins in requirements-test.txt). The only thing that differs is -n 2.
REFUSAL MODE rc reason printed INTERNALERROR lines
broad except serial 4 yes 0
broad except -n 2 1 NO 35
bare skip serial 4 yes 0
bare skip -n 2 1 NO 35
no counted assertion (in-test) serial 1 yes 0
no counted assertion (in-test) -n 2 1 yes 0
The split is collection-time versus test-time. pytest_collection_modifyitems raises pytest.UsageError, which pytest reports cleanly in a serial run and which xdist turns into an INTERNALERROR when it escapes inside a worker. A VacuityError raised inside a test body is a normal test failure and reports identically either way.
So every rule enforced at collection is affected, and they are the rules whose whole value is naming the thing to fix:
Why nobody has seen it
No gate runs the corpus under -n. ci.yml:116 is pytest -q $FILES with no -n, and nothing in test/*.sh drives the pytest corpus at all.
But xdist is not incidental here. It is pinned in requirements-test.txt, the layer registers an xdist hook (pytest_xdist_node_collection_finished in _RunShape) — an unpinned runner without xdist dies in check_pending before collection, which is how I first mismeasured #924 — and _RunShape's own comment records a measurement taken under --max-worker-restart=0. The layer is written for a parallel run it is never gated on.
Why it is worth fixing rather than documenting
This layer's premise is that a refusal must say what to do about it. A refusal whose reason is replaced by a pluggy traceback has been reduced to an exit code, and rc=1 is the same code a failing test gives — so a reader cannot tell "your test is wrong" from "your test is refused" without reading 35 lines of someone else's stack.
It also fails in the more annoying direction for a newcomer: the first thing they will do with a slow suite is reach for -n auto.
Not proposed here, deliberately
I have not decided between catching the error in the hook and reporting it through a path xdist preserves, refusing -n outright with a message, or moving the checks to a hook that runs on the controller. Each is a different claim about where the layer should live, and the right one is worth an argument rather than a guess. What is established is the table above.
Verification for whatever fix lands
The table is the test. A fix must make the -n 2 rows read rc=4, reason printed, INTERNALERROR 0, with the serial rows unchanged — and the in-test row must stay unchanged in both modes, since it is the control that shows the defect is specific to collection-time refusals rather than to xdist generally.
Measured on main 9cfa32d.
Under
-n, a collection-time vacuity refusal loses the sentence that says what was wrong. The run still fails, so nothing is falsely certified — but the reader is handed a 35-line pluggy traceback instead of the reason, and the exit code changes from 4 to 1.Found while checking that #924's fix behaves under xdist. It is not caused by that change; it is the behaviour of every refusal this hook raises, and it is on main today.
Measured
Same offending file, same tree, same runner (pytest 9.1.1, pytest-xdist 3.8.0 — the pins in
requirements-test.txt). The only thing that differs is-n 2.The split is collection-time versus test-time.
pytest_collection_modifyitemsraisespytest.UsageError, which pytest reports cleanly in a serial run and which xdist turns into anINTERNALERRORwhen it escapes inside a worker. AVacuityErrorraised inside a test body is a normal test failure and reports identically either way.So every rule enforced at collection is affected, and they are the rules whose whole value is naming the thing to fix:
@pytest.mark.skipexcept Exceptionpytest.raisesblock that pins no SQLSTATEWhy nobody has seen it
No gate runs the corpus under
-n.ci.yml:116ispytest -q $FILESwith no-n, and nothing intest/*.shdrives the pytest corpus at all.But xdist is not incidental here. It is pinned in
requirements-test.txt, the layer registers an xdist hook (pytest_xdist_node_collection_finishedin_RunShape) — an unpinned runner without xdist dies incheck_pendingbefore collection, which is how I first mismeasured #924 — and_RunShape's own comment records a measurement taken under--max-worker-restart=0. The layer is written for a parallel run it is never gated on.Why it is worth fixing rather than documenting
This layer's premise is that a refusal must say what to do about it. A refusal whose reason is replaced by a pluggy traceback has been reduced to an exit code, and
rc=1is the same code a failing test gives — so a reader cannot tell "your test is wrong" from "your test is refused" without reading 35 lines of someone else's stack.It also fails in the more annoying direction for a newcomer: the first thing they will do with a slow suite is reach for
-n auto.Not proposed here, deliberately
I have not decided between catching the error in the hook and reporting it through a path xdist preserves, refusing
-noutright with a message, or moving the checks to a hook that runs on the controller. Each is a different claim about where the layer should live, and the right one is worth an argument rather than a guess. What is established is the table above.Verification for whatever fix lands
The table is the test. A fix must make the
-n 2rows readrc=4, reason printed,INTERNALERROR 0, with the serial rows unchanged — and the in-test row must stay unchanged in both modes, since it is the control that shows the defect is specific to collection-time refusals rather than to xdist generally.Measured on main
9cfa32d.