Skip to content

A collection-time vacuity refusal loses its reason under pytest-xdist: rc 4 becomes 1 and the message is replaced by a 35-line INTERNALERROR #963

Description

@jdatcmd

Under -n, a collection-time vacuity refusal loses the sentence that says what was wrong. The run still fails, so nothing is falsely certified — but the reader is handed a 35-line pluggy traceback instead of the reason, and the exit code changes from 4 to 1.

Found while checking that #924's fix behaves under xdist. It is not caused by that change; it is the behaviour of every refusal this hook raises, and it is on main today.

Measured

Same offending file, same tree, same runner (pytest 9.1.1, pytest-xdist 3.8.0 — the pins in requirements-test.txt). The only thing that differs is -n 2.

REFUSAL                         MODE     rc   reason printed   INTERNALERROR lines
broad except                    serial   4    yes             0
broad except                    -n 2     1    NO              35
bare skip                       serial   4    yes             0
bare skip                       -n 2     1    NO              35

no counted assertion (in-test)  serial   1    yes             0
no counted assertion (in-test)  -n 2     1    yes             0

The split is collection-time versus test-time. pytest_collection_modifyitems raises pytest.UsageError, which pytest reports cleanly in a serial run and which xdist turns into an INTERNALERROR when it escapes inside a worker. A VacuityError raised inside a test body is a normal test failure and reports identically either way.

So every rule enforced at collection is affected, and they are the rules whose whole value is naming the thing to fix:

Why nobody has seen it

No gate runs the corpus under -n. ci.yml:116 is pytest -q $FILES with no -n, and nothing in test/*.sh drives the pytest corpus at all.

But xdist is not incidental here. It is pinned in requirements-test.txt, the layer registers an xdist hook (pytest_xdist_node_collection_finished in _RunShape) — an unpinned runner without xdist dies in check_pending before collection, which is how I first mismeasured #924 — and _RunShape's own comment records a measurement taken under --max-worker-restart=0. The layer is written for a parallel run it is never gated on.

Why it is worth fixing rather than documenting

This layer's premise is that a refusal must say what to do about it. A refusal whose reason is replaced by a pluggy traceback has been reduced to an exit code, and rc=1 is the same code a failing test gives — so a reader cannot tell "your test is wrong" from "your test is refused" without reading 35 lines of someone else's stack.

It also fails in the more annoying direction for a newcomer: the first thing they will do with a slow suite is reach for -n auto.

Not proposed here, deliberately

I have not decided between catching the error in the hook and reporting it through a path xdist preserves, refusing -n outright with a message, or moving the checks to a hook that runs on the controller. Each is a different claim about where the layer should live, and the right one is worth an argument rather than a guess. What is established is the table above.

Verification for whatever fix lands

The table is the test. A fix must make the -n 2 rows read rc=4, reason printed, INTERNALERROR 0, with the serial rows unchanged — and the in-test row must stay unchanged in both modes, since it is the control that shows the defect is specific to collection-time refusals rather than to xdist generally.

Measured on main 9cfa32d.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions