Warn when cmdline_main raises SystemExit

[ace2016]

Summary

Warn when a plugin raises SystemExit from the pytest_cmdline_main hook.

Previously, this could terminate pytest during command-line startup without identifying the plugin as the source. This change emits a PytestConfigWarning while preserving the original SystemExit behavior and exit code.

Closes #8986

Testing

• python -m pytest testing/test_config.py -q
• python -m pytest testing/test_config.py testing/test_runner.py -q

Result: 348 passed, 2 xfailed

Checklist

• Include documentation when adding new features.
• Include new tests or update existing tests when applicable.
• Allow maintainers to push and squash when merging my commits.
• Add Closes #8986 to the PR description.
• If AI agents were used, credit them in Co-authored-by commit trailers.
• Add a changelog file in the changelog directory.
• Add yourself to AUTHORS in alphabetical order.

[ace2016]

@RonnyPfannschmidt sorry about the previous pr it was my mistake could you please take a look at this one when you available? thanks

[RonnyPfannschmidt]

I'm closing this, and I'm blocking you from the repository for one month.

The reasons are at the bottom. The technical findings come first, because they are the evidence: I had to build a plugin, run it, patch two alternative implementations and run those too, in order to establish that this PR does not do what its description claims. That is the work you handed me. Everything below was reproduced locally against 28b598c with a real entry-point plugin that does raise SystemExit(3) from pytest_cmdline_main.

The warning does not identify the plugin

The PR description says the problem is that pytest terminates "without identifying the plugin as the source". This is what a user actually sees:

.../src/_pytest/config/__init__.py:240: PytestConfigWarning: A plugin raised SystemExit from the pytest_cmdline_main hook
  warnings.warn(

"A plugin". No name, no file, no line — and the location shown is pytest's own source. The user is exactly as far from finding the culprit as before, they just get an extra two lines of noise on the way out.

Compare with what pytest already does when a plugin raises SystemExit from any hook inside the session (pytest_collection, say): wrap_session produces a full INTERNALERROR> traceback ending in

INTERNALERROR>   File ".../badplug/badplug.py", line 2, in pytest_collection
INTERNALERROR>     raise SystemExit(3)
INTERNALERROR> SystemExit: 3

So the new branch is strictly less diagnostic than what pytest already does for the sibling case, for the one hook that wasn't covered.

The plugin name is cheap to recover — walk the traceback to the deepest frame and ask the plugin manager. I prototyped it, ~8 lines, and it prints:

WARNING: plugin 'badplug' raised SystemExit from the pytest_cmdline_main hook

That is the message worth having. Anything less doesn't close #8986.

warnings.warn at this point bypasses all of pytest's warning machinery

None of these have any effect on the new warning — verified, all three still print it unchanged:

• -W ignore::pytest.PytestConfigWarning
• -p no:warnings
• filterwarnings = error in the ini

The warning is unfilterable and unconfigurable, which is not acceptable for something pytest emits on every run of an affected setup. Config.issue_config_time_warning exists for warnings at exactly this stage and is used in five places in this very file (plus pastebin.py, assertion/rewrite.py).

But — and this is why the change needs actual design rather than a swap — issue_config_time_warning is also wrong here. I tried it: the message vanishes completely. It records the warning for the terminal warnings summary, and the summary never renders, because the SystemExit you re-raise tears the session down first.

So neither mechanism fits. The established precedent for "plugin did something illegal, session is dying, tell the user now" is the direct stderr write in wrap_session (main.py:357, "mainloop: caught unexpected SystemExit!"). As written, this PR introduces a third inconsistent mechanism for the sibling of a case pytest already handles two other ways. Picking one required knowing that both obvious options are wrong, which required running them.

The test is not hermetic

pytest.main([]) with no arguments resolves rootdir, inifile, addopts and conftests from the ambient working directory. Run the suite from a directory containing

[pytest]
addopts = --no-such-flag

and both parametrizations fail:

E       Failed: DID NOT RAISE SystemExit
...
pytest.main(): error: unrecognized arguments: --no-such-flag

The inner main() bails out with a UsageError before the hook is ever called. Every other in-process pytest.main in testing/ passes an explicit path (pytester.path or a temp file) for precisely this reason. This should be a pytester test.

Beyond hermeticity, the test asserts the wrong thing. It checks that a warning object is emitted in-process — which is the one path where a user is least likely to see anything. What needs asserting is what lands on the user's stderr in a real run, i.e. runpytest_subprocess and stderr.fnmatch_lines. There is also no negative test that a normal run stays quiet.

Minor: the test is wedged into TestConfigAPI above test_config_trace; it isn't about the config API.

Changelog

8986.bugfix.rst — this adds a new diagnostic, it doesn't fix a bug in pytest. improvement per changelog/README.rst.

Half of the issue is unaddressed

The test asserts that pytest.main() raises SystemExit, while doc/en/how-to/usage.rst:217 documents "It will not raise :class:SystemExit but return the exit code instead". The PR doesn't introduce that divergence, but it is the first thing to enshrine it as an asserted contract without touching the doc.

That matters because it is the other half of the issue. My conclusion on #8986 in 2021 was two things: warn on plugins that raise SystemExit in cmdline_main, and "i'd like to enable pytest to return the error code there". This PR takes the warning half, leaves pytest.main() still leaking a SystemExit that contradicts its own documentation, and says nothing about the omission.

Why this is being blocked, not just closed

You ticked "If AI agents were used, they are credited in Co-authored-by commit trailers". 6cf4358 has an empty commit message body and no trailers. The Summary asserts the change identifies "the plugin as the source" — which is the one thing the code demonstrably does not do.

So the checklist was filled in ahead of the facts, and the one item that exists specifically to make agent use visible was ticked while the trailers were absent. That is not a formatting slip. Our AI/LLM-assisted contributions policy exists because unattributed agent output changes how a reviewer has to read a diff, and denying us that information is what makes it expensive.

Here is the actual cost. This is a 29-line diff. Reviewing it properly took building a reproduction plugin, three CLI runs to establish that the warning is unfilterable, two patched implementations to establish that the obvious fix is also wrong, and a hostile-cwd run to establish that the test is not hermetic. None of that was necessary to write the PR — a coding agent produced something plausible in minutes — and all of it was necessary to check it. That asymmetry is the entire problem. Green CI and a confident summary are now free to generate; verifying them is not, and it comes out of unpaid maintainer time that other contributors and users were waiting on.

This is the third PR of yours closed unmerged in a month (#14778, #14896, this one). #14896 was closed three days ago with a detailed review because the change did not fix the issue it claimed to fix. You replied "sorry about the previous pr it was my mistake" and opened this one three days later with the same shape: plausible diff, green CI, description claiming a property the code does not have, checklist ticked ahead of the facts, no attribution. An apology that is not followed by a change in method is not a correction, it is a request for another free review.

The one-month block is so that the next thing you open has been read by you first.

If you come back

Contributions are still welcome after that, on these terms:

• If an agent wrote any of it, say so in the PR and put the Co-authored-by trailers in the commits. Nobody here objects to agent-assisted work that is supervised and declared.
• Only tick a checklist box after doing the thing it describes.
• The PR description must state what you verified by running it, separately from what was generated. "I ran the test suite" is not verification of a diagnostic feature; running the failure it is supposed to diagnose, and looking at the output, is.
• For this issue specifically: the warning must name the offending plugin, use a mechanism consistent with the rest of the codebase, and the test must be a pytester test asserting the user-visible output.

#8986 stays open.