fix(py_test): fix Windows crash in py_test main validation - #4079
Merged
rickeylev merged 2 commits intoAug 19, 2026
Merged
Conversation
rdesgroppes
force-pushed
the
fix-current-interpreter-executable-sibling-files
branch
5 times, most recently
from
August 19, 2026 01:31
1b51fb7 to
6925faa
Compare
`py_test` main validation (`--validate_test_main=enabled`) crashes on Windows, because it invokes the interpreter via `exec_tools.exec_interpreter`, which resolves through `current_interpreter_executable()`'s relocated copy of the interpreter. The crash may be reproduced on existing tests in the repo, for instance: ``` bazel build \ --@rules_python//python/config_settings:validate_test_main=enabled \ //tests/validate_test_main:validate_test_main_test ... ERROR: .../tests/validate_test_main/BUILD.bazel:3:8: Validating py_test main //tests/validate_test_main:validate_test_main_test failed: (Exit -1073741515): python.exe failed: error executing PyValidateTestMain command ``` ``` cd tests/integration/validate_test_main bazel build \ --@rules_python//python/config_settings:validate_test_main=enabled \ //:good_test ... ERROR: .../tests/integration/validate_test_main/BUILD.bazel:11:8: Validating py_test main //:good_test failed: (Exit -1073741515): python.exe failed: error executing PyValidateTestMain command ``` `-1073741515` is `STATUS_DLL_NOT_FOUND`: the relocated copy can't find its DLLs beside itself. This is also the Windows-local manifestation of bazel-contrib#2703 (`exec_interpreter` broken on RBE): the same relocation severs the interpreter from files resolved relative to itself, just triggered differently: RBE's copy materialization there, a lack of symlink privilege here. Colocating the DLLs alone (a first attempt) traded this for a second, still fatal error, `ModuleNotFoundError: No module named 'encodings'`, because the copy still can't find its stdlib. Patching each missing file individually doesn't scale: DLLs today, stdlib tomorrow, whatever else a future toolchain needs beside itself after that. `_maybe_add_test_main_validation` now uses `actions_run()` with `exec_runtime` instead of `exec_tools.exec_interpreter`'s relocated `DefaultInfo.files_to_run`, matching `PyExecToolsInfo.exec_interpreter`'s own documented recommendation and the pattern `common.bzl`'s `actions_run()` and `py_zipapp_rule.bzl` already use. `exec_runtime.interpreter` is the real file, used directly, with its real files as plain action inputs, so nothing is relocated and nothing loses its siblings. With the proposed fix[^1], above examples now build cleanly, with no relocated runfiles tree for the interpreter at all, and `tests/integration/validate_test_main`'s `inert_test` still fails with its intended "will not run any tests" message rather than a crash. `tests/integration/validate_test_main_test` is the corresponding integration test, but it was not exercised on Windows, where it was failing on `OSError: [WinError 193] %1 is not a valid Win32 application` in `tests/integration/runner.py`'s Bazel-in-Bazel invocation, itself unable to run `bazel_from_env`'s `#!` shebang line the way POSIX's `exec` does. The present change therefore fixes this, by resolving the shebang's interpreter itself, and enables the test on Windows. [^1]: This does not fix `exec_interpreter`/bazel-contrib#2703 itself: `precompile.bzl` still resolves the interpreter via the relocated path and would need the same migration.
rdesgroppes
force-pushed
the
fix-current-interpreter-executable-sibling-files
branch
from
August 19, 2026 01:38
6925faa to
4d48490
Compare
rdesgroppes
marked this pull request as ready for review
August 19, 2026 01:49
py_test main validationpy_test main validation
Update news fragment with Sphinx MyST cross-reference syntax and issue URL format. Wrap the exec_runtime toolchain check condition in py_executable.bzl to adhere to 80 columns, and align subprocess stdout/stderr demarcation banners in runner.py with conventions.
rickeylev
approved these changes
Aug 19, 2026
rickeylev
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for the clean PR! Just needed a couple style fixes. Otherwise LGTM
rickeylev
enabled auto-merge
August 19, 2026 03:53
rdesgroppes
deleted the
fix-current-interpreter-executable-sibling-files
branch
August 19, 2026 05:51
Contributor
Author
@rickeylev merci ! |
rdesgroppes
added a commit
to rdesgroppes/rules_python
that referenced
this pull request
Aug 19, 2026
`python/private/py_executable.bzl`'s `_maybe_add_test_main_validation` fix (bazel-contrib#4079) noted `precompile.bzl` as a remaining user of the same `exec_interpreter` relocation issue, needing the same migration. There was no existing test exercising `_precompile`'s action at all: `tests/base_rules/precompile`'s suite is `analysis_test`-only, checking declared providers, never actually running the precompiler. Reproducing this on Windows therefore required a real `bazel build`, via the new `test_precompile_enabled_succeeds`: ``` bazel test \ //tests/base_rules/precompile:test_precompile_enabled_succeeds ... ERROR: .../tests/base_rules/precompile/BUILD.bazel:3:22: Python precompiling .../test_precompile_enabled_succeeds_main.py into .../test_precompile_enabled_succeeds_main.cpython-311.pyc failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../multiplex-worker-1-PyCompile.log ---8<---8<--- (empty) ---8<---8<--- End of log ---8<---8<--- ``` The worker crashes at startup, unable to find its DLLs, before it can write anything to its own log or respond over the worker protocol. `_precompile` now uses `actions_run()` with `exec_runtime`, exactly as `_maybe_add_test_main_validation` does, instead of `exec_tools_info.exec_interpreter[DefaultInfo].files_to_run`. Reproducing and fixing this also uncovered two more problems, both specific to the precompiler's worker mode and unrelated to `exec_interpreter`. First, `tools/precompiler/precompiler.py`'s persistent worker reads each JSON request as a single line via `asyncio.StreamReader`, whose default 64KiB limit is exceeded once every interpreter distribution file, previously hidden by relocation into a much smaller symlink tree, shows up as an actual, individually-digested action input: ``` ValueError: Separator is not found, and chunk exceed the limit ``` A CPython 3.11 distribution's ~2,260 inputs measure ~470KiB this way; `1 << 22` (4MiB) leaves ample headroom. Second, the worker's default implementation, `_AsyncPersistentWorker`, can't start on Windows at all: `asyncio`'s `ProactorEventLoop` fails to wrap `stdin`/`stdout` as pipe transports, with: ``` OSError: [WinError 6] The handle is invalid ``` Bazel gives workers anonymous pipes (`CreatePipe`) for stdio, which never support overlapped I/O, so `asyncio`'s `ProactorEventLoop` can't register them with an I/O completion port. This is unrelated to precompiling's relocation bug: nothing exercises this worker on Windows today. `_SerialPersistentWorker`, the blocking-I/O alternative already present in the file, has no such issue, so `--worker_impl` now defaults to `serial` on Windows. `tests/base_rules/precompile:test_precompile_enabled_succeeds` is a real, executing `py_test` with `precompile = "enabled"`, added alongside the analysis-only suite to close this gap: it forces the precompiler action to actually run, and needs no CI wiring since it carries no tag excluding it from the existing Windows job's default test sweep.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
py_testmain validation (--validate_test_main=enabled) crashes on Windows, because it invokes the interpreter viaexec_tools.exec_interpreter, which resolves throughcurrent_interpreter_executable()'s relocated copy of the interpreter.The crash may be reproduced on existing tests in the repo, for instance:
-1073741515isSTATUS_DLL_NOT_FOUND: the relocated copy can't find its DLLs beside itself.This is also the Windows-local manifestation of #2703 (
exec_interpreterbroken on RBE): the same relocation severs the interpreter from files resolved relative to itself, just triggered differently: RBE's copy materialization there, a lack of symlink privilege here.Colocating the DLLs alone (a first attempt) traded this for a second, still fatal error,
ModuleNotFoundError: No module named 'encodings', because the copy still can't find its stdlib.Patching each missing file individually doesn't scale: DLLs today, stdlib tomorrow, whatever else a future toolchain needs beside itself after that.
_maybe_add_test_main_validationnow usesactions_run()withexec_runtimeinstead ofexec_tools.exec_interpreter's relocatedDefaultInfo.files_to_run, matchingPyExecToolsInfo.exec_interpreter's own documented recommendation and the patterncommon.bzl'sactions_run()andpy_zipapp_rule.bzlalready use.exec_runtime.interpreteris the real file, used directly, with its real files as plain action inputs, so nothing is relocated and nothing loses its siblings.With the proposed fix1, above examples now build cleanly, with no relocated runfiles tree for the interpreter at all, and
tests/integration/validate_test_main'sinert_teststill fails with its intended "will not run any tests" message rather than a crash.tests/integration/validate_test_main_testis the corresponding integration test, but it was not exercised on Windows, where it was failing onOSError: [WinError 193] %1 is not a valid Win32 applicationintests/integration/runner.py's Bazel-in-Bazel invocation, itself unable to runbazel_from_env's#!shebang line the way POSIX'sexecdoes.The present change therefore fixes this, by resolving the shebang's interpreter itself, and enables the test on Windows.
Footnotes
This does not fix
exec_interpreter/exec tools exec_interpreter broken on RBE #2703 itself:precompile.bzlstill resolves the interpreter via the relocated path and would need the same migration. ↩