Repository navigation
test: Warm PowerShell before Windows round trips - #380
Merged
leongdl merged 1 commit intoOct 16, 2026
Merged
Conversation
TestReprPwshWindowsValidation fails intermittently on windows-latest with subprocess.TimeoutExpired after 10 seconds. The first PowerShell launch on a fresh hosted runner pays a one-time cold start that can exceed the 10-second per-call timeout in _pwsh_eval, so whichever test launches PowerShell first is killed, and a killed launch can leave the next one cold too. Run 38812686088 (PR OpenJobDescription#379, a CHANGELOG-only change) lost the first four tests of the class this way, each at about 10.1 s, while the other nine passed. The same timeout appears in 8 of the last 60 Code Quality runs, on Python 3.10, 3.12, 3.13 and 3.14, and run 35389550086 passed on rerun at the same SHA. Add a class-scoped autouse fixture that pays the cold start once, with a 120-second deadline, through the same _pwsh_eval path the tests use: a .ps1 file run with -File that calls Write-Output. It asserts the output, so a broken PowerShell still fails the class instead of being skipped. The per-call timeout stays at 10 seconds and no test assertion changes. Mark the class timeout(180), because pytest-timeout counts class-scoped fixture setup against the first test and the project-wide --timeout=30 would otherwise cut a long warm-up short. Verified on macOS with PowerShell 7.6.7 behind a shim that simulates the cold start. The class is Windows-only, so this is a simulation, not a Windows reproduction. Without the fix, a 41 s cold start fails the same four tests as run 38812686088, at 10.01 s each, and a 12 s cold start fails test_pwsh_string_simple_roundtrip alone. With the fix both pass 13/13, and the cold start lands in the first test's setup. Each of these mutants fails the class under a 41 s cold start: no warm-up, a 10 s warm-up deadline, and no timeout marker. Only repeated green Code Quality runs on real Windows runners prove the fix. At roughly one failure in 50 Windows jobs, one green run is weak evidence. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
|
Claude review · advisory Reviewed Open: 0 blocking · 0 should-fix · 0 nit. ✅ Nothing blocking. Fix or reply to each thread; the next revision's review re-checks open threads and resolves those it agrees are handled. Resolving a thread yourself also closes it. Later revisions review only what changed. |
Merged
AlexTranAmz
approved these changes
Oct 16, 2026
leongdl
enabled auto-merge (squash)
October 16, 2026 19:35
crowecawcaw
approved these changes
Oct 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes: no issue filed. Flaky failure seen on #379, run 38812686088.
What was the problem/requirement? (What/Why)
TestReprPwshWindowsValidationintest/openjd/expr/test_strings.pyfails intermittently onwindows-latestwithsubprocess.TimeoutExpiredafter 10 seconds. On a fresh hosted runner, the first PowerShell launch has a one-time startup delay that can run past the 10-second per-call timeout in_pwsh_eval. Whichever test launches PowerShell first gets killed, and a killed launch can leave the next one cold as well.Evidence:
test_pwsh_string_simple_roundtrip, the first test in the class.What was the solution? (How)
_pwsh_evalpath the tests use: a.ps1file run with-Filethat callsWrite-Output. It asserts the output, so a broken PowerShell still fails the class instead of being skipped.@pytest.mark.timeout(180). pytest-timeout counts class-scoped fixture setup against the first test, and the project-wide--timeout=30would otherwise cut a long warm-up short.What is the impact of this change?
Test-only. One file, 13 lines added. No change to
src/.How was this change tested?
hatch run testpasses 6330 tests both before and after the change.hatch run lintandhatch run typingare clean.The class is Windows-only and I developed this on macOS, so the evidence is a simulation, not a Windows reproduction. A shim in front of PowerShell 7 simulated a slow first launch:
test_pwsh_string_simple_roundtrip)Each of these mutants fails the class under a 41 s cold start: no warm-up fixture, a 10 s warm-up deadline, and no timeout marker.
Only repeated green runs on real Windows runners prove the fix. The failure rate is roughly one in 50 Windows jobs, so a single green run is weak evidence. Please rerun the Code Quality Windows matrix a few times before merging.
Was this change documented?
Is this a breaking change?
No.
Does this change impact security?
No. It is a test-only change.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.