ci: shard Windows go test into 6 balanced parallel jobs (~15m → <5m) - #95
Merged
Conversation
Windows go test took ~15 min: ~330 serial tests, each spawning several git subprocesses, and Windows CreateProcess+Defender scan is ~10x Linux. Inline Defender exclusions were already applied and are not enough. In-process t.Parallel() is unsafe (the package swaps ~14 global function-seams), so shard across processes. - .github/scripts/ci_shard.py: LPT makespan-minimization shard selector (stdin test list -> go test -run regex; --timings calibration; --verify). - .github/scripts/test_ci_shard.py: partition/determinism/balance unit tests. - ci.yml: drop windows from the test matrix; add a sharded test-windows job (shard 0..5); auto-merge-sync now needs [test, test-windows]. Mirrors operatorstack/intelligence-flow (.github is control-plane, not projected by labkit). Note: renames the Windows required status check; branch protection must be updated after merge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Shard the Windows
go testleg ofci.ymlinto 6 balanced parallel runners so Windows CI drops from ~15 min to a target < 5 min. Mirrors the same fix landing upstream inoperatorstack/intelligence-flow(PR #166) — where it took the Windows suite from ~15 min to ~2 min per shard.Why
The Boatstack Go suite (
boatstack/) is ~330 tests in one package, run strictly serially (t.Parallel()is used nowhere). Almost every test builds a real on-disk git repo, spawning severalgitsubprocesses — ~1,000–2,000 process spawns across the suite. On Windows eachCreateProcess(+ Defender scanning fresh compile output) is ~10× the Linuxfork+exec, so the serial suite is ~15 min on Windows vs ~1 min on Unix. Process-spawn latency, not CPU, is the bottleneck. The inline Defender exclusions already inci.ymlare necessary but not sufficient.In-process
t.Parallel()is unsafe here: the package swaps ~14 mutable package-global function-seams and uses manyt.Setenvsites, so parallel tests in one process would race. Job-level sharding runs each shard as a separate process (globals are per-process), preserving today's exact serial semantics while cutting wall-clock ~N×.How
.github/scripts/ci_shard.py— an LPT makespan-minimization controller (P || Cmax).go test -list '^Test' ./...piped in → thego test -runregex for--index i --total 6. Optional--timings <json>calibrates weights against a measured envelope;--verifyasserts an exact partition..github/scripts/test_ci_shard.py— partition/determinism/balance/escaping unit tests.ci.yml—test(Unix) stays the full unsharded reference; newtest-windowsjob runsmatrix: shard [0..5]with the existing runtime-detect + inline Defender steps. Windows build/installer validation runs once (shard 0).auto-merge-syncnowneeds: [test, test-windows]so sync-PR auto-merge still waits on the Windows leg.Verified locally against this repo's suite: 328 tests partition into 55/55/55/55/54/54; controller unit tests (15) green; YAML parses.
Why a separate PR (not a sync)
.github/is Boatstack-owned control plane and is not projected by labkit from the monorepo, so this fix cannot ride the normal upstream sync — it is applied here directly.Sharding renames the Windows status check (
test (windows-latest)→test-windows (shard 0..5)). After merge, update branch-protection required-status-checks, or PRs will wait on a check that no longer runs.