You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
perf(ci): test-windows-guardrails takes p50 522 s from three spawn-heavy suites run serially #5989
test-windows-guardrails takes p50 522 s (p95 565 s) because three bash suites spawn heavily on Git Bash and run serially in one job.
Measurements
Step
p50
p95
min
max
Cases
Log span
s/case
Test secret-pattern-detection
159.5 s
176.2 s
99
185
217
158.5 s
0.73
Test block-hook-bypass
134.0 s
147.1 s
84
158
800
134.6 s
0.17
Test block-windows-drive-tmp
80.0 s
87.1 s
54
95
625
79.7 s
0.13
Test run-guards
62.0 s
67.1 s
36
71
Test hardcoded-path-check
61.0 s
72.1 s
39
81
Job: p50 520 s (push) / 522 s (PR), p95 565 s / 558 s, n=20 each. Test-step p50 sum 497.5 s; fixed overhead 24.5 s (checkout 12 s, coverage-manifest 1 s, setup and post steps). Runs 37075818834..37091316717.
Suite
Linux forks (strace proxy)
Linux execve calls
secret-pattern-detection
10,154
11,091
block-hook-bypass
3,067
25,607
block-windows-drive-tmp
2,607
19,504
Windows time per Linux fork is not constant (block-hook-bypass 43.7 ms/fork), so the counts rank where spawns are but do not convert straight into seconds.
Cause
Per-case process spawns on Git Bash, about 140 ms per fork/exec (stated in the workflow and test comments, not re-measured). No case dominates in any of the three suites; the largest single case is 6.7 s (UNC/no-project in secret-pattern-detection). secret-pattern-detection already collapses its payload reads into one jq process (secret-pattern-detection.sh:89-90), so the remaining cost is per-case. The suites assert no timing in these logs.
In-job concurrency (background & + wait, logs buffered per suite) on the 4-vCPU runner. Not measured; pilot it on these suites first because they assert no timing, unlike hook-utils and markdown-format.
Per-suite spawn reduction: batch cases per hook invocation where the test harness allows, and cut per-case subshells and jq/grep calls. Measure the per-case fork count before changing anything.
Accuracy guard
Options 1 and 2 keep every case and command unchanged. For option 3, each change must keep the case count identical (217, 800, 625) and the PASS lines unchanged; compare the full PASS/FAIL list before and after. No assertion is removed or loosened.
Related
#5886 (CI lane work). #5935 also edits plugins/guardrails/hooks/secret-pattern-detection.test.sh (+16 lines for the ghs_ token format), which slightly lengthens the longest suite; coordinate. #5830 (13-minute PR run). Observed on #5958.
Closing as delivered by proposed fix 1 (parallel jobs). #6002 folded test-windows-guardrails into the split test-windows jobs, and #6023 rebalanced it to six jobs. The guardrails suites now run in different jobs: secret-pattern-detection in test-windows-secrets, block-hook-bypass and hardcoded-path-check in test-windows-guards, block-windows-drive-tmp in test-windows-lib, and run-guards in test-windows-rename-1.
Measured on #6023 (dispatched run 37101504440 and the PR run): every Windows job took 162-239 s, against the 522 s guardrails job measured here. Every case still runs.
Options 2 and 3 are set aside, not filed. Test-windows is no longer the slowest check on a PR: on #6023's ready run the Windows jobs finished at 06:07:56, the ci.yml lanes at about 06:07:00, and claude-review at 06:09:13. Spawn reduction would save at most about 30-40 s (the floor is secret-pattern-detection at about 160-176 s plus job overhead), and no PR would get to green any sooner. Reopen if the review lane gets faster and Windows becomes the slowest again.
test-windows-guardrailstakes p50 522 s (p95 565 s) because three bash suites spawn heavily on Git Bash and run serially in one job.Measurements
Job: p50 520 s (push) / 522 s (PR), p95 565 s / 558 s, n=20 each. Test-step p50 sum 497.5 s; fixed overhead 24.5 s (checkout 12 s, coverage-manifest 1 s, setup and post steps). Runs 37075818834..37091316717.
Windows time per Linux fork is not constant (block-hook-bypass 43.7 ms/fork), so the counts rank where spawns are but do not convert straight into seconds.
Cause
Per-case process spawns on Git Bash, about 140 ms per fork/exec (stated in the workflow and test comments, not re-measured). No case dominates in any of the three suites; the largest single case is 6.7 s (UNC/no-project in secret-pattern-detection). secret-pattern-detection already collapses its payload reads into one jq process (
secret-pattern-detection.sh:89-90), so the remaining cost is per-case. The suites assert no timing in these logs.Proposed fix
Either, per the measured options:
&+wait, logs buffered per suite) on the 4-vCPU runner. Not measured; pilot it on these suites first because they assert no timing, unlike hook-utils and markdown-format.jq/grepcalls. Measure the per-case fork count before changing anything.Accuracy guard
Options 1 and 2 keep every case and command unchanged. For option 3, each change must keep the case count identical (217, 800, 625) and the PASS lines unchanged; compare the full PASS/FAIL list before and after. No assertion is removed or loosened.
Related
#5886 (CI lane work). #5935 also edits
plugins/guardrails/hooks/secret-pattern-detection.test.sh(+16 lines for theghs_token format), which slightly lengthens the longest suite; coordinate. #5830 (13-minute PR run). Observed on #5958.🤖 Generated with Claude Code