CI perf: release-readiness — 10-second gate waits up to 5 min for a Depot boot in front of coverage and e2e (~1–3 min per run in slow-start periods) #1795

Filed by the CI profiler routine (run 2026-10-28 20:16 UTC).

Measurement

Window: 2026-10-28 16:18 → 20:16 UTC. Jobs fetched (filter=all) for all 20 non-stub CI runs: 1 merge_group, 1 push, 18 pull_request. Depot slow-start (job started_at → first step over 15 s) was 62% of Depot jobs in this slice (275/447, gap p50 35 s, p90 144 s, max 329 s), the worst since #1512 was filed.

release-readiness is a ~10-second job (checkout, two python3 scripts/ci-reuse-*.py probes, release-lint.sh), but it runs on depot-ubuntu-24.04-4 and coverage, e2e-build, test, node-addon, docker-base, the cargo/yarn legs and, through e2e-build, the whole e2e fan-out wait for it (needs: release-readiness at .github/workflows/ci.yml lines 148, 362, 607, 765, 1055, 1162, 1176). So its Depot boot sits at the head of the critical path, before any real work starts.

On a fast host it finishes at 0.20–0.28 min (9 of 20 runs).

Where the time goes

In the merge_group run for #1748: release-readiness 0.01 → 5.31 (5.1 min of boot) → coverage (1/2), e2e-build created at 5.31 → e2e-build done 6.68 → 13 e2e legs created at 6.68, each with its own 38–117 s boot → last e2e at 11.38 → ci-ok 11.48. clippy (also slow, 116 s boot) finished at 3.86, so with a fast gate the e2e fan-out could have started at ~3.9 instead of 6.7.

Root cause

#1747 moved ci-ok to ubuntu-latest because a seconds-long job on Depot pays the Depot boot. release-readiness is the same kind of job at the other end of the run: tiny, but on Depot, and a gate for nearly everything. When Depot is in a slow-start episode (#1512), its boot is serialised in front of the boots of the jobs it gates, so the run pays two slow boots back to back.

Proposed fix

In .github/workflows/ci.yml, change release-readiness's runs-on to ubuntu-latest (the same GitHub-hosted runner #1747 gave ci-ok; drop the DISABLE_DEPOT_RUNNERS switch for this job or keep it as a no-op). The job needs only git, python3, bash and GH_TOKEN, all on the hosted image. Leave every needs: edge as is, so the reuse outputs (reuse, pr_unchanged) and the release lint keep gating the same jobs.

Expected saving

Replaying this slice's 19 completed runs with release-readiness finishing at work + 6 s (hosted queue in this slice: 2–7 s for ci-ok) and keeping the clippy and per-job boot times as measured:

• merge_group Keep Go hosted pins after go mod tidy drops the zip line #1748: 11.48 → ~8.7 min (−2.8) on the critical path.
• All runs: p50 −0.0, p90 −1.1, max −2.8 min; sum −6.9 min over 19 runs in 4 h. Fast-host periods save nothing; slow-start periods (23–64% of Depot jobs in 7 of the last 10 dashboard rows) save 0.5–3 min per run.
• ~0.1k Depot Linux job-min/day moved to free hosted minutes (~120 non-draft runs/day × ~0.3–5 min of booking per gate job).
• macOS / Windows: none.

Coverage and risk

No test moves or is skipped; only the host of one gate job changes. Required checks ci-ok and clippy keep their names. Risk: the ubuntu-latest backlog of #1372 (9–18 min waits) would now delay the gate too; since #1747 the hosted queue has stayed at 1–7 s for ci-ok (31 + 17 runs), and the job can fall back to Depot via the existing DISABLE_DEPOT_RUNNERS-style variable if that recurs.

Effort

S (one runs-on line plus its comment).

ROI

saving ≈ 0.1 (k job-min/day) + ~1 (critical-path / PR-latency min in slow-start periods) = 1.1; × confidence 0.7 ÷ effort 1 = 0.8.

Generated by Claude Code