The starvation check keys on RUN status — a matrix run whose jobs are all being picked up alarms as 'queued 30+ min' (#447 was a false positive)

[avrabe]

What

fleet-status.yml's starvation step lists runs with status=queued and alarms when the run's created_at is 30+ minutes old. A matrix run keeps run-level status queued while any leg waits for a runner, even while its other jobs are starting and finishing normally. So a large matrix on a busy fleet reads as starved when it isn't.

Evidence: #447, the check's first real alarm

The check could only start measuring after #437 installed gh on the light runners (#436). Its first alarm was #447, "Kani on main — queued 32 min", and it was a false positive:

	time (2026-09-17)
Kani run 35211315805 created (push to main, 7ea22ac)	10:35:51
first job started	10:35:54 (3 s later)
monitor ran (schedule, run 35214076493) and alarmed	11:07:59
last of 47 jobs started	11:22:28
run completed	11:22:37 — success

Jobs were being picked up continuously. The fleet was busy, not starving.

Why it matters

Starvation is supposed to mean nothing is picking work up. An alarm that fires on every large Kani matrix during a busy hour trains people to ignore it, which is the failure mode the monitor was built to end (#436, #429).

Fix direction

Measure jobs, not runs, the same lesson scripts/ci-wedge-watch.rs already encodes ("run status is not job status"). A run is starving only if it has queued jobs and no job in it has started for 30+ minutes, or equivalently, if its oldest queued job has waited 30+ minutes with nothing started since. The detector's Rust tool, which already enumerates jobs, is the natural place for it.

Also: the alarm step tries --label ci, and this repo has no ci label. The fallback without a label worked (#447 has none), so either create the label or drop the flag.

Planned under SWREQ-RELAY-CIFLOW-P01 (v1.142), which covers "required gates / monitors not decided by runner luck". Refs #436, #447.

🤖 Generated with Claude Code

[avrabe]

Second false positive today (#455), with the measurement

#455 fired at 19:19Z for Kani on the wit-bindgen-0.62.0 branch, reported as "queued 51 min". Measured on that run (35258925212): created 18:27:24Z, first job started 18:32:22Z, all 47 jobs started. The fleet was busy with the #451 Kani soak and three PRs, not starving.

So the starvation check has now produced two alarms and both were false (#447, #455), while the wedge detector's one alarm (#450) was true. Each false alarm opens an issue, so the noise is not free: it trains the reader to ignore the alarm that matters.

The fix in this issue is unchanged — measure the oldest QUEUED JOB with no job started since, rather than run status. It is currently scoped to CIFLOW-P01 (v1.142). Two false positives in one day is the argument for pulling it forward; that is a maintainer call, and I am not moving it unasked.