The drain fans out one agent short: a live-agent count of 1 with no agent visible

[suleimansh]

🤖 automated

Found dogfooding #1640/#1644/#1645 on main (chain write-up: #1204 (comment)).

What happened. Concurrent agents: 3, three entries on the queue, no agent running on the project — the Agents panel said "No agents working right now", the last run on the project had been DONE for ten minutes. The drain's Run now reported:

tf-1334-dogfood: started 2 agents: draining the queue entry "[Add a root `.gitignore` …"; draining the queue entry "[Add a `package.json` …"

Two of three. The third entry sat on the queue.

Why two. The fan-out is picks = open.slice(0, concurrency - activeAgents) (auto-pm.ts:922), so activeAgents read 1 at 01:20:55. activeAgentCount (daemon-runtime.ts:840) is live pids on the project plus the starting set:

for (const [key, pid] of activeAgents) { if (keyBelongsTo(key, targetProjectId) && isPidAlive(pid)) live++ }
return live + [...starting].filter(key => keyBelongsTo(key, targetProjectId)).length

The pid half is re-checked against the OS, so a dead process cannot hold a slot. The starting half is not: a key that is added on spawn and never removed — a spawn path that throws between the two, or a refused start that skips the cleanup — costs the project one slot for the rest of the daemon's life, invisibly. Not diagnosed which it was; by 01:27 no process of the earlier runs was alive, and the daemon does not log the count.

Timeline on that project: 01:03 three planners via the sweep (all DONE by 01:06, locks released); 01:06 one triage via the card's plain start (DONE 01:10); 01:20 the drain click → count 1.

Fix shape. Either log the breakdown on every stand-down/fan-out decision (live=N starting=M), so the next occurrence names itself — or make starting self-healing the way activeAgents is (a key older than the spawn timeout is dropped). Both are small; the first is what would have made this an answer instead of a question.

Repro. Not reliable yet. Three sweep-started runs, one card-started run, then a drain click on a project the panel shows idle.

[suleimansh]

🤖 automated

Diagnosed from the daemon's own log of that session and the rig's tf-data history — the slot was a real live process, not a leaked starting key.

What held the slot. The card-started triage at 01:06 (2026-08-22T22-06-41-065Z) did its work — 7e78c24 "[triage-quick] queue the three planned tickets" landed on tf-data at 01:10:01 — but its process never exited and the run was never archived. tf-data has no "archive session 2026-08-22T22-06-41-065Z" commit until 02:15:46, and that one is the daemon's shutdown writing a failed marker. The daemon's last lines at 02:17:

[framework] run 2026-08-22T22-06-41-065Z failed to start: its process was killed by SIGTERM before reporting anything
[framework] keeping worktree …/tf-agent-2026-08-22T22-06-41-065Z: main is not on the remote …
[framework] stopped 1 agent(s)

So at the 01:20 drain click the daemon's table held one live pid for the project — that process — and concurrency − activeAgents = 3 − 1 = 2. The count was correct; the Agents panel reads each run's own status, and this run had reported nothing since its turn.

Not the starting set. starting.add(key) at the spawn is paired with finally { starting.delete(key) } (daemon-runtime.ts), so a key cannot outlive the spawn.

Why that process lingered is still open. The 01:27 card-started triage (the one that aborted) was archived normally at 01:27:50, and today's sweep-started triage (#1649's dogfood) archived one second after finishing — so neither "card-started" nor "triage" is the condition. The run's worktree — and with it its stderr.log — is gone, so what its epilogue was waiting on (a push prompt, the browser stream, a gate keepalive) cannot be read back. It wrote to tf-data at 01:10 and then sat for 65 minutes.

What the PR does. Makes the next occurrence name itself: the daemon reports the slots it holds (run id + pid) instead of a count, the sweep says them on the card and in the log — a cap stand-down reads 2 runs are already going (<id> (pid N), …), a short fan-out reads started 2 agents alongside 1 already going (<id> (pid N)) — and the shutdown line lists the ids it stopped. With that, the 01:20 line would have said started 2 agents alongside 1 already going (2026-08-22T22-06-41-065Z (pid …)), and ps on that pid would have answered the rest.