The idle reaper spares an agent whose backgrounded tool call is still running, and restarts its idle clock when the job ends

[webdevcody]

Closes #78. An agent you switched away from could lose a backgrounded tool call five minutes later, when the IDLE REAPER killed its whole process tree; the reaper now sees that job in the process tree, waits for it, and restarts the idle clock when it ends.

Contents: 🐛 Symptom · 🔍 Cause · ✅ Fix · 📸 Before / After · 🔁 State · ⚠️ Risk · 🔧 Technical overview · 🧪 Proof · 📝 Notes

🐛 Symptom

Start a long background task inside a Claude session — a run_in_background Bash call, a Monitor watch — let the turn end, and switch to another session, project or worktree. Once the old session has sat unwatched for session_idle_timeout (5m by default), the DAEMON kills its PTY, and the SIGHUP → SIGKILL sweep takes the background child with it. The conversation resumes seamlessly on the next attach, so nothing says the work is gone. Terminals were already spared while a command runs; agents were not.

🔍 Cause

The reaper's only exemption for an agent was its hook-fed status, RUNNING or NEEDS FEEDBACK. That status flips to FINISHED the instant the turn's Stop hook fires, and a backgrounded tool call is precisely a child that outlives its turn — the status machine has no notion of it. The issue's suggested one-liner, the terminals' "any child alive" pgrep -P check, does not work for agents: every Claude session in this repo already holds a stdio MCP server (and a caffeinate while it works) as a permanent child, so that check would have exempted every agent from reaping for good.

✅ Fix

• A detached job spares the agent. The reaper reads the agent's process tree once per candidate past the timeout and keeps the session when any process under the CLI leads a terminal session of its own.
  • That is how Claude spawns a backgrounded Bash call or a Monitor watch, and how Codex runs a shell command: cut loose with setsid, the s flag in ps.
  • Helpers the CLI keeps inside its own session — MCP servers, caffeinate, Codex's code-mode host — do not count, so an idle agent still reads as idle and still gets reaped.
• The idle clock restarts when the job ends. A spared agent is restamped, not just skipped, so it gets the full Settings › Sessions › session_idle_timeout again from the moment its job finishes.
  • That moment is when the CLI wakes to read the result; a sweep landing in the gap would have killed it mid-notification.
  • An agent with a long-lived detached child (a dev server it started) is never reaped while it runs — the deal a terminal with a job already gets.
• Unchanged. RUNNING and NEEDS FEEDBACK agents, attached sessions, in-view worktrees, RUN TERMINALS and terminals with a command running are spared exactly as before; the PREWARM POOL keeps its own reaper; "off" still switches reaping off.
• Docs. The session_idle_timeout doc comment no longer promises to spare pinned agents, a feature that no longer exists; How it works and Configuration name the new exemption.

📸 Before / After

No PNG: this is a DAEMON mechanism with no screen of its own — the affected session looks identical before and after, which is the bug. The proof is the process tree the reaper now reads. A live Claude session in this repo, with a backgrounded sleep 240 Bash call and a Monitor watch running (ps -axo pid=,ppid=,pgid=,stat=,etime=,command=, indented by parent):

95705 pg=95705 Ss+   02:00 claude --model fable …                 ← the PTY child; leads the session, foreground job
  96156 pg=95705 S+   01:59 …/Python.app/Contents/MacOS/Python …  ← stdio MCP server: same session, same group
  9228  pg=95705 S+   00:07 caffeinate -i -t 300                  ← helper: same session
  8534  pg=8534  Ss   00:21 /bin/zsh -c source …snapshot… …       ← run_in_background Bash: its own session (s)
    8540 pg=8534 S    00:21 sleep 240
  9266  pg=9266  Ss   00:07 /bin/zsh -c source …snapshot… …       ← Monitor watch: its own session (s)
    9270 pg=9266 S    00:07 sleep 200

Before this PR the reaper looked only at the row's status and killed the lot. After it, the two Ss rows spare the session; the S+ rows alone would not. Codex's shell tool places its command the same way (sleep 60 under codex exec shows as SNs, while codex-code-mode-host shows as SN).

🔁 State

flowchart TD
  sweep["IDLE REAPER sweep, every 15 s"] --> aged{"unwatched past<br/>session_idle_timeout?"}
  aged -- no --> keep["keep"]
  aged -- yes --> status{"RUNNING or<br/>NEEDS FEEDBACK?"}
  status -- yes --> keep
  status -- no --> tree{"a process under the CLI<br/>leading its own session?"}
  tree -- "yes: backgrounded Bash,<br/>Monitor, Codex shell" --> spare["spare + restart the idle clock"]
  tree -- "no: only MCP servers,<br/>caffeinate, code-mode host" --> kill["kill_session: SIGHUP, then SIGKILL the tree"]
  status -. "before #78: straight to the kill" .-> kill
  classDef new fill:#dff5e1,stroke:#2e7d32,color:#000
  class tree,spare new

Loading

⚠️ Risk

Verdict: 🟢 Low risk — one more exemption in an existing sweep, read from a ps the DAEMON already runs, with no protocol or store change.

	Level	Why
🔒 Security & production	Low	No new surface: the DAEMON already execs ps with fixed arguments in the kill sweep; this adds one read-only stat= column and no user-controlled input reaches argv. The failure mode is conservative — a ps that fails, or an unknown child pid, counts as busy and spares the session.
⚡ Performance	Low	One ps -axo (~10 ms) per sweep candidate that is already past the timeout and not RUNNING — a handful at most every 15 s, off the PTY byte path and the TUI draw. Terminals already paid the same per-candidate pgrep.
🧩 Fit with the codebase	Low	Mirrors the terminals' shell_has_children exemption; the ps parse is shared with the kill sweep's process-group walk, which keeps its behaviour and its test.

Rollback: git revert of the merge undoes everything — no PROTOCOL VERSION bump, no store migration, no pushed branch; only the config doc comment and two docs pages change alongside the code.

🔧 Technical overview

• The line that mattered. crates/nebula-daemon/src/registry.rs — reap_idle_sessions spared an agent only on Running | NeedsFeedback; now a FINISHED agent is spared, and restamped via touch_session, when agent_has_detached_job finds a session leader under its PTY child.
• The check. crates/nebula-daemon/src/pty/mod.rs — detached_job_under runs ps -axo pid=,ppid=,pgid=,stat= and asks detached_job_in_table whether any strict descendant of the child has the s flag. parse_ps_table and descendants are shared with process_groups_in_table, which the kill sweep uses; a table without a stat column still parses.
• Why the session, not the process group. Under zsh the agent is the PTY child, but under bash (or a claude function) the login shell forks it as a job in a group of its own, so "a group other than the child's" would flag the agent itself. A session boundary is only ever created on purpose, by the CLI, for work it means to keep running.
• Why it was missed. The existing reaper test spares a terminal with sleep 30 and reaps an idle agent; no test gave an agent a child that outlives its turn.
• Why not the issue's pgrep -P. Measured on this machine: every idle Claude session has a live MCP-server child in its own process group, so "any child" would have switched reaping off for agents entirely.
• Not verified. cursor-agent could not be probed (it needs a login); a tool child it does not detach would simply keep today's behaviour.

🧪 Proof

• Regression test. idle_agent_with_a_detached_job_is_spared_until_it_ends in crates/nebula/tests/e2e_pty.rs — two stand-in agents: one whose child runs in a session of its own is spared while the resident is reaped, then reaped itself a full timeout after the job ends; fails on origin/main, passes here.
• Unit test. detached_job_is_a_session_leader_below_the_agent in crates/nebula-daemon/src/pty/mod.rs — synthetic ps tables for the idle, busy, and bash-wrapped shapes, plus the kill sweep's existing table test.
• Gate. cargo clippy --workspace --all-targets reports no errors (its warnings are all in code from Agent harnesses move to a config registry, with Muse and Grok on board #79, none in the files here); cargo test — 1241 tests pass: nebula-core 30, nebula-daemon 267, nebula-tui 837, the nebula binary 49, browser_cli 7, config_cli 5, help_cli 6, tunnel_cli 3, e2e_pty 30 (the new test included), e2e_tui 7 of 9 — the two e2e_tui failures (nebula_open_from_inside_a_session_raises_the_file_tabs, tui_projects_worktrees_agents_navigation) fail identically on untouched main. Run in a detached worktree at origin/main + this commit, so no other session's uncommitted work was in the build.

📝 Notes

• Branches from origin/main after Agent harnesses move to a config registry, with Muse and Grok on board #79; no conflicts with main at the time of writing.
• cargo fmt --check is already red on origin/main in files this PR does not touch (harness.rs, session_title.rs, agent_picker.rs, the TUI config.rs, …, from Agent harnesses move to a config registry, with Muse and Grok on board #79); the files here are formatted, and a follow-up cargo fmt --all on main would clear the rest.

🤖 Generated with Claude Code

[claude]

ps_table() never checks ps's exit status — Command::output() only returns Err on spawn failure, so a ps that spawns but exits non-zero (e.g. a ps that doesn't support the -axo pid=,ppid=,pgid=,stat= format, like BusyBox/Alpine) yields Ok with empty stdout, i.e. Some(""), not None.

That flows into detached_job_under: detached_job_in_table("", root) parses zero rows and returns false ("not busy"), which contradicts that function's own doc comment two lines above it: "A failed ps counts as busy: never kill what can't be inspected."

Since agent_has_detached_job uses this to decide whether the idle reaper spares a Finished-status agent with a live backgrounded job, a ps invocation that runs-but-fails on a given host would silently reintroduce the exact #78 bug this PR fixes: the reaper would kill the session while its backgrounded job is still running.

Suggested change

	fn ps_table() -> Option<String> {
	let out = std::process::Command::new("ps")
	.args(["-axo", "pid=,ppid=,pgid=,stat="])
	.output()
	.ok()
	.and_then(|o| String::from_utf8(o.stdout).ok())
	.unwrap_or_default();
	process_groups_in_table(&table, root)
	.ok()?;
	String::from_utf8(out.stdout).ok()
	}
	fn ps_table() -> Option<String> {
	let out = std::process::Command::new("ps")
	.args(["-axo", "pid=,ppid=,pgid=,stat="])
	.output()
	.ok()?;
	if !out.status.success() {
	return None;
	}
	String::from_utf8(out.stdout).ok()
	}