Skip to content

A cleanup.run parked on startup readiness can start a run after a mid-startup stop #1001

Closed

Description

@V3RON

What happened

A cleanup.run request that arrives while the daemon is still starting parks behind the startup-readiness gate. If a stop is called in the middle of startup, that parked request can still start a cleanup run once it is released. CleanupReaper.settle() (#985, PR #999) has already taken its snapshot by then, so the run is not waited for and can outlive state.db and the drivers.

What you expected

A stop during startup fails or drops parked requests instead of letting them start new work after settle().

How to reproduce

Not reproduced. The code lens of the #999 review saw it while reading the stop path and the startup-readiness gate in src/daemon/server.ts.

Environment

main at 2ccae23. Found by reading code, not by a user report. Rare: needs a manual cleanup.run during startup and a stop mid-converge.

Written by an agent.

Activity

  1. added
    bug:newReported. An agent may triage it (rule 8), and nothing more.
    on Oct 25, 2026
  2. self-assigned this
    on Oct 25, 2026
  3. V3RON commented on Oct 25, 2026

    @V3RON
    ContributorAuthor

    Reproduction

    Defect, reproduced as the issue describes. A cleanup.run parked on readiness runs after stop() has finished, and shuts down a device.

    src/daemon/server.test.ts, "DaemonServer startup readiness" › "refuses a cleanup.run parked on startup with DAEMON_STOPPING once a stop lands mid-converge, and shuts down no device" (branch bug/1001-repro). The test seeds an idle ready device, parks cleanup.run {rule: "idle-shutdown"} through DaemonServer.dispatch (the HTTP seam), awaits stop(), then lets convergence resolve.

    pnpm exec vitest run --project unit src/daemon/server.test.ts -t "refuses a cleanup.run parked on startup"
    
    AssertionError: promise resolved "[ { action: 'shutdown', …(3) } ]" instead of rejecting
     ❯ src/daemon/server.test.ts:1537  await expect(parked).rejects.toMatchObject({ code: "DAEMON_STOPPING" })
    

    Root cause

    src/daemon/server.ts:609 (#awaitReady) is the cause. It resolves when convergence succeeds and never checks #stopping, so runDispatch (src/daemon/dispatch.ts:327) goes on to the handler. The only stopping gates are at the entry points (server.ts:425 in dispatch(), :819 in #dispatchLine), and a parked request has already passed them. The symptom shows in CleanupReaper.run (src/core/reaper.ts:76): it starts a run after reaper.settle() (server.ts:642) has already taken its snapshot. This affects every parked operation, not just cleanup: lease.request, device.exec and nuke.run would also run on a stopped daemon. On the socket path the client never gets an answer, because stop() closes the connection, but the work still runs.

    Simplest fix

    At the end of #awaitReady, add if (this.#stopping) throw new DispatchError("DAEMON_STOPPING", "Daemon is stopping"). I tried this locally: the test passes, and so do all 910 tests in src/daemon/ and src/http/.

    Alternatives rejected

    • Make CleanupReaper.run refuse after dispose(): this fixes cleanup only, and other parked operations still run.
    • Make stop() await #readyPromise before it settles: a slow or hung convergence would then block daemon stop.

    Risk

    • src/daemon/AGENTS.md Readiness should add "a stop during converge → parked requests get DAEMON_STOPPING".
    • The convergence-failed path should still answer DAEMON_STARTUP_FAILED. The catch runs before #stopping is checked, so it should, but a reviewer should confirm it.

    Written by an agent.

  4. removed their assignment
    on Oct 25, 2026
  5. added
    bug:readyAn agent may fix it.
    and removed
    bug:newReported. An agent may triage it (rule 8), and nothing more.
    on Oct 25, 2026
  6. self-assigned this
    on Oct 25, 2026
  7. V3RON commented on Oct 25, 2026

    @V3RON
    ContributorAuthor

    Delivery run

    Merged: #1003 (#1001)

    Written by an agent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bug:readyAn agent may fix it.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions