A health recovery that starts while the daemon stop waits for cleanup is not waited for #1000

Activity

1. V3RON

   added

   bug:newReported. An agent may triage it (rule 8), and nothing more.Reported. An agent may triage it (rule 8), and nothing more.

   on Oct 25, 2026
2. V3RON

   self-assigned this

   on Oct 25, 2026
3. V3RON commented on Oct 25, 2026

   

   V3RON

   on Oct 25, 2026

   ContributorAuthor

   More actions

   Reproduction

   Defect, and wider than reported: a stop waits for no health-monitor work, so a recovery reboot started before the stop also outlives it.

   src/daemon/main.test.ts, "leaves no recovery reboot $name running once the drivers are disposed" (describe … (#1000)), driven through startDaemon with a driver that holds the recover boot:

   pnpm exec vitest run --project unit src/daemon/main.test.ts -t "#1000"
   
   started before the stop:
     AssertionError: expected [ [ 'recover started', 'recover finished', 'dispose' ] ]
     to deep equally contain [ 'recover started', 'dispose', 'recover finished' ]
   due while the stop waits for a cleanup run:
     AssertionError: expected [ [ 'recover started', 'recover finished', 'dispose' ], [ 'dispose' ] ]
     to deep equally contain [ 'recover started', 'dispose', 'recover finished' ]
   

   Case 2 accepts both fixes: the stop waits, or the recovery never starts.

   Root cause

   LeaseHealthMonitor.dispose() (src/leasing/lease-health-monitor.ts:77) only cancels timers. The tick (:101) and each recovery (:257) are fire-and-forget promises nobody keeps, so there is nothing to await, and DaemonServer.#stop (src/daemon/server.ts:643) awaits nothing from the monitor before options.dispose (:649) disposes drivers and closes state.db. The armed tick during reaper.settle() (:642) is just one way in; #dispatchRecovery (:251) also ignores #disposed, so a running tick can start a recovery after dispose.

   Simplest fix

   LeaseHealthMonitor.settle() awaiting the running tick and recoveries; #dispatchRecovery returns once disposed. In #stop, dispose the monitor beside reaper.dispose(), then await its settle() after reaper.settle(); update step 4 of docs/internal/flows/startup.md. Tried locally (not pushed): the test, server.test.ts and lease-health-monitor.test.ts pass.

   Alternatives rejected

   • Only disposing before the reaper wait: case 1 still fails.
   • Draining via leasing.settle(): runs after the monitor is disposed; the server owns its stop, like the reaper.
   • Aborting the reboot: makeReady takes no signal; a half reboot is worse than waiting.

   Risk

   • A stop may now wait up to one recovery reboot (~30 s); check the SIGTERM and update paths.
   • After dispose() resolves a backoff early, #recover must still return without rebooting.
   • A tick's #giveUp → releaseDeviceLost may run during the stop, before leasing.settle() drains reclaims.

   Written by an agent.
4. V3RON

   removed their assignment

   on Oct 25, 2026
5. V3RON

   added

   bug:readyAn agent may fix it.An agent may fix it.

   and removed

   bug:newReported. An agent may triage it (rule 8), and nothing more.Reported. An agent may triage it (rule 8), and nothing more.

   on Oct 25, 2026
6. V3RON

   self-assigned this

   on Oct 25, 2026
7. V3RON

   added 3 commits that reference this issue on Oct 25, 2026

   fix(daemon): a stop waits for health ticks and recoveries in flight (#…

   ...

   9ad26e5

   test(leasing): settle waits for a health tick running at dispose and …

   ...

   e6f886f

   fix(daemon): a stop waits for health ticks and recoveries in flight (#…

   ...

   2522402
8. V3RON

   closed this as completedin #1004on Oct 25, 2026
9. V3RON commented on Oct 25, 2026

   

   V3RON

   on Oct 25, 2026

   ContributorAuthor

   More actions

   Delivery run

   Merged: #1004 (#1000)

   Written by an agent.