Skip to content

fix(exec): claim the exec hold atomically and release only the claim's own - #854

Merged
unohee merged 1 commit into
mainfrom
fix/exec-hold-release-race
Oct 25, 2026
Merged

unohee merged 1 commit into
mainfrom
fix/exec-hold-release-race

Conversation

@unohee

@unohee unohee commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

Closes the TOCTOU window left open in #847 (reported there as the one unfixed concurrency item).

exec --issue read isExecHeld, then called markExecHold. Two orchestrators deploying the same issue could both read "not held" and both write. When the first deploy was then refused it cleared the hold, although the second deploy's run depended on it, and idle_fill could take that run.

  • claimExecHold reads and writes under one IMMEDIATE transaction and returns { wasHeld, stamp }. Each claim gets a strictly increasing updated_at, so two claims never share a stamp even within one millisecond.
  • releaseExecHoldIf(issueId, stamp) deletes the row only if it still carries that stamp. A later claim, or an exec answer re-mark, moves updated_at, so the row is left alone.
  • exec deploy uses these through new claimHold / releaseHold seams (replacing isHeld / clearHold); exec answer still uses markExecHold.

Verification

  • npm run ci passes. test:coverage on this tree: 7905 passed, 1 failed in src/support/fileLock.test.ts. It passes 3/3 alone and this diff does not touch it (load-dependent, the same pattern as the verify tests earlier in this stack).
  • Mutations: making the release unconditional fails 4 tests; making claim always report wasHeld: false fails 3.
  • New store tests cover the interleaving that used to strand a live run (claim A, claim B, release A leaves B's hold), same-millisecond claims, expiry, an exec answer re-mark, and fail-closed on an unwritable store. New CLI tests run the deploy against the real store (execDeployHold.test.ts, split out because execCommand.test.ts is at its line cap).

Known remaining behaviour

If both of two concurrent deploys are refused, the second one sees wasHeld: true and keeps the hold, so it stays until the 24h TTL. That errs toward protecting a run rather than stranding one.

…s own (AGT-4785 review)

Two orchestrators deploying one issue could both read 'not held', both write,
and the first refused deploy then cleared the hold the second still needed.
claimExecHold reads and writes under one immediate transaction and returns a
strictly increasing stamp; releaseExecHoldIf deletes only that stamp.
@unohee
unohee merged commit d8b63a6 into main Oct 25, 2026
13 of 14 checks passed
@unohee
unohee deleted the fix/exec-hold-release-race branch October 25, 2026 03:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant