Repository navigation
fix(lock): release cannot delete a lock taken over during it - #360
Conversation
… contention test models abandonment as exit release() was check-then-act. It read the lock, saw its own token and unlinked the path. A takeover landing between that read and the unlink wrote the new holder's token into the same inode, and the old holder then deleted the new holder's lock, so a third process got in with wx while the new holder was still inside. release() now takes the same hardlink claim that breakStaleLock takes for that token before it checks and unlinks, so a release and a takeover of one token exclude each other. When the claim already exists a takeover is under way and the lock is left alone. Any other link error fails every takeover too, so the plain check still runs there. The claim name is derived in one helper used by both sides. The new unit test replays a takeover just before each fs call the release makes on the lock and asserts exactly one holder remains. It fails on the old release at the unlink step. The stale-lock contention test modelled abandonment as a live process sleeping with the lock held under a 300 ms age bound. On a loaded runner a live holder was starved past that bound and legitimately broken by age, which the test reported as two holders. It now passes staleMs Infinity as the file locks do, and a holder abandons by exiting with the lock held, in waves of 8 processes so each wave also opens on a dead holder. A starved live holder is never broken, and a dead holder is broken by every spinning contender at once.
There was a problem hiding this comment.
All reported issues were addressed across 2 files
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
jwfing
left a comment
There was a problem hiding this comment.
Summary
The normal release/takeover race is addressed, but the fallback after a hardlink failure can reintroduce the same race.
Requirements context
I assessed the change against the PR description, the existing lock invariants and tests, and the context in linked PRs #345 and #358. No separate lock design document was found; the intended contract is that release must never unlink a lock that a concurrent stale takeover has acquired, while the contention test must not treat a scheduler-starved live holder as abandoned.
Findings
Critical
src/commands/compute.ts:1798-1807— OnlyEEXISTfails closed. Every otherlinkSyncerror falls through to the original read-then-unlink sequence, but an error such asENOSPC,EDQUOT, or a transient I/O failure does not prove a concurrent takeover will also fail later. An interleaving can therefore be: A's claim creation fails; A reads its token; filesystem conditions recover; B creates the claim and writes B's token into the inode; A unlinks B's lock. This is the exact race the PR intends to eliminate. Release should retry or fail closed whenever it cannot establish the claim, and the deterministic test should cover this error path.
Suggestion
(none)
Information
test/ssh-orchestration.test.ts:606-637— The deterministic test otherwise exercises takeover at each normal filesystem step and is a strong regression test for the successful-claim path.test/ssh-orchestration.test.ts:1167-1262— Using process exit withstaleMs = Infinityremoves the prior scheduler-dependent age assumption while retaining a positive stale-takeover control.src/commands/compute.ts:1794-1985— No new untrusted-input, authentication, secret-logging, dependency, or other security concern was found. Hashing the token also keeps it out of the claim filename.src/commands/compute.ts:1794-1810— The additional synchronous hash, hardlink, and unlink occur only during local lock release; no material production performance concern was found. The contention test remains bounded by its timeout.- Repository dependencies were unavailable locally (
tscwas not installed), so I could not rerun the suite in this checkout. The head commit's Linux and Windows test checks are both green.
Verdict
Request changes: fail closed when the release cannot acquire its arbitration claim before merging.
From review. A transient link error such as ENOSPC fell back to the old read-then-unlink, which a takeover could still land inside. Release now leaves the lock unless the error says the disk has no hardlinks, where no takeover can claim either.
jwfing
left a comment
There was a problem hiding this comment.
Summary
The change correctly serializes release and stale takeover using the same token-derived hardlink claim, preventing an old holder from deleting a replacement lock.
Requirements context
I assessed the change against the PR description and the lock protocol documented in src/commands/compute.ts:1775-1785 and src/commands/compute.ts:1916-1953. No separate lock specification was found; the linked PRs provide integration context, but the required behavior is defined primarily by the PR description and these in-code invariants.
Findings
Critical
(none)
Suggestion
test/ssh-orchestration.test.ts:609-629,test/ssh-orchestration.test.ts:643-653— The deterministic tests mutate the process-globalnode:fsexports and synchronize them into named ESM imports. This is restored safely withfinally, but it conflicts with the repository convention to inject side-effect dependencies rather than mock globals (.claude/skills/developing-insta-cli/SKILL.md:17-19) and could become fragile if these tests are later made concurrent. Consider exposing a narrow filesystem dependency seam for the lock implementation.
Information
src/commands/compute.ts:1794-1809,src/commands/compute.ts:1954-1987— Functionally, release and takeover now contend on the same claim. A failed claim ordinarily leaves the lock intact, while the explicitly unsupported-hardlink fallback preserves release behavior on filesystems where takeover cannot use the claim either.test/ssh-orchestration.test.ts:606-667,test/ssh-orchestration.test.ts:1196-1291— Coverage exercises takeover at each release step, fail-closed and unsupported-link errors, and simultaneous recovery from genuinely exited holders. The positive control also prevents a permanently wedged implementation from passing trivially.src/commands/compute.ts:1793-1809,src/commands/compute.ts:1982-1987— No security-relevant regression found: token material is hashed in the claim filename, no secrets are logged, and no authentication, network, dependency, or command-execution surface changes.src/commands/compute.ts:1794-1809— No material production performance concern found. Release adds one hash calculation and a bounded pair of hardlink operations on an already synchronous, local lock path; there are no new unbounded production loops or allocations.- Validation was static because the review was explicitly read-only; I did not run the test suite, which writes temporary files.
git diff --check origin/main...HEADwas clean.
Verdict
Approved under the supplied rubric: zero Critical findings. The dependency-injection point is non-blocking.
## What Version bump for v0.1.23. Since v0.1.22: - #360: a lock's release can no longer delete a lock taken over during it, and the contention test models abandonment as the holder exiting. - #358: a template's health check path is optional, and `template draft` and `template info` show it. InsForge/instacloud-platform#641, which accepts an empty path as none, is in production. - #362: `template drafts --json` prints one summary per template, and its help says so. InsForge/instacloud-platform#650, which serves the summary rows, is in production (08bc4324). ## How `package.json` and `package-lock.json` move from 0.1.22 to 0.1.23, nothing else. ## Verify - The three changes above merged with green Linux and Windows CI, and main's CI is green after each. - After merge the tag `v0.1.23` on main runs the release workflow (binaries and npm). `npm view insta version` should print 0.1.23. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
What
The Linux
testjob has failed on main since 5cdb4bc in "simultaneous stale-lock recovery still admits one holder" ("two contenders were inside the lock at once"), 24 of 104 Linux attempts since 10-01. Two causes, both fixed here.release()read the lock, then unlinked it. A takeover landing between the two writes the new holder's token into the same inode, so the old holder unlinked the NEW holder's lock and a third process got in withwx. In production only the renewal lock can hit it (a live holder broken by its 60 s age rule), so the cost was a possible duplicate renewal.How
src/commands/compute.tsrelease(:1794): takes the same hardlink claimbreakStaleLockuses as its arbiter before it checks and unlinks. When the claim cannot be taken (EEXIST is a takeover in progress, ENOENT means the lock is gone, ENOSPC or EIO says nothing about a concurrent takeover), release fails closed and leaves the file, to be broken like an abandoned lock. The one exception isLINK_UNSUPPORTED(EPERM, ENOTSUP, EOPNOTSUPP, ENOSYS), a disk without hardlinks, where no takeover can claim either, so release keeps the old check rather than wedging every lock there. The claim is removed infinally.takeoverClaim(:1986) is now the one place the claim name is built, used by both.breakStaleLockdocblock: the old line that said a late release "reads a foreign token" is replaced, and the residual paragraph now also covers a release killed between its link and its unlink (that token's lock wedges until removed by hand, as with a breaker killed mid-takeover).test/ssh-orchestration.test.ts:staleMs = Infinityas the file locks do, and abandons by the holder process exiting while it holds. 3 waves of 8 processes, each child abandons its last acquisition. A live holder is never broken however long it is starved, and every exit puts all spinners into breaking the same dead lock at once.Verify
release, the takeover-inside-release test fails with "a takeover before release step 2 left the wrong number of holders: expected [ 'B', 'C' ] to have a length of 1".LINK_UNSUPPORTEDemptied it fails with "a disk without hardlinks never released".releasefails only the takeover-inside-release test. The claim removed frombreakStaleLock(takeover still writes) fails the takeover-inside-release test every time and the contention test in 4 of 10 idle runs, so it still exercises simultaneous breaking.npm run typecheckandnpm testpass on Node 22.22.1 (102 files, 2107 tests).Not changed: the contention test no longer covers breaking a live holder by age under contention. The deterministic tests at :586 and :606 and the slow renewal holder test cover that. The test now spawns 24 processes instead of 8 (about 5 s idle, 10 s under heavy load, 60 s timeout).
#345 rewrites the same test without touching the lock, and its own
testjob is red on the release race. This change unblocks #358.🤖 Generated with Claude Code