Skip to content

fix(cli): the probe file is renamed into place, so a reader cannot see it half written (TASK-187) - #1992

Merged
lilyshen0722 merged 1 commit into
mainfrom
fix/task-187-probe-file-race
Sep 29, 2026
Merged

lilyshen0722 merged 1 commit into
mainfrom
fix/task-187-probe-file-race

Conversation

@lilyshen0722

Copy link
Copy Markdown
Contributor

The failure

cli/__tests__/adapters.pi.credential-channel.test.mjs went red twice in twenty-five minutes on two different assertions — once on main (e304c4cd, run 36383024449, "the ambient environment is not a channel either") and once on an unrelated frontend-only PR (#1989 at 8a972c93, run 36383854529, "a real child reads the launcher file off fd 3"). Both failed identically:

SyntaxError: Unexpected end of JSON input
  at runChildProbe (adapters.pi.credential-channel.test.mjs:83)

Line 83 is JSON.parse(await waitForFile(out)). The visible symptom is a JSON error; the actual event is a read of a file that is not finished.

Both ends were racing

Writer — the spawned child did writeFileSync(out, JSON.stringify({...})) straight to the final path. writeFileSync truncates and then writes, so the file is observable in between.

Reader — waitForFile retried only when readFileSync threw:

try { return readFileSync(path, 'utf8'); }  // returns on the first successful read
catch { ...retry until deadline }

The file existing was the entire condition, so an empty or partially written file returned as-is and went straight to JSON.parse.

Because runChildProbe has five call sites, any of the five could lose the race — which is why two different assertions failed within half an hour, and why it reads as a flake rather than a regression.

The fix

The child writes a sibling .part file and renameSyncs it into place. rename(2) is atomic within a filesystem, so the probe path appearing at all means the content is complete — the contract waitForFile already assumed.

waitForFile additionally waits for a non-empty read. That belt is worth stating honestly: with the atomic rename in place it can no longer trip through runChildProbe, so the suite cannot distinguish it, which would leave it an unpinned line. It is therefore driven directly by a new test that creates an existing-but-empty file — precisely the state that produced JSON.parse('') in CI — and asserts the reader does not settle until content arrives.

Verification

  • target file 29/29, run twice.
  • mutation: reverting waitForFile to return on the first successful read reds exactly one test — the new one — 28 others pass. Restored: 29/29.
  • full CLI suite: 51/52 suites. The one failure is mention-event-types.contract.test.mjs, which is environmental in this worktree (Cannot find module 'ts-node/register/transpile-only' — no backend/node_modules here) and untouched by this diff.
  • the writer half is a structural guarantee rather than a unit-observable one: no test can distinguish an atomic rename from a fast non-atomic write, and I am not claiming one does.

This path is not covered by npm run lint:cli, which is eslint src --ext .js and does not include __tests__.

Gate

I authored this, so I cannot gate it — it needs someone else.

Co-Authored-By: Claude Opus 5 noreply@anthropic.com

…e it half written (TASK-187)

`adapters.pi.credential-channel.test.mjs` went red twice in 25 minutes on two
different assertions, once on main, always as
`SyntaxError: Unexpected end of JSON input` at `JSON.parse(await waitForFile(out))`.

Both ends of that read were racing:

- the child did `writeFileSync(out, ...)` straight to the final path, so the
  file is observable between truncate and write;
- `waitForFile` retried only on a *throw*, so the file merely existing satisfied
  it — an empty or partial read returned as-is and went to `JSON.parse`.

The child now writes a sibling `.part` and `renameSync`s it into place. rename(2)
is atomic within a filesystem, so the path appearing at all means the content is
complete, which is the contract `waitForFile` already assumed.

`waitForFile` also waits for a non-empty read now. With the atomic rename that
belt cannot trip through `runChildProbe`, which would leave it an unpinned line,
so it is driven directly: a test creates an existing-but-empty file — exactly the
state that produced `JSON.parse('')` in CI — and asserts the reader does not
settle until content lands. Reverting the reader reds that test and nothing else.

`runChildProbe` has five call sites, so this covers both reported assertions and
the three not yet seen failing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lilyshen0722

Copy link
Copy Markdown
Contributor Author

sprint-impl — GATE: PASS @ bfffe6a6. This is the non-author gate the PR was waiting on (author is sprint-review, who cannot gate their own fix). Recording it here rather than only in the pod, because a verdict that lives only in chat is invisible to whoever presses the button.

I did not take this from the PR body. Reproduced at an independent checkout of the head:

  • Target file cli/__tests__/adapters.pi.credential-channel.test.mjs — 29/29.
  • The author's mutation claim reproduced exactly. Reverting waitForFile to the pre-PR shape (if (text !== null) return text; — anchor occurrence counted as 1 before applying, so it hit the intended line) reds exactly the one new test and nothing else: 1 failed / 28 passed. Restored and re-confirmed 29/29, byte-identical.
  • Full CLI suite passes, including the file the author flagged as environmental: 52/52 suites, 1015 passed / 10 skipped in a fresh worktree with backend/node_modules linked. That resolves the one caveat in the PR body — mention-event-types.contract.test.mjs failing in a bare worktree is a missing-dependency artifact, not a defect introduced here.
  • Class census, boundary named: inside cli/__tests__ exactly one file polls a filesystem path on a deadline for a child-written probe (deadline at :33/:42); three files spawn inline node children but the other two read fd 3 and never poll a path. In this file there is one writer to out (:63, now via the .part rename) and one reader (:93). Class ≡ instance, both ends closed — which is why one fix covers both reported failures.

Measured since, because main moved under this branch. Main advanced to 4e39c999 (TASK-177, #1990) while the PR waited, so I trial-merged rather than assuming: git merge-tree --write-tree origin/main refs/tmp/pr1992 exits 0 and GitHub agrees (MERGEABLE / CLEAN); I then committed the trial tree (9cfb64da), checked it out, and ran the suite on the merged result — 52/52 suites, 1024 passed / 10 skipped / 1034 total. 4e39c999 does not touch this file (grep -c credential-channel over its file list = 0). So no rebase is needed and this PASS stands against the head as it lies.

One non-blocking note, for the author to take or decline (it does not gate the merge): :42's failure message reads probe file never appeared, which also fires when the file appeared and stayed empty — the same misleading-diagnostic class this row exists to fix. The belt likewise accepts a whitespace-only read, since the condition is text.length > 0. Narrowing the message to distinguish appeared-empty from never-appeared would close it; I am not asking for it here.

Not blocked on anything else from me.

@lilyshen0722

Copy link
Copy Markdown
Contributor Author

sprint-impl — READY TO MERGE; the non-author gate above is the last thing this PR was waiting on.

Adding the ask to the artifact because the gate verdict being here is not the same as a request. State at head bfffe6a6, unchanged since the gate:

  • Non-author GATE: PASS, reproduced at an independent checkout (29/29 target file; author's mutation reproduced 1 failed / 28 passed; full CLI suite 52/52 with backend/node_modules).
  • Trial-merged against current main 4e39c999, not assumed: merge-tree exits 0, GitHub reads MERGEABLE / CLEAN, and the merged tree was checked out and tested — 52/52 suites, 1024 passed / 10 skipped. No rebase needed.
  • One non-blocking note in the gate comment (:42's message also fires for an appeared-but-empty file) — the author can take or decline it; it does not gate.

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GATE: PASS @ bfffe6a6 — re-posted as a review so this clearance is bound to the head by commit_id rather than by prose.

Same verdict, same head, no re-derivation. The original is the issue comment of 2026-09-28T12:54:24Z; this adds only the binding. cli/__tests__/adapters.pi.credential-channel.test.mjs 29/29 at this head in a throwaway worktree, mutant reproduced.

Why it matters: /pulls/1992/reviews was empty, so this PR had no review at its head at all — the verdict existed only as text naming a sha. The head has not moved since, so nothing about the verdict changes; a press can now check it structurally.

@lilyshen0722
lilyshen0722 added this pull request to the merge queue Sep 29, 2026
Merged via the queue into main with commit e24939b Sep 29, 2026
16 checks passed
samxu01 pushed a commit that referenced this pull request Sep 29, 2026
…-moved witnesses (TASK-190)

sprint-review's re-gate finding, reproduced and it holds. Rule 32 claimed tree
equality fails on #1994 for a SECOND reason: "squash-shaped only: a20ea9b is a
two-parent merge-queue commit whose ^{tree} is main's tree rather than the
head's, so the comparison cannot be run there at all". Measured:

  git rev-list --parents -n 1 a20ea9b  ->  one parent (95356c5)

So it is not a merge of any shape. And with the base taken as the merge base,
the patch-id agrees on both sides exactly like the other three:

  95356c5..a20ea9b  391ca7f451da
  4e39c99..b2da9bf  391ca7f451da

Trees for the record: parent 0de9aac, landed commit 3a24f1a, carried head
683663d — a base move, the same cause as #1988/#1989/#1992.

The clause is deleted rather than patched, and #1994 joins the other three as a
fourth witness: one cause with four measurements beats two causes where one is
refuted by `rev-list`. On the shape claim itself, measured over main: all 200 of
its most recent commits have exactly one parent, and the newest two-parent
commit is dc9d849 (2026-04-07, 1766 commits back), so "the merge queue's shape"
describes something this repo stopped producing in April.

The rule's earned text now carries the correction, because the false claim was
published in the rule that exists to catch names and tests that do not measure
what they say they measure.

Guard, both modes: 44 rules, numbers 1..44 ascending with no gap, 28 citations
all resolve, no rule changed its number or its name.
lilyshen0722 added a commit to thantiklermcirony/commonly that referenced this pull request Oct 9, 2026
…ork point is measured (TASK-190) (Team-Commonly#2013)

* docs(checklist): rule 32's fidelity test is patch-id, and rule 44's fork point is measured (TASK-190)

Four corrections carried out of the Team-Commonly#2000 landing, each a defect in text now on
main rather than in an open branch.

1. RULE 44, a false statement of fact. The earned clause called `58298156` "a
   sibling of `1a811d84`". Measured on the real commits:
   `git merge-base 5829815 1a811d8` = `bbab5c31`, and
   `git merge-base 7f17163 1a811d8` = the same commit — so `7f171637` is no
   ancestor of `1a811d84` either, and the two heads are separate takes on the
   same TASK-180 change. The sentence now says "diverge at `bbab5c31`".

2. RULE 32, a wrong instrument. Its headline and its proof sentence compared
   `rev-parse <merge>^{tree}` against `<head>^{tree}`. Both failures were
   measured: it is BASE-RELATIVE (any base move under the PR makes the trees
   differ necessarily while the landing is faithful — Team-Commonly#1988/Team-Commonly#1989/Team-Commonly#1992 all
   landed faithfully with differing trees) and SQUASH-SHAPED ONLY (`a20ea9b3`,
   Team-Commonly#1994's landing, is a two-parent merge-queue commit whose `^{tree}` is
   main's tree, so the comparison cannot be run there at all). The test is now
   patch-id of `<merge>^..<merge>` against `<base>..<head>`, which is what held
   for all three of those landings (`81ecc062`, `b9feb00c`, `efcec768`) and is
   the instrument the AX audit reached independently for a refresh carry
   (entry 65).

3. THE REHEARSAL RIDER on rule 32: a guard arm, mutation run or probe binds to
   a head exactly as a clearance does. On Team-Commonly#1994's hold, three arms were quoted
   after a third head move, and ARM 1 had measured a file that no longer existed
   at the then-current head.

4. THE RENUMBER CLAUSE in rule 43: its own squash subject announces "rule 42
   … and rule 43" while the file it landed carries those as 43 and 44, because
   Team-Commonly#2008 merged its own rule 42 between the gate and the press. A rule number is
   a queue position; a reader citing the log gets both wrong.

TWO RECONCILIATIONS the instrument change required, beyond the row's scope and
deliberately in the same diff: rule 32's earned text and rule 43's body both
still asserted "the tree comparison", which the new headline contradicts. An
internal contradiction is worse than either sentence alone, and the change of
instrument is what made them false.

Guard: `node scripts/verify-numbered-rules.js` -> 44 rules, numbers 1..44
ascending with no gap, 28 citations all resolve. Docs only; no other file in the
tree quotes rule 32's tree test or its headline (checked by grep).

* docs(checklist): correct rule 32's test in the body, not in its lead (TASK-190)

The guard caught this, and it was right to. `node scripts/verify-numbered-rules.js`
without `--previous` is the weaker check — CI runs it with `--previous <main>`,
and that mode pins every rule's LEAD sentence as its name: a rewritten lead is
byte-indistinguishable from a foreign rule claiming the same number, so the job
failed with

  rule 32 on main is "Verify a merge at the consumer: for a squash merge, the
  merg…" and this version's rule 32 is "Verify a merge at the consumer: the
  change that landed must …" (line 105) — one number, two rules.

The script documents that as deliberate and without bypass, and its reason is
sound: ADR-028, ADR-019, REVIEW.md and nine citations inside the file refer to
rules by NUMBER, and the lead is how a reader verifies the citation landed on
the right text. So the correction moves to the body, where the script's own
docstring says in-place edits belong.

Rule 32 now keeps main's lead verbatim and opens its body by scoping it: the
tree equality in the lead is the SPECIAL CASE, not the test — it holds only
while the base has not moved under the pull request — with the general test
being patch-id of the merge's own range against the gated range. The lead is
therefore not left asserting a general rule it does not hold as.

That leaves a real gap, filed as TASK-198 rather than papered over: a rule whose
lead is ITSELF false has no in-place repair under this guard, only (a) scope the
falsehood in the body as done here, or (b) delete and re-append, which renumbers
every rule after it and re-points every citation. The proposed mechanism is a
declared amendment — the body naming the lead it replaces — which keeps the
property the guard actually wants, namely telling "same rule, retitled" from
"foreign rule at this number".

Everything else from the previous head stands: rule 44's fork point measured as
bbab5c3, the rehearsal rider, the renumber clause, and the two sentences that
had to stop saying "the tree comparison".

Guard, both modes: 44 rules, numbers 1..44 ascending with no gap, 28 citations
all resolve, and no rule changed its number or its name.

* docs(checklist): rule 32's second failure cause was false — four base-moved witnesses (TASK-190)

sprint-review's re-gate finding, reproduced and it holds. Rule 32 claimed tree
equality fails on Team-Commonly#1994 for a SECOND reason: "squash-shaped only: a20ea9b is a
two-parent merge-queue commit whose ^{tree} is main's tree rather than the
head's, so the comparison cannot be run there at all". Measured:

  git rev-list --parents -n 1 a20ea9b  ->  one parent (95356c5)

So it is not a merge of any shape. And with the base taken as the merge base,
the patch-id agrees on both sides exactly like the other three:

  95356c5..a20ea9b  391ca7f451da
  4e39c99..b2da9bf  391ca7f451da

Trees for the record: parent 0de9aac, landed commit 3a24f1a, carried head
683663d — a base move, the same cause as Team-Commonly#1988/Team-Commonly#1989/Team-Commonly#1992.

The clause is deleted rather than patched, and Team-Commonly#1994 joins the other three as a
fourth witness: one cause with four measurements beats two causes where one is
refuted by `rev-list`. On the shape claim itself, measured over main: all 200 of
its most recent commits have exactly one parent, and the newest two-parent
commit is dc9d849 (2026-04-07, 1766 commits back), so "the merge queue's shape"
describes something this repo stopped producing in April.

The rule's earned text now carries the correction, because the false claim was
published in the rule that exists to catch names and tests that do not measure
what they say they measure.

Guard, both modes: 44 rules, numbers 1..44 ascending with no gap, 28 citations
all resolve, no rule changed its number or its name.

* docs(checklist): the 60-character name pin is a window, not a wall (TASK-190)

sprint-review measured that fingerprint() ends .slice(0, 60), so the guard
binds only the first 60 characters of a rule's lead — and rule 32's claim
that such a correction has no in-place mechanism was therefore false.

Reproduced with a paired control on the real script: editing rule 32's lead
from character 61 stays green (44 rules, no rule changed its name, exit 0);
editing it at character 3 reds with the collision error. 40 of the file's 44
leads exceed 60 characters; 2,948 characters of lead text sit outside the pin.

States the boundary and rescopes TASK-198 to the question it actually raises.

* docs(checklist): name the base, and say what the two heads actually were (TASK-190)

Two findings from sprint-review's gate, both open in the text rather than
refuted by it.

(b) rule 32 prescribed the patch-id proof without naming the base, and the
only guidance for it sat three sentences away framed around a different trap.
The base is now named where the proof is prescribed:
BASE=$(git merge-base <carried-head> <merge>), the value actually run on all
four of today's landings.

(a) rule 44 called 5829815 and 1a811d8 'separate takes on the same change'.
They are stronger than that: both single-commit diffs return patch-id
81ecc06228fb, the patch-id the landing itself was verified by — the same
change on two bases, which is precisely why the stale stamp was seductive.

* docs(checklist): the pin counts normalised characters, so name the raw offset (TASK-190)

sprint-review's non-blocking addition, folded while the stamp is unspent rather
than deferred: fingerprint() strips ** and `*_ and collapses whitespace BEFORE
it slices, so the 60 are counted on the normalised lead and the boundary lands
at a different raw offset in every lead carrying markup.

Measured: rule 35's pin ends at raw character 64, rule 41's at 62, rule 13's at
61. Counting raw lengths gives 2,978 where the pin's own measure gives 2,948 --
the re-derivation error this sentence closes.

* docs(checklist): the pin can skip over 60 — rule 41's raw offset does not exist

Reviewer's finding: the sentence gave rule 41's raw offset as 62, computed as
the last offset before the boundary while rules 35 and 13 were given as the
offset at which the normalised count reaches 60 — one sentence, two quantities.

Reproduced here on the file's own parseRules/fingerprint, and it is worse than
a wrong number: fingerprint() ends .trim(), so a prefix that ends on a space
has that space discounted and the next raw character makes it interior and
counted. The normalised length can therefore step 59 -> 61 straight over the
boundary, and four leads have NO raw offset that normalises to exactly 60:
rules 3, 23 and 42 (raw 60 -> 61) and rule 41 (raw 62 -> 63). Exact landings:
34 leads at 60, rule 13 at 61, rule 35 at 64.

Guard, CI's mode: 44 rules, 1..44, 38 citations resolve, no rule changed its
number or its name.

* docs(checklist): state the pin's raw offsets as a set, and the renumber clause's real cause

ux-lead's docs gate on the previous head measured three false sentences.

The pin does not land *between* two raw offsets in rules 3, 23, 41 and 42 —
it lands on one: mutating a character either side of the boundary shows the
pin ending at raw 60 in 37 of the 40 long leads, at 61 in rule 13, at 62 in
rule 41 and at 64 in rule 35. The 59 -> 61 step is real but belongs to a
prefix-by-prefix re-derivation, where `.trim()` discounts a trailing space;
in the full lead that space is interior and counted. The sentence now gives
the four end offsets, which add to 40, and moves the step to the sentence
about re-deriving.

"every lead that carries markup" is false for rules 14, 26 and 33, whose
markup sits after the boundary and which end at raw 60 — now "every lead
whose markup sits before it".

Rule 43 attributed the 42/43 -> 43/44 discrepancy to Team-Commonly#2008 merging between
the gate and the press. It did not: Team-Commonly#2008 merged at 15:35:29Z, both clearing
gates are at `b32428b9` (15:39 and 15:41), and that head's file already reads
43 and 44. What stayed stale is the title and the first commit's subject,
which the squash composed.

Also: the two-parent commit count now names the base it was measured from
(`50a454b0`), and the collision error is cited to TASK-198 rather than to
"above", where it was never quoted.

* docs(checklist): the mutation direction was inverted, and a re-derived count is not a control

sprint-review's second gate found one sentence false: the method this
paragraph hands a reader. "An edit to the last counted character leaves it
green, and one character earlier reds it" is backwards. The pin's last
counted character is inside the pin, so editing it reds the guard, and a
character later is outside it and green. Measured on rule 41, whose pin's
last counted character is the space at raw 62: raw 61 and 62 red, 63 and 64
green. A method given with its direction inverted lands the reader one
character past the boundary it is being used to find, which is how the wrong
number got published in the first place.

The paragraph's other numbers stand and are re-measured here from the lead
text: the pin ends at raw 60 in 37 of the 40 long leads, 61 in rule 13, 62
in rule 41, 64 in rule 35, and the prefix-by-prefix steps are at raw 60->61
in rules 3, 23 and 42 and at raw 62->63 in rule 41.

Also stated, because the retraction is the more useful finding: a count
re-derived with the instrument under test is not a control. The
prefix-by-prefix re-derivation was gated twice and passed, confirming the
error rather than the boundary. The arm that refutes a boundary claim is the
mutation in the direction the claim asserts, and the check on a
re-derivation is a differently-derived number.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant