Filed by Claude Code from a measurement handed back by a dev agent during the patch round on #15968.
The defect
node scripts/pm/dispatch-gates.mjs --commands prints an "Artifact rosters" block. Two of its entries are reported GREEN while measuring nothing about the PR under review:
pnpm check:partof-closing-keywordpnpm check:single-claim-paths
Both pnpm script names resolve to a --self-test invocation only. Their green is a statement about the checker's own fixtures, not about the pull request.
The invocation that actually judges a PR is the bare node form, and on a developer machine it does not run at all:
node scripts/check-partof-closing-keyword.mjs
→ exit 2, "NOT WIRED … judged nothing"
node scripts/check-single-claim-paths.mjs
→ exit 2, "NOT WIRED — PR_NUMBER is not set"
So a seat harvesting the roster block sees two greens that read as PR clearance and are not. The correct reading of those two rows is NOT MEASURED (exit 2), and the roster presents the opposite.
Why this matters more than a cosmetic mislabel
This is the failure class this repo has now hit twelve times: a reading that cannot fail is indistinguishable from one that passed. A --self-test cannot fail because of anything in the PR, so its green carries no information about the PR — yet it sits in the same harvested list, in the same shape, as gates whose green does.
The repair principle that keeps being re-derived: name the axis your control discriminates on, and check that it is the axis that can fail. A row in a harvested gate roster should either judge the diff or say plainly that it cannot.
Reproduction
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands
# read the "Artifact rosters" block; note the two pnpm rows above
# then compare against the bare node invocations, capturing the exit code
# BEFORE any pipe:
node scripts/check-partof-closing-keyword.mjs > /tmp/a.log 2>&1; echo $? # 2
node scripts/check-single-claim-paths.mjs > /tmp/b.log 2>&1; echo $? # 2
The discriminator is the echoed command in the roster: for these two rows it ends in --self-test, for the rows whose green is meaningful it does not.
Not asserted
Whether this is by design is not settled here. The roster block is documented as sitting outside the reconciliation total, and it is possible the two --self-test entries are there deliberately as checker-health rows rather than as PR verdicts. If so, the ask is narrower: the block should distinguish a checker-health row from a PR-judging row, so a harvest cannot read one as the other.
No grading of domain:* or type is applied here — that is triage's call.
Filed by Claude Code from a measurement handed back by a dev agent during the patch round on #15968.
The defect
node scripts/pm/dispatch-gates.mjs --commandsprints an "Artifact rosters" block. Two of its entries are reported GREEN while measuring nothing about the PR under review:pnpm check:partof-closing-keywordpnpm check:single-claim-pathsBoth pnpm script names resolve to a
--self-testinvocation only. Their green is a statement about the checker's own fixtures, not about the pull request.The invocation that actually judges a PR is the bare
nodeform, and on a developer machine it does not run at all:So a seat harvesting the roster block sees two greens that read as PR clearance and are not. The correct reading of those two rows is NOT MEASURED (exit 2), and the roster presents the opposite.
Why this matters more than a cosmetic mislabel
This is the failure class this repo has now hit twelve times: a reading that cannot fail is indistinguishable from one that passed. A
--self-testcannot fail because of anything in the PR, so its green carries no information about the PR — yet it sits in the same harvested list, in the same shape, as gates whose green does.The repair principle that keeps being re-derived: name the axis your control discriminates on, and check that it is the axis that can fail. A row in a harvested gate roster should either judge the diff or say plainly that it cannot.
Reproduction
The discriminator is the echoed command in the roster: for these two rows it ends in
--self-test, for the rows whose green is meaningful it does not.Not asserted
Whether this is by design is not settled here. The roster block is documented as sitting outside the reconciliation total, and it is possible the two
--self-testentries are there deliberately as checker-health rows rather than as PR verdicts. If so, the ask is narrower: the block should distinguish a checker-health row from a PR-judging row, so a harvest cannot read one as the other.No grading of
domain:*or type is applied here — that is triage's call.