Repository navigation
testing: task-end judge reviews test files a git pull or checkout brought in #6016
Description
Problem
The task-end test judge reviews tests the session never wrote. In one session it judged 10, 89, 10, 14 and 4 test blocks across five stop events, all from harness-ops, disk-hygiene, guardrails, harness-config and skill-quality. None of them were in the session's change, which was two ADR markdown files (#6000).
Cause
plugins/testing/hooks/test-scan-bash.sh (lines 38-58) takes every path in Claude Code's tool_response.bashEditDiff.changedFiles whose basename matches a test glob, runs test-scan on it as a Write or Edit, and leaves the per-write session record that the task-end judge reads. A Bash call that rewrites the tree from git history reports every file it touched in changedFiles. Examples are git pull --ff-only after gh pr update-branch, git merge origin/main, git switch and git checkout. Claude Code's report is accurate, because those files did change on disk. The hook's reading of a checkout as authorship is the defect.
Impact
- Each stop event blocks with a "show the user each verdict" instruction about code outside the change, so a docs-only PR produced five rounds of irrelevant verdicts, one of them a FLAG on another plugin's test.
- Judge runs are spent on code nobody in the session wrote.
- Verdicts about the session's own tests get buried among the unrelated ones.
Expected
Only tests the session authored are recorded. Possible fixes, cheapest first:
- Skip the record when the Bash command is a git tree-rewriting operation (
pull,merge,switch,checkout,reset,rebase,cherry-pick,stash). - Or record a changed file only when its new content differs from the blob it has at
HEADafter the call. Files git checked out match a committed blob, while a file the session wrote does not, until the session commits it.
Option 2 is command-agnostic, so it also covers gh pr checkout and a script that runs git.
Repro
With test_guards_enabled and test_judge_enabled on and bashEditDiffEnabled true: on a branch behind main, run gh pr update-branch <n> then git pull --ff-only through the Bash tool, then end the turn. The judge reviews test files that came in from main.
Activity
kyle-sexton commentedon Oct 3, 2026
Another occurrence, this time from
git rebase(testing 0.21.3, Windows/Git Bash). It addsrebaseto the commands this issue lists and matches the cause described above. Related: #6060 (verdicts routed to the parent session instead of the writing subagent).c3376b14d). The judge's next run judged 11 blocks, 0 FLAG; 10 of them were inlib/prerequisites.test.mjs.git log origin/main..HEAD -- lib/prerequisites.test.mjsprints nothing on that worktree, and its last commits are main's79fe3dcc4(refactor(prerequisites): convert the existing prerequisites.json files and move the fleet reader to the node checker #5980) andd2724acf5(feat(prerequisites): one prerequisites schema, a node checker and a ci gate for undeclared tools #5919). It arrived only through the rebase.Option 2 above (record a changed file only when its content differs from its
HEADblob after the call) covers this case without listing commands, since a rebased-in file matches a committed blob.Source: local handoff item
20261003-075100-test-judge-judges-files-from-a-rebase(retired into this comment).kyle-sexton commentedon Oct 4, 2026
This was generated by AI during triage.
Decision defaulted: record a Bash-changed test file only when its content differs from its HEAD blob after the call (option 2), not a git-command skip list (option 1) — veto before merge
Basis: verified, plugins/testing/hooks/test-scan-bash.sh:47-70 (every
bashEditDiff.changedFilespath matching a test glob is scanned and recorded; no command or HEAD check exists, testing 0.22.5); option 2 also covers thegit rebasecase in #6016 (comment), where the owner endorses it.Agent Brief
Type: Bug
Summary: The task-end test judge should only see test files the session wrote, not files a git checkout, pull, merge, rebase or similar brought into the tree.
Current behavior:
The PostToolUse Bash hook that forwards
tool_response.bashEditDiff.changedFilesto test-scan treats every changed path whose basename matches a test-file glob as a Write or Edit by the session. It leaves the per-write session record the task-end judge reads. Any Bash call that rewrites the tree from git history (git pull,merge,switch,checkout,rebase,gh pr checkout, a script that runs git) therefore makes the judge review tests the session never wrote, and each Stop event blocks with verdicts about unrelated code.Desired behavior:
HEADis neither scanned nor recorded: git put it there.HEAD, or is untracked / absent fromHEAD, is scanned and recorded exactly as today.git diff/git statussee it), so line-ending conversion (autocrlf on Windows) does not count as a session edit.Key interfaces:
Acceptance criteria:
Out of scope:
Routing: decision-defaulted, work class C3 (bug fix, one plugin hook, reversible). Priority medium: noisy and wasteful, not blocking other work.
kyle-sexton commentedon Oct 4, 2026
This was generated by AI during work-loop.
Ratify autonomous dispatch of this C3 bug-fix item (with its defaulted decision)?
Classification:
work-class: scoped(C3), bug-fix-shaped (confirmed at triage). Intended dispatch: the work-loop worker lane, through/work-items:work, to a draft PR the lane never merges. First drain is not complete and no item-level ratification exists, so it is queued here. Ratify through/work-items:attend-queue.Lane: work-loop, instance melo-worker-1, cycle 2.