fix(claude-code-review): tell the reviewer not to redirect/pipe gh pr diff - #542
Conversation
… diff On a large PR, the reviewer would reach for `gh pr diff ... > file; wc -l file` (or a pipe) to chunk the diff, and every such compound command was denied wholesale, while writing to any file (including /tmp) is a hard sandbox block rather than a permission prompt. It burned through the denial budget on variant after variant with no verdict ever produced (33 denials measured on ucdavis/win#78, run 32415477507). Fixes #541.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Claude Code's Bash tool caps inline output at ~30,000 characters; past that a valid result comes back as a preview plus a harness-saved file path, not the full text. This is true of the very PR this fix cites as its motivating evidence (ucdavis/win#78, 114,587 bytes / 2,464 lines). Tell the reviewer to Read that harness-saved path instead of claiming the bare command always returns everything inline -- the file itself is fine to read, since it's the harness's own write, not the agent's. Addresses review finding on PR #542.
Non-blocking review note on PR #542: "anywhere ... no retry or different path will get past" read as a universal property of every runs-on runner, when it's only confirmed for the observed sandbox. Reworded to state what was actually tried and observed blocked.
|
Both items from this review round are addressed:
Reviewed commit for this write-up was 17de2c2; current head is 72b7dda. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Claude finished review — View run Code Review: Morrison-Lab/gha PR #542What changed since the last reviewThe last review round (commit One new commit has landed since then, bringing the head to
This is the exact, explicitly-non-blocking item the last review round flagged for awareness ("not raising this as a finding") and did not require for the "Ready for merge" verdict. VerificationI compared this commit's wording against The new changelog wording adds the "both tried and both blocked" clause matching that observational framing, but its main clause still reads as a categorical present-tense assertion — This doesn't rise to a blocking finding: as the prior round noted, this text is a historical changelog entry, not prompt text that governs the live reviewer agent's behavior, and the substantive fix (the false inline-output claim that does govern live behavior) was already verified fixed. I'm not re-raising it as a new finding — it's the same non-blocking observation as before, now partially rather than fully addressed. No other files changed in this commit, and no new issues were introduced. VerdictReady for merge — the prior verdict (2026-08-20T23:20:10Z, reviewing commit Reviewed commit: f7e8697 |
Closes #543. The reviewer was not attempting something forbidden. It wanted the diff in a FILE so it could chunk or count a large one, and had no route to one: a command chaining `gh pr diff` with a redirect is rejected as a whole, and the agent cannot write a file itself. So it retried variants until the denial count crossed the stub-retry threshold and the run ended with no verdict -- 33 denials on ucdavis/win#78, 6 on gha#543's reproduction, 8 and $4.95 on gha#555, the last with #542's prompt paragraph forbidding exactly that construction already live at @v2. A new step writes the diff to the workspace before the agent starts, and the prompt names that absolute path -- absolute because Claude Code's Read tool requires one. Bash(gh pr diff:*) stays allowed, so a denial afterwards means something has genuinely gone wrong, which is what the threshold is trying to measure. Failure to save is not fatal: the partial file is removed, the path is left empty, and the reviewer keeps today's route. Verified against a stub gh across four shapes, including gh writing bytes then failing. Not verifiable on its own PR -- claude-review.yml pins @v2, so both review rounds ran the pre-fix workflow. #568 tracks extracting the step into a tested composite action once the tag has moved.
Closes #541.
What
On a large PR, the reviewer would try to chunk
gh pr diff's output bysaving it to a file (
> /tmp/pr.diff, or amkdired checkout subdir) andthen
wc/greping it. Both mechanisms are dead ends in this sandbox:Bash(gh pr diff:*))with
;/&&/|/a redirect is denied as a whole command, eventhough the allowed part matches exactly.
/tmp, or a subdirectory created withmkdirinside the checkout — is a hard block in this sandbox, nota permission prompt, so no allowlist entry can satisfy it.
The reviewer tried 15+ variants of this before exhausting its denial
budget, per the issue's own downloaded execution artifact (ucdavis/win#78,
run 32415477507):
permission_denials_count:33,total_cost_usd:2.29, noverdict ever produced.
Fix
This needs no new tool grant —
Bash(gh pr diff:*)is already allowlisted;the failure is in how the reviewer used it. A bare, unredirected
gh pr diff <n> --repo <owner>/<repo>call already returns the whole diffas the Bash tool's own result text, with nothing written to disk at all.
Added an explicit instruction to
run-claude-review-attempt's--append-system-prompttelling the reviewer to always callgh pr diffbare and read the result inline, and to switch to that form immediately
(rather than trying another redirect/pipe variant) if a chained attempt is
denied. Documented the mechanism as a fourth stub-review cause alongside
the existing gha#173/#218/#392 comments in the same file.
Test plan
python3 -c "import yaml; yaml.safe_load(open(...))")(no broken quoting/escaping)
run-claude-review-attemptwraps a live
anthropics/claude-code-actioncall (per this repo's ownCLAUDE.md), so this is validated the same way the rest of that file's
prompt fixes have been: by real reviews once released. Will watch the
next few dispatched reviews on large-diff PRs for the previous
redirect/pipe denial signature.
🤖 Generated with Claude Code