Stop wasting review turns retrying tools the sandbox denies#28
Merged
Conversation
Claude Code exposes its full standard toolset to the model regardless of --allowedTools scoping; an allow-list only gates execution at call time, it doesn't hide the tool from the model's context (that requires a bare deny rule, which we don't use here). So the model has no way to know in advance which of its visible tools will actually work in this sandboxed review — it only finds out by attempting the call and eating a permission denial, one turn at a time. That was cheap and mostly invisible on Sonnet 4.6. Now that the org default is Sonnet 5, reviews retry disallowed tools noticeably more (build/lint/test attempts, LS, a WebFetch for the org CLAUDE.md link), and on larger diffs the wasted turns tip the review past max_turns entirely — ht-android hit three straight max-turns failures on one PR today, while a same-day, same-config sibling PR with a slightly smaller diff passed with turns to spare. Fix, without touching max_turns: - Tell the prompt directly which tools are actually available, so the model stops discovering the boundary by trial and error. - Add LS to the allow-list — already granted one step later, in the sticky-comment update. - Fold the Mobile and Infrastructure rubric sections — previously reachable only via a WebFetch to the org CLAUDE.md — directly into the prompt, since the rest of that file was already duplicated here. Removes the WebFetch attempt instead of just tolerating its denial.
jaysonv0341
approved these changes
Jul 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Claude Code exposes its full standard toolset to the model regardless of
--allowedToolsscoping; an allow-list only gates execution at call time, it doesn't hide the tool from the model's context (that requires a bare deny rule, which we don't use here). So the model has no way to know in advance which of its visible tools will actually work in this sandboxed review — it only finds out by attempting the call and eating a permission denial, one turn at a time.That was cheap and mostly invisible on Sonnet 4.6. Now that the org default is Sonnet 5, reviews retry disallowed tools noticeably more (build/lint/test attempts,
LS, aWebFetchfor the orgCLAUDE.mdlink), and on larger diffs the wasted turns tip the review pastmax_turnsentirely —ht-androidhit three straight max-turns failures on one PR today, while a same-day, same-config sibling PR with a slightly smaller diff passed with turns to spare.Fix, without touching
max_turns:LSto the allow-list — already granted one step later, in the sticky-comment update.WebFetchto the orgCLAUDE.md— directly into the prompt, since the rest of that file was already duplicated here. Removes theWebFetchattempt instead of just tolerating its denial.Ticket
N/A — ad hoc infra fix, not tied to a ticket.
Type of Change
Testing Done
Only changes the shared review workflow's prompt and tool allow-list — no functional or schema change. Not yet run against a live PR; will confirm by watching turn counts and permission-denial counts on the next few PR reviews across repos.
Checklist