claude-review: gha#198's high-denial stub signature recurred (12 denials, docs-only diff, nothing posted)

[d-morrison]

gha#198 is closed, but its signature recurred today on
Morrison-Lab/wai#83, a documentation-only PR touching one .qmd and
inst/WORDLIST.

Run: https://github.com/Morrison-Lab/wai/actions/runs/32424955329/job/96604892908

"is_error": false,
"num_turns": 16,
"total_cost_usd": 1.77551245,
"permission_denials_count": 12

Step 21, "Resolve final review outcome", failed. Steps 22 and 23 ("Post
quota-exhausted comment", "Post review comment") were skipped, so nothing was
posted at all --- zero reviews and zero comments on the PR --- and
require-review went red. From the PR page there is no way to tell this apart
from a reviewer that simply has not run yet.

The guard behaved exactly as designed:

Claude review states no verdict (no '### Verdict' heading or 'Verdict:' line
anywhere in its output) -- looks like an incomplete/stub review, not a finished
one (gha#173, Lacaedemon/sparta#590).
::warning::permission_denials_count=12 exceeds the stub-retry threshold (5) --
this looks like gha#198's pattern, not gha#185's; not marking as retryable.

So this is not a regression in the detection. It is the underlying condition
recurring, on a repo that already fixed its own instance of the related
Morrison-Lab/wai#49 (closed 2026-08-08 by migrating that repo's caller onto
this reusable workflow).

What is worth checking

The run's claude_args allowlist contains no Task:

mcp__github_inline_comment__create_inline_comment,Bash(python3:*),Bash(maxima:*),
WebFetch,WebSearch,Bash(curl:*),Bash(gh pr view:*),Bash(gh pr diff:*),
Bash(gh pr list:*),Bash(gh issue view:*),Bash(gh issue list:*),Bash(gh search:*)

while the appended system prompt in the same run tells the agent to use "the
code-review plugin's own same-run parallel sub-agent fan-out" (with
run_in_background: false, per gha#392). If the plugin's command does fan out
and Task is not grantable, every one of those calls is a denial, which would
account for a double-digit count on a two-file docs diff where there is very
little else to be denied for.

I cannot confirm that, because the job log records only the count and not
the tool names --- which is gha#540, where I have posted the same evidence.
Fixing #540 first would make this issue answerable in one read rather than by
inference.

Cost

$1.78 and three minutes of runner time, spent on a run that posted nothing.
The intended review is unobtainable while this holds: self-review-fallback
correctly says not to re-dispatch the same reviewer on this signature, so the
PR has no route to an external verdict.

[d-morrison]

Second data point in flight, recorded here rather than left in a chat thread
so whoever picks this up can check it directly.

A follow-up push to the same PR (Morrison-Lab/wai#83, head 1c9c852)
triggered a fresh review run:

https://github.com/Morrison-Lab/wai/actions/runs/32426882608/job/96610539814

As of 2026-08-20 16:05 PDT it is still pending, so its denial count and
verdict are not yet known. Two outcomes are worth distinguishing when it
lands:

• It stubs again with a high denial count. That makes the condition
  reproducible on an unchanged repo config across two consecutive heads, which
  rules out a one-off and points at the allowlist rather than at run-to-run
  variance.
• It completes with a verdict. That makes the first failure intermittent,
  which is a different and harder problem, and would argue against the Task
  hypothesis in the issue body above rather than for it.

Either way the diagnostic is the same one gha#540 is about: without the
denied tool names in the job log, a second high-denial stub still would not say
which tool, so this issue stays answerable only by inference until #540
lands.

Note also that the PR currently has zero reviews and zero review comments
across both runs, so require-review has been red on this PR since it was
marked ready. The self-review posted there is a fallback per
self-review-fallback and is explicitly not an external verdict.

[d-morrison]

Claude Code (remote session, gia sweep) is working on this — paws off until I'm done.

Scoping note up front, so the split is visible rather than discovered at merge:

This issue carries two separable halves.

1. "Nothing was posted at all." Fully actionable now, no diagnosis needed. When Resolve final review outcome exits 1, every downstream posting step is gated on steps.resolve-final.outcome == 'success', so Post quota-exhausted comment, Post review comment, Post cost comment, and Collapse previous Claude review comments all skip together. The thread stays silent, the spend goes unrecorded, and the previous round's review comment stays expanded — which reads as a current verdict. This is the same silent-thread class gha#360 fixed for the push path and gha#379 for the Gemini path, and this repo already has the pattern for it (report-push-failure, report-gemini-failure).

2. Which tool is being denied. Not answerable from here. The Task hypothesis in the issue body is inference, and it is in some tension with gha#392's own measurement on Morrison-Lab/ai-config#986, where eight Agent calls did execute — so Agent/Task was evidently not denied in that run. Settling it needs a fresh reproduction on Morrison-Lab/wai, which is outside this session's repo scope.

I am shipping half 1. It is the half that makes half 2 answerable from the PR page on the next occurrence, rather than only from a downloaded execution artifact — gha#544 landed the denied-tool names this morning (Denied tools: Taskx6 Bashx3 …), but they currently reach the job log and an ::warning:: annotation only, never the PR. Carrying them into a posted failure comment is what dissolves the "#540 first" dependency the issue body notes.

So the PR will say Refs #543, not Closes #543 — half 2 keeps this issue open.

---

Generated by Claude Code

[d-morrison]

Unclaiming — the reporting half shipped in #548, merged as d23fd48. This issue stays open for the half it does not address.

What shipped

A no-verdict review now posts a > [!CAUTION] comment naming the failure kind, the denied tools, and the cost, and linking the run. So the specific symptom this issue opens with — "nothing was posted at all ... from the PR page there is no way to tell this apart from a reviewer that simply has not run yet" — is closed.

It also carries gha#544's denied tool names to the thread, which they never reached before (job log and one annotation only). That is what dissolves this issue's own stated dependency:

Fixing #540 first would make this issue answerable in one read rather than by inference.

#540 landed this morning; the names now reach the PR page. So the next recurrence of this signature will name its denied tools on the thread, without downloading an execution artifact.

What is still open, and why

The root cause — which tool the reviewer is being denied on Morrison-Lab/wai#83 — is unresolved. Two reasons I did not attempt it:

1. The Task hypothesis in the body is in tension with the evidence. gha#392's own measurement on Morrison-Lab/ai-config#986 recorded eight Agent calls that executed — so Agent/Task was evidently not denied in that run. That does not refute the hypothesis for wai#83, but it does mean the allowlist omission is not sufficient on its own to explain a denial storm.
2. Settling it needs a fresh reproduction on Morrison-Lab/wai, which is outside this session's repo scope.

The cheapest route now is to let the next occurrence report itself. When a wai review next fails this way it will post the denied tool names directly, and this issue becomes answerable from that comment.

Also filed #549 for an unrelated attribution imprecision the #548 review surfaced.

---

Generated by Claude Code

[d-morrison]

Reproduction for the deferred half: the denied tool is Bash, and the shape is compound commands with redirection

#548 shipped the reporting half and left this open, saying the root-cause half "needs a reproduction on Morrison-Lab/wai". A reproduction arrived here instead, on this repo, two hours later — and because the reporting half is merged but not yet released, the run demonstrated the original silent-thread symptom at the same time.

Run 32506208972, reviewing #550 at ee539f4. The guard's own log:

permission_denials_count=6 (max_denials=5)
Denied tools: Bashx6 (sample: Bash: cd /home/runner/work/gha/gha && git show HEAD:CLAUDE.md > /tmp/before_claude.md 2>/dev/null; git diff HEAD -- CLAUDE....)
::warning::permission_denials_count=6 exceeds the stub-retry threshold (5) — this looks like gha#198's pattern, not gha#185's; not marking as retryable.
::error::Claude review states no verdict

All six denials are Bash, and the sample shows why. The allowlist in run-claude-review-attempt grants specific Bash(git ...)-shaped prefixes. What the reviewer actually reached for is a compound command: a cd, an &&, an output redirect to /tmp, a 2>/dev/null, and a ; chaining a second command. No prefix-matched allowlist entry covers that, so it is denied whole — the individual git show and git diff inside it are irrelevant, because the matcher never sees them as separate commands.

This is the second instance of one pattern, not a new one. gha#541's subject was a reviewer reaching for gh pr diff ... > /tmp/pr.diff — the same shape, a permitted command made unmatchable by a redirect. That makes redirection-to-/tmp the recurring form rather than a one-off, which is what a fix should target.

What this run cost, and why it matters for the threshold

The run spent $3.42 over 21 turns and produced no verdict. Then the stub-retry gate refused a second attempt, because 6 denials is one over the threshold of 5 — so the run was classified as gha#198's "the reviewer gave up" pattern when it is in fact a permissions gap with a specific, fixable cause. That classification is defensible as a retry gate and is misleading as a diagnosis, which is the distinction self-review-fallback.md already draws: a high denial count says nothing about whether a manual re-run recovers.

Suggested direction (not implemented here)

Two candidates, and I have not measured either:

1. Grant the reviewer a writable scratch path rather than trying to enumerate command shapes — the denials are all about getting output somewhere, and every workaround the reviewer improvises will be a new unmatchable compound.
2. Widen the allowlist to cover the compound forms actually observed, which is narrower but reactive: it fixes the two shapes on record and waits for the third.

The first looks better on the evidence, since the failure recurs through a different command each time and the constant is the redirect. Worth measuring before committing to either.

Leaving this issue open; #550 is being driven separately.

---

Generated by Claude Code

[d-morrison]

Third shape, one hour later — and this one is a plain allowlist gap, not a command-form problem

ai-config#1833's review just failed the same way: 10 Bash denials, retry refused, $2.73 spent, no verdict, silent thread. Sample from the guard:

Denied tools: Bashx10 (sample: Bash:  gh api repos/Morrison-Lab/ai-config/issues/1833/comments --paginate --jq '.[] | select(.user.login | test("claude"; ...)

gh api is not in allowedTools at all. The current list grants gh pr view, gh pr diff, gh pr list, gh issue view, gh issue list, and gh search — every gh verb the reviewer might want except the general one. So unlike the two shapes already on this issue, nothing here is about compound commands or redirection: a plain, single, read-only invocation was denied because the command is simply not granted.

That makes three distinct shapes, and they want different fixes:

#	Case	Shape	Fix class
1	gha#541	gh pr diff ... > /tmp/pr.diff	allowed command, unmatchable form
2	gha#550's review	cd ... && git show ... > ... ; git diff ...	allowed commands, unmatchable form
3	ai-config#1833's review	gh api repos/.../comments --paginate --jq ...	command not granted

The sharp part: the reviewer was running our own documented recipe

That gh api ... /issues/<N>/comments --paginate --jq '.[] | select(.user.login | test("claude"...))' is, near-verbatim, the command gha's own CLAUDE.md prescribes under "Re-check for latest review findings before reporting PR status" — the one that exists precisely because filtering on an author login silently returns the previous round's comment. We instruct the reviewer to run it and then deny it. Ten times in one run.

Why the obvious fix needs a decision first

Adding a blanket Bash(gh api:*) is not obviously safe: gh api takes -X POST/--method, so it can reach the mutation endpoints the deny list explicitly closes (Bash(gh pr comment:*)). And prefix-matching a mutation flag is exactly the unreliable form the permissions docs call out when they explain why Bash(command:rm *) is ignored — the flag can sit anywhere in argv, and --method is an alias for -X.

But that argument may already be moot, and this is worth checking before designing around it: Bash(curl:*) is already in allowedTools. If the reviewer's step has a usable token in its environment, curl is strictly more capable than gh api, and the security case for withholding gh api collapses — it would be withholding the ergonomic path while leaving the raw one open. I have not verified what the token exposure actually is in that step, so I am flagging it as the question to settle rather than as a finding.

If it does hold, the cheap fix is granting Bash(gh api:*) and relying on the existing deny entries plus the review-only prompt, rather than trying to enumerate safe subcommands.

Cost so far today

Three no-verdict runs across two repos in about ninety minutes — $3.42, $2.73, and one more still in flight — every one of them lost to this. None of the three left a comment on its PR, because #548's failure notice is merged but not yet at @v2.

---

Generated by Claude Code

[d-morrison]

Answering the question I posed rather than leaving it open

I flagged the security objection to granting Bash(gh api:*) — that -X POST reaches the mutation endpoints the deny list closes — and said it might be moot because Bash(curl:*) is already allowed, but that settling it needed checking the token exposure. Checking it turned out to need no new measurement: run-claude-review-attempt/action.yml already contains the reasoning, applied to a different tool.

From its own comment block, on why python3 -c was un-denied:

This job gets no git-write capability from these, but on a same-repo (non-fork) PR it does hold CLAUDE_CODE_OAUTH_TOKEN / ANTHROPIC_API_KEY as secrets. Bash(python3:*) already permits running arbitrary .py files, so ALSO denying python3 -c bought little real protection while blocking the inline SymPy one-liners […] so -c is allowed.

That is the same argument, and it was already accepted here. The allowlist grants two arbitrary-execution paths — Bash(python3:*) and Bash(curl:*) — and gh is demonstrably authenticated in that environment, since Bash(gh pr view:*) and Bash(gh pr diff:*) are granted and reviewers use them successfully every round.

So the objection does not survive: withholding gh api withholds the ergonomic path while leaving the general ones open. A deny that a curl one-liner walks around is a usability cost, not a security control — which is exactly what the python3 -c paragraph concluded about a narrower version of the same question.

What I am and am not claiming

Established (from the file and from observed reviewer behaviour, not from a run I performed):

• The step holds Anthropic secrets on same-repo PRs — stated in the action's own comment.
• Bash(python3:*) permits arbitrary .py execution — stated there too, as the premise of an accepted decision.
• Bash(curl:*) is in allowedTools.
• gh is authenticated in that step, since the granted read subcommands work.

Not established, and not needed for the conclusion: which environment variable or config file carries the GitHub token, and whether python3 reaches it as easily as gh does. I have not executed anything in that environment. The argument holds without it, because it turns on gh api being no broader than what is already granted rather than on any particular exfiltration path being easy.

Suggested fix, now unblocked

Grant Bash(gh api:*) and rely on the existing deny entries plus the review-only prompt, rather than trying to enumerate safe subcommands. Enumeration was the fragile option anyway: -X can sit anywhere in argv and --method is an alias, which is the same unreliable prefix-matching the permissions docs invoke when explaining why Bash(command:rm *) is ignored outright.

Worth doing soon rather than at leisure. Today's tally on this bug is four no-verdict runs across two repos, roughly $9, and one of them was the reviewer being denied ten times while running the command CLAUDE.md itself prescribes for re-checking review findings.

Not claiming this issue.

---

Generated by Claude Code

[d-morrison]

Third occurrence, and the first one where #542's prompt fix was demonstrably live

v2 slid to df8f8e9 at ~21:03Z today, so fe515d7 (#542 -- the prompt paragraph telling the reviewer to call gh pr diff bare) reached this repo's own dogfood caller. The next review after the slide hit this anyway.

Run 32526051617, on #555, head f139910, referenced_workflows resolving to the post-slide tag:

Denied Bash calls	8, over the stub-retry threshold of 5
Classification	high-denial (gha#198), so deliberately not retried
Verdict	none
Cost	$4.9507 across 2 attempts
Sample denial	gh pr diff 555 --repo Morrison-Lab/gha > /tmp/pr555.diff 2>&1; wc -l /tmp/pr555.diff

That sample is the exact construction run-claude-review-attempt's own prompt forbids in as many words:

To read the PR diff, call gh pr diff <number> --repo <owner>/<repo> BARE -- no > redirect, no | pipe, no ;/&& chain ... If a redirected or piped attempt is denied, do not try a different redirect target.

So the count is now at least three, across two repos:

1. ucdavis/win#78, run 32415477507 -- 15+ redirect/pipe/mkdir variants, no verdict (recorded in the action's own comment block)
2. the reproduction already on this issue -- 6 denials, $3.42, no verdict
3. this one -- 8 denials, $4.95, no verdict, with the prompt fix live

Why (3) is the one that settles the design question

The first two are consistent with "the reviewer had not been told". The third is not. Bash(gh pr diff:*) is on the allowlist, the bare form works, the prompt says so explicitly, and the reviewer still chained a redirect eight times and starved itself out.

That is this repo's own principle, verbatim from CLAUDE.md:

A prompt tells the model what to do, and the model may not. When a constraint matters, look for a mechanical form of it before concluding that a better-worded prompt is the ceiling.

It is also the same shape as gha#532 -> #550: the prompt asked the reviewer not to spawn background agents, the reviewer did it anyway, and the fix that worked was a permission rule plus a gate adjustment, not better wording. shared/principles/deterministic-tools.md's third-occurrence bar is met here.

The mechanical option

A Bash(command:...) parameter rule cannot help -- Claude Code documents command as a primary content field and refuses such rules with a startup warning, precisely because a compound command would bypass them. So the constraint has to remove the opportunity:

Drop Bash(gh pr diff:*) from --allowedTools and leave only the pre-written diff file the action already provides. The prompt already tells the reviewer that file exists and not to recreate it; removing the alternative turns that instruction from a request into the only available path. A denied gh pr diff then costs one denial instead of eight, because there is no variant to retry.

Worth checking before implementing: whether any review path genuinely needs gh pr diff beyond the head diff the action writes (a diff against a different base, a specific file). If one does, the file the action writes is the thing to widen, not the allowlist.

Immediate consequence for #555

#555 cannot obtain a Claude verdict while this stands -- every attempt reproduces it, and the high-denial classification correctly refuses to retry, so re-running only spends money. Its require-review is red for this reason rather than for anything in the diff.

---

Generated by Claude Code