Skip to content

feat(claude-review): set a pending status while the review runs - #114

Open
crowecawcaw wants to merge 2 commits into
aws-deadline:mainlinefrom
crowecawcaw:claude-review-pending-status
Open

crowecawcaw wants to merge 2 commits into
aws-deadline:mainlinefrom
crowecawcaw:claude-review-pending-status

Conversation

@crowecawcaw

@crowecawcaw crowecawcaw commented Oct 8, 2026 •

Copy link
Copy Markdown

What was the problem/requirement? (What/Why)

The Claude review commit status is only set when a review finishes. While a review runs, the PR shows just the collect check, which goes green in seconds, and then nothing until results post. Authors can't tell a running review from one that never started.

#95 tried to fix this with a check run opened from the agent's job. Since #99 that job holds a read-only token, and the commit status already exists, so this PR replaces #95.

What was the solution? (How)

  • resolve (new, read-only): the existing PR-resolution step, moved out of review unchanged (including the debounce and the skip cases).
  • start (new, no model, no permissions block): sets Claude review to pending ("Review in progress") on the head SHA. Best effort: without statuses: write it warns and the run continues.
  • review: gated once at the job level on resolve, in place of the per-step if:s. Its token is unchanged and still read-only.
  • post: now also runs when review fails before preparing its context (for example, a checkout fails). In that case it sets the status to error rather than leaving pending on the PR head. Otherwise it behaves as before.

A run cancelled by a newer push leaves pending on a commit that is no longer the head, so the PR doesn't show it.

What is the impact of this change?

  • Callers that already grant statuses: write get the pending status with no changes. Callers that don't, including this repo's own caller, see no change apart from a warning annotation.
  • Adds one short job (~10–20s of runner startup) before the review job starts.

How was this change tested?

  • test/test_claude_pr_review.py passes (65).
  • actionlint and zizmor 1.23.1 report the same findings as mainline, nothing new.
  • End-to-end on fork PR test: pending status (aws-deadline/.github#114) crowecawcaw/deadline-cloud#35. Its callers point at this branch and grant statuses: write.
    1. Full review: a file with 2 planted bugs. Status went pending ("Review in progress"), then failure ("Open: 2 blocking, 0 should-fix, 1 nit"). Both bugs were found.
    2. Incremental review: fixed both bugs. Status went pending, then success, with 2 threads resolved.
    3. Review fails before preparing: pointed the caller's tooling_ref at a ref that doesn't exist, so review fails at the tooling checkout. post ran only Report unfinished review, and the status went pending, then error ("Review did not finish; push again or re-run to retry").
  • On the fork's runners, start and review both queued for about 3 minutes before starting. pending appeared roughly 2 minutes after the collect check finished.
  • Not tested end-to-end: a failure inside the post job itself (the second commit's case). That path uses the same step and failure().

Was this change documented?

Yes, in the workflow header and job comments.

Is this a breaking change?

No.


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

The "Claude review" commit status was only set when a review finished,
so while it ran the PR showed only the collect check, which goes green
in seconds. Authors couldn't tell a running review from one that never
started.

PR resolution moves into its own read-only resolve job, so a new start
job, which runs no model, can set the status to pending before the
review job starts. The review job's token stays read-only. post now
also runs when the review job fails before preparing its context, and
sets the status to error, so pending doesn't stay on the PR head.

Supersedes aws-deadline#95, which used a check run from the model's job.

Signed-off-by: Stephen Crowe <6042774+crowecawcaw@users.noreply.github.com>

jobs:
review:
# Resolve the PR, then set the pending status before the review starts.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should fix: The PR commits compiled bytecode: .github/scripts/__pycache__/claude_pr_review.cpython-314.pyc and test/__pycache__/test_claude_pr_review.cpython-314-pytest-9.0.2.pyc. These are local build artifacts. They go stale as soon as the source changes, and they bloat history. The repo has no .gitignore, so the next local test run will add them again.

Fix: remove both files from the PR and add a .gitignore containing __pycache__/ and *.pyc.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 429bb5d: removed both files and added a .gitignore for __pycache__/ and *.pyc.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed: Both .pyc files are gone from the PR head, and the new .gitignore excludes pycache/ and *.pyc.

@@ -527,12 +576,13 @@ jobs:
submodules: false

- name: Post review

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should fix: If the Post review step fails, the head commit stays at the pending status that start set, and it stays there indefinitely. post() calls gh with check=True in several places (fetch_threads, fetch_summary, the summary PATCH/POST), so a transient GitHub API error or a failed artifact download ends the job before set_status runs. The PR then shows "Review in progress" with nothing running. Before this PR the status was simply absent in that case, which the docs define as "not picked up".

Fix: add a final step that runs on failure() and sets state=error for context=Claude review, the same way Report unfinished review does.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 429bb5d: Report unfinished review now runs last, on !cancelled() && (prepared != 'true' || failure()), so a failed download, checkout, or Post review step replaces pending with error.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed: 'Report unfinished review' now runs as the last step on !cancelled() && (prepared != 'true' || failure()), so if the download, checkout, or Post review step fails, the pending status becomes error.

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Claude review · advisory

Reviewed 429bb5d (changes since 2aba70c): 0 new, 2 resolved.

Open: 0 blocking · 0 should-fix · 0 nit. ✅ Nothing blocking.

Fix or reply to each thread; the next revision's review re-checks open threads and resolves those it agrees are handled. Resolving a thread yourself also closes it. Later revisions review only what changed.

If a step in post failed before Post review set the status (an artifact
download, or a GitHub API error in the script), the head stayed at the
pending status that start set. Report unfinished review now runs last
and also covers that case.

Also drop the __pycache__ files committed by mistake, and ignore them.

Signed-off-by: Stephen Crowe <6042774+crowecawcaw@users.noreply.github.com>
@crowecawcaw
crowecawcaw marked this pull request as ready for review October 8, 2026 16:51
@crowecawcaw
crowecawcaw requested a review from a team as a code owner October 8, 2026 16:51

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant