Skip to content

fix(ci): harden comment-driven review workflows - #1255

Merged
Wibias merged 1 commit into
devfrom
fix/workflow-comment-spam-hardening-clean
Aug 8, 2026
Merged

fix(ci): harden comment-driven review workflows#1255
Wibias merged 1 commit into
devfrom
fix/workflow-comment-spam-hardening-clean

Conversation

@Wibias

@Wibias Wibias commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • remove direct issue_comment reruns from the write-capable PR quality gate
  • revalidate CodeRabbit readiness from the successful CodeRabbit commit status on the trusted default branch
  • paginate status-SHA association and proceed only when exactly one open PR still has that SHA as its current head
  • restrict label-triggered gate runs to gui-screenshot-waived; legacy maintainer comments remain compatibility evidence on later PR events
  • serialize PR gate and PR hygiene writes on the same live-head SHA across webhook types
  • prevent issue-comment translation jobs from allocating runners for PR threads and bot comments
  • document the trust boundary and add executable fail-closed and successful-resolution regression coverage

Test plan

  • bun test tests/zz-pr-coderabbit-readiness-revalidation.test.ts tests/ci-workflows.test.ts — 129 pass, 0 fail
  • node --test .github/scripts/*.test.cjs — 451 pass, 0 fail
  • bun run typecheck — pass
  • bun run privacy:scan — pass
  • git diff --check — pass

Notes

  • CodeRabbit status-comment edits no longer synchronize the privileged gate
  • ordinary label changes no longer allocate the privileged runner
  • stale, missing, ambiguous, and paginated-multiple CodeRabbit status SHA associations fail closed before PR mutation
  • exactly one open current-head PR is proven to resolve into the downstream gate using a non-default PR number regression
  • screenshot-waiver reporting is covered both when it clears the sole quality failure and on the READY path
  • enforce-target and pr-hygiene share the same current-head concurrency lock
  • the branch is exactly one commit ahead of current dev and zero commits behind
  • default-branch workflow behavior becomes live only after promotion to main

Summary by CodeRabbit

  • New Features

    • Added the gui-screenshot-waived label for approved screenshot-check waivers.
    • Added reliable review-status validation for pull requests.
  • Bug Fixes

    • Reduced unnecessary responses to bot and pull-request comments.
    • Improved handling of stale, ambiguous, or mismatched review signals.
    • Ensured checks evaluate the appropriate revision independently.
  • Documentation

    • Updated guidance for screenshot waivers, review status, and pull-request checks.
  • Tests

    • Expanded coverage for review events, waiver labels, validation, and workflow behavior.

@coderabbitai

coderabbitai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The PR replaces issue-comment gate triggers with successful CodeRabbit status events and selected label events. It resolves status commits to one current open PR, uses default-branch automation, adds label-based screenshot waivers, and filters issue-comment translation.

Changes

Workflow hardening

Layer / File(s) Summary
Issue-comment filtering
.github/workflows/enforce-issue-quality.yml, tests/zz-pr-coderabbit-readiness-revalidation.test.ts, devlog/_plan/...
Issue-quality translation excludes pull requests and bot comments before runner-consuming steps.
Review signal routing
.github/workflows/enforce-pr-target.yml, .github/scripts/enforce-pr-target.test.cjs, tests/ci-workflows.test.ts, tests/helpers/enforce-pr-target-harness.ts, .github/workflows/pr-hygiene.yml
The gate accepts successful CodeRabbit status events and selected label events. Status events resolve exactly one open PR by commit SHA, require a matching current head, and use the default branch for trusted checkout. Concurrency uses the live head SHA.
Screenshot waiver and readiness state
.github/workflows/enforce-pr-target.yml, tests/ci-workflows.test.ts, docs-site/src/content/docs/contributing/pr-quality.md, devlog/_plan/...
The gui-screenshot-waived label removes missing_ui_screenshot failures and appears in draft, ready, recovery, and gate-clearing notices. Legacy maintainer comments remain compatible on later PR events.
Workflow verification and rollout contract
tests/ci-workflows.test.ts, tests/zz-pr-coderabbit-readiness-revalidation.test.ts, docs-site/src/content/docs/contributing/pr-quality.md, structure/06_docs-and-release.md, .github/workflows/_agent_fix_1255_followup.yml, .github/scripts/_agent_patch_1255_followup.py, devlog/_plan/...
Tests, documentation, and rollout tooling cover status triggers, trusted checkout, label waivers, exact PR association, fail-closed ambiguity handling, and deployment behavior.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CodeRabbit
  participant EnforcePRTarget
  participant GitHubAPI
  participant PullRequest
  CodeRabbit->>EnforcePRTarget: emit successful status for commit SHA
  EnforcePRTarget->>GitHubAPI: resolve associated open pull requests
  GitHubAPI-->>EnforcePRTarget: return PR association
  EnforcePRTarget->>PullRequest: validate current head and review data
  EnforcePRTarget->>EnforcePRTarget: update enforcement state
Loading

Possibly related PRs

Suggested reviewers: ingwannu, lidge-jun

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: hardening CI review workflows that previously relied on comment-driven triggers.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/workflow-comment-spam-hardening-clean

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Deterministic PR hygiene checks passed.

@Wibias Wibias added the gui-screenshot-waived Maintainer waiver for false-positive GUI screenshot requirements label Aug 8, 2026 — with ChatGPT Codex Connector

Wibias commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator Author

No GUI changes. This PR only changes GitHub Actions workflow orchestration, tests, and contributor documentation.

@github-actions github-actions Bot added the bug Something isn't working label Aug 8, 2026
@Wibias
Wibias marked this pull request as ready for review August 8, 2026 04:31

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/enforce-pr-target.yml (1)

963-970: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Report the waiver after it clears the last failure.

Lines 967-970 add the waiver message only to failureNotices. If missing_ui_screenshot is the only failure, Lines 525-528 remove it and this failure branch does not run. The gate then updates the PR without reporting whether the label or a legacy maintainer comment waived the screenshot.

Create one screenshotWaiverNotice value and include it in the no-quality-failure draft and ready comment paths as well as failureNotices.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/enforce-pr-target.yml around lines 963 - 970, Define a
shared screenshotWaiverNotice value near failureNotices, preserving the
label-versus-maintainer wording, and include it in the no-quality-failure draft
and ready comment paths as well as failureNotices. Ensure the waiver is reported
when missing_ui_screenshot is the only failure and clears successfully.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@devlog/_plan/260808_workflow_comment_spam_hardening/000_plan.md`:
- Line 54: Update Task 3’s plan statement to require the label as the immediate
GUI screenshot waiver trigger while explicitly preserving legacy
maintainer-comment recognition on later PR events. Ensure it reflects the
existing guiOverrideComments and hasGuiOverride({ comments }) compatibility
contract rather than stating that the label is the only waiver signal.

---

Outside diff comments:
In @.github/workflows/enforce-pr-target.yml:
- Around line 963-970: Define a shared screenshotWaiverNotice value near
failureNotices, preserving the label-versus-maintainer wording, and include it
in the no-quality-failure draft and ready comment paths as well as
failureNotices. Ensure the waiver is reported when missing_ui_screenshot is the
only failure and clears successfully.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 87210e1f-5a25-4500-b0d7-17146dc3fe12

📥 Commits

Reviewing files that changed from the base of the PR and between ace2e67 and 4e44fc7.

📒 Files selected for processing (9)
  • .github/scripts/enforce-pr-target.test.cjs
  • .github/workflows/ci.yml
  • .github/workflows/enforce-issue-quality.yml
  • .github/workflows/enforce-pr-target.yml
  • .github/workflows/pr-review-signal.yml
  • devlog/_plan/260808_workflow_comment_spam_hardening/000_plan.md
  • docs-site/src/content/docs/contributing/pr-quality.md
  • tests/ci-workflows.test.ts
  • tests/zz-pr-coderabbit-readiness-revalidation.test.ts

Comment thread devlog/_plan/260808_workflow_comment_spam_hardening/000_plan.md Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4e44fc7cdb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/pr-review-signal.yml Outdated
Comment thread .github/workflows/enforce-pr-target.yml Outdated
Comment thread .github/workflows/enforce-pr-target.yml Outdated
Comment thread .github/workflows/enforce-pr-target.yml
@Wibias
Wibias force-pushed the fix/workflow-comment-spam-hardening-clean branch from 461aea2 to c10a759 Compare August 8, 2026 05:11

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/enforce-pr-target.yml (1)

498-512: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Tie gui-screenshot-waived waivers to verified maintainer authorization. The workflow removes missing_ui_screenshot for any presence of the label, but GitHub Triage members can apply repository labels without write access. Keep the maintainer-only waiver boundary by:

  • .github/workflows/enforce-pr-target.yml#L498-L512: only clear the screenshot failure when the label event is approved by a trusted maintainer, and persist that authorization in the bot-owned gate state.
  • tests/ci-workflows.test.ts#L2657-L2676: add a case where gui-screenshot-waived is added by a non-maintainer/low-privilege label actor and missing_ui_screenshot remains.
  • docs-site/src/content/docs/contributing/pr-quality.md#L43-L48: keep the “maintainer can waive…” claim only after the workflow enforces it.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/enforce-pr-target.yml around lines 498 - 512, Update the
workflow’s screenshot-waiver handling around screenshotWaivedByLabel and
hasGuiOverride so the gui-screenshot-waived label only clears
missing_ui_screenshot when its addition is authorized by a trusted maintainer,
and persist that authorization in the bot-owned gate state. In
tests/ci-workflows.test.ts lines 2657-2676, add coverage for a non-maintainer or
low-privilege label actor confirming the failure remains. In
docs-site/src/content/docs/contributing/pr-quality.md lines 43-48, retain the
maintainer-waiver claim after the workflow enforces this authorization boundary.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/enforce-pr-target.yml:
- Line 33: The concurrency group in
.github/workflows/enforce-pr-target.yml:33-33 must use the pull request head
SHA, falling back to the event SHA and run ID, so all events for one live PR
head serialize together. Update the related assertions in
tests/ci-workflows.test.ts:879-883 to require the shared SHA-based key and
verify serialization across event types.
- Around line 153-159: The associated-PR lookup in the workflow must traverse
every paginated response before deciding uniqueness. In
.github/workflows/enforce-pr-target.yml:153-159, replace the single-page
listPullRequestsAssociatedWithCommit call with github.paginate; in
tests/helpers/enforce-pr-target-harness.ts:86-98, add page-based associated-PR
fixtures; in tests/helpers/enforce-pr-target-harness.ts:716-721, return the
requested associated-PR page; and in tests/ci-workflows.test.ts:2678-2697, add a
page-two matching candidate case that asserts zero mutations.

In `@tests/zz-pr-coderabbit-readiness-revalidation.test.ts`:
- Around line 45-67: Strengthen the readiness revalidation test around the
workflow job, checkoutStep, and gateStep assertions to validate behavior rather
than token presence. Add fixtures or exact normalized-expression checks proving
only successful CodeRabbit status events run, status events check out
github.event.repository.default_branch, and zero, multiple, or mismatched
current-head PRs fail closed before any write; ensure computed checks are
actually enforced.

---

Outside diff comments:
In @.github/workflows/enforce-pr-target.yml:
- Around line 498-512: Update the workflow’s screenshot-waiver handling around
screenshotWaivedByLabel and hasGuiOverride so the gui-screenshot-waived label
only clears missing_ui_screenshot when its addition is authorized by a trusted
maintainer, and persist that authorization in the bot-owned gate state. In
tests/ci-workflows.test.ts lines 2657-2676, add coverage for a non-maintainer or
low-privilege label actor confirming the failure remains. In
docs-site/src/content/docs/contributing/pr-quality.md lines 43-48, retain the
maintainer-waiver claim after the workflow enforces this authorization boundary.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0edad601-a80d-4b8b-8bfa-f2e6e95c62a5

📥 Commits

Reviewing files that changed from the base of the PR and between 4e44fc7 and 461aea2.

📒 Files selected for processing (8)
  • .github/scripts/enforce-pr-target.test.cjs
  • .github/workflows/enforce-pr-target.yml
  • devlog/_plan/260808_workflow_comment_spam_hardening/000_plan.md
  • docs-site/src/content/docs/contributing/pr-quality.md
  • structure/06_docs-and-release.md
  • tests/ci-workflows.test.ts
  • tests/helpers/enforce-pr-target-harness.ts
  • tests/zz-pr-coderabbit-readiness-revalidation.test.ts

Comment thread .github/workflows/enforce-pr-target.yml Outdated
Comment thread .github/workflows/enforce-pr-target.yml Outdated
Comment thread tests/zz-pr-coderabbit-readiness-revalidation.test.ts Outdated
@Wibias
Wibias force-pushed the fix/workflow-comment-spam-hardening-clean branch from c10a759 to b73f6a4 Compare August 8, 2026 05:18

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/ci-workflows.test.ts`:
- Around line 2679-2740: Add a focused positive status-resolution test alongside
the existing CodeRabbit status tests, using exactly one open PR whose head SHA
matches statusSha and whose PR number differs from the harness default. Assert
the resolved PR number is passed to the downstream pulls.get/read calls using
the harness’s raw callsTo argument shape, and verify the gate proceeds rather
than failing closed.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a4b5e76a-626d-40eb-b9bd-f80db12d101f

📥 Commits

Reviewing files that changed from the base of the PR and between 461aea2 and b73f6a4.

📒 Files selected for processing (5)
  • .github/workflows/enforce-pr-target.yml
  • .github/workflows/pr-hygiene.yml
  • tests/ci-workflows.test.ts
  • tests/helpers/enforce-pr-target-harness.ts
  • tests/zz-pr-coderabbit-readiness-revalidation.test.ts

Comment thread tests/ci-workflows.test.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/_agent_fix_1255_followup.yml:
- Around line 52-58: Make the branch-writing automation idempotent and
serialized: in .github/workflows/_agent_fix_1255_followup.yml lines 52-58, exit
successfully when git diff --cached --quiet shows no staged test changes; in
.github/scripts/_agent_patch_1255_followup.py lines 27-29, check for each test
before inserting it; and in .github/workflows/_agent_fix_1255_followup.yml lines
3-6, add concurrency keyed by github.ref with cancel-in-progress: false.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0d599cde-78dc-4dcb-92b4-cd7166b8d912

📥 Commits

Reviewing files that changed from the base of the PR and between b73f6a4 and d406865.

📒 Files selected for processing (3)
  • .github/scripts/_agent_patch_1255_followup.py
  • .github/workflows/_agent_fix_1255_followup.yml
  • tests/ci-workflows.test.ts

Comment thread .github/workflows/_agent_fix_1255_followup.yml Outdated
@Wibias
Wibias force-pushed the fix/workflow-comment-spam-hardening-clean branch from 9c17091 to 0993c53 Compare August 8, 2026 05:45
@Wibias
Wibias merged commit d55b903 into dev Aug 8, 2026
25 checks passed
@Wibias
Wibias deleted the fix/workflow-comment-spam-hardening-clean branch August 8, 2026 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working gui-screenshot-waived Maintainer waiver for false-positive GUI screenshot requirements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant