feat: implement issue #701 — Canary blocker: add-to-project next->ring0 (cum_fail=1, PRE_EXISTING)#702
Conversation
…g0 (cum_fail=1, PRE_EXISTING)
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Warning Review limit reached
Next review available in: 49 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe add-to-project canary gate now allowlists restricted-secrets ChangesAdd-to-project canary gate
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Dev-Lead — waiting on PR blockers (intent: review-changes)PR: #702 |
|
Note @don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds a new benign failure class reusable-setup-restricted-secrets to the add-to-project agent's gate configuration in standards/canary-rings.json to handle environmental startup failures due to restricted secrets, along with corresponding unit tests in tests/canary_rollout.bats. Feedback suggests improving the test that checks for the absence of the version_independent property by using .version_independent != true instead of has("version_independent") | not to make it more robust.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
|
CI checks on this PR are still running. Once they complete, re-mention Posted by the donpetry-bot PR-review cascade. |
|
@donpetry-bot I'm on it — starting a fresh review now. Results will appear in a few minutes. |
Dev-Lead — fix-reviews (applied)Changes committed and pushed. |
|
|
CI checks on this PR are still running. Once they complete, re-mention Posted by the donpetry-bot PR-review cascade. |
|
@donpetry-bot I'm on it — starting a fresh review now. Results will appear in a few minutes. |
Dev-Lead — review-changes (no-changes)No changes were needed for this PR. |
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: MEDIUM
Reviewed commit: 2d001e89e1c2cfb283ea9025d7694ad8338ccb39
Review mode: triage-approved (single reviewer)
Summary
Adds a single benign-failure class (reusable-setup-restricted-secrets) to the add-to-project canary gate in standards/canary-rings.json, allowlisting the environmental 'Set up job' startup failure of the 'Auto-add to Initiatives project' caller workflow (secrets withheld in restricted-secret contexts, e.g. Dependabot-authored events). Three new BATS tests validate the class metadata, functional workflow/step matching (including non-matches for other steps and unrelated workflows), and that the class is NOT version_independent. Verified against scripts/canary-rollout.sh: _benign_patterns only applies non-version_independent classes at differs=0 (byte-identical), so this class cannot mask a differs=1 candidate regression. Triage assessment confirmed correct.
Linked issue analysis
Closes #701 (canary blocker: add-to-project next->ring0, cum_fail=1, triage PRE_EXISTING). The blocker reported a 'Set up job' failure of the add-to-project reusable in petry-projects/.github-private — environmental (secrets withheld), not a candidate regression. This PR is the fix-forward the blocker asks for: the failure class is allowlisted so it no longer counts toward cum_fail, while remaining inert at differs=1. Substantively addresses the issue.
Findings
No blocking findings.
- Prior reviewer suggestion (gemini-code-assist): replace 'has("version_independent") | not' with '.version_independent != true' — addressed in commit 2d001e8; the sole review thread is resolved.
- Config-only change plus tests; no scripts or workflows modified. Matching semantics verified against the consumer (_benign_patterns in scripts/canary-rollout.sh).
- Secret scan: run_secret_scanning MCP tool not available in this environment; gitleaks CI check passed and the diff contains no secret-like content.
- Risk rated MEDIUM (not LOW) because the change alters rollout-gate failure accounting, though the non-version_independent design bounds the blast radius to byte-identical candidates.
CI status
All checks green: CI (Lint, ShellCheck, Agent Security Scan, gitleaks), Canary-rollout Tests (lint+bats), CodeQL, SonarCloud quality gate, AgentShield, dependency audits, CodeRabbit — all SUCCESS; remaining checks SKIPPED (not applicable).
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: MEDIUM
Reviewed commit: 2d001e89e1c2cfb283ea9025d7694ad8338ccb39
Review mode: triage-approved (single reviewer)
Summary
Adds a tightly-scoped benign_failure_class ('reusable-setup-restricted-secrets') to the add-to-project next->ring0 gate in standards/canary-rings.json, so a pre-existing/environmental 'Set up job' startup failure (secrets withheld in restricted-secret contexts, e.g. Dependabot-authored events) no longer counts toward cum_fail. Backed by three new bats tests verifying the class's presence/shape, functional workflow+step matching (including non-matches for in-reusable steps and unrelated workflows), and that it is NOT version_independent so it can never mask a differs=1 regression.
Linked issue analysis
Closes #701 (canary blocker: add-to-project next->ring0, cum_fail=1, PRE_EXISTING). The blocker was a 'Set up job' startup failure of the reusable in petry-projects/.github-private, triaged PRE_EXISTING (candidate byte-identical). This PR is the fix-forward: the exact failure signature (workflow 'Auto-add to Initiatives project', step 'Set up job') is now classified benign at differs=0 only. Substantively addresses the issue.
Findings
- Gate-config change is conservatively scoped: exact workflow+step match, excludes only when the reusable is byte-identical (not version_independent), so it cannot suppress a real candidate regression.
- Tests functionally drive benign_match with the class's own regexes and assert non-leakage to sibling steps and unrelated workflows.
- Prior cascade review approved this exact SHA (risk MEDIUM); the single gemini-code-assist thread (low-priority, jq brittleness) is resolved and outdated — the test now asserts '.version_independent != true', which is robust to the key being absent.
- Secret-scanning MCP tool unavailable in this run; gitleaks CI check passed and the diff contains no credential-like content.
- No new dependencies, no workflow/script changes, no security-sensitive surface touched.
CI status
All required checks green: Lint, Lint and bats, ShellCheck, CodeQL, Agent Security Scan, AgentShield, Secret scan (gitleaks), npm audit, SonarCloud (quality gate passed), CodeRabbit. Remaining checks skipped (ecosystem not present).
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
2 participants
Closes #701
Implemented by dev-lead agent. Please review.
Summary by CodeRabbit
Bug Fixes
Tests