Skip to content

chore(claude): add a test-reviewer subagent - #1517

Closed
gabrielseco wants to merge 6 commits into
mainfrom
chore/test-reviewer-agent
Closed

gabrielseco wants to merge 6 commits into
mainfrom
chore/test-reviewer-agent

Conversation

@gabrielseco

@gabrielseco gabrielseco commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds a Claude Code subagent that reviews the tests on a branch. It checks them against the repo's testing rules and proves each test fails when the behaviour it covers is removed.

Why

  • A test that still passes with its fix reverted proves nothing. On fix(onboarding): refetch employment after basic info #1515 the first version of the test passed only because the mock already returned the new job title.
  • Mocks encode assumptions about the API. Hand-built schemas, handlers that ignore the payload, and calling flow methods directly instead of clicking through the step all hid real bugs this week.
  • A dry run on fix(onboarding): refetch employment after basic info #1515 found two gaps the author missed: a refetch-ordering claim that no test covered, and a PR description that said a failed update throws when it doesn't. Both are fixed there.

What changed

Toggle details
  • .claude/agents/test-reviewer.md: the subagent. It reads CLAUDE.md fresh each run instead of keeping its own copy of the rules. It then:
    • checks each new or changed test against those rules, plus a short list of problems from recent PRs (mocks that hide the bug, inline schemas, skipped wiring, rendering before load, negative assertions with no wait, ordering claims, duplicated coverage);
    • removes the behaviour each test covers, runs the test, and restores the file from a backup. It never uses git checkout or git stash, so it can't lose uncommitted work;
    • reports findings with a table of mutation results. It doesn't edit, commit or push.
  • CLAUDE.md: a new testing convention, "A test for a fix or feature must fail without it", which links to the agent.
  • No library code changes.

Screenshots

N/A

Related Resources

Testing

🤖 Generated with Claude Code


Note

Low Risk
Documentation and Claude agent config only; no product code, APIs, or CI behaviour changes.

Overview
Adds a Claude Code test-reviewer subagent (.claude/agents/test-reviewer.md) for branch test review: it reloads testing rules from CLAUDE.md, audits new/changed tests (including repo-specific pitfalls like MSW mocks that hide bugs, inline schemas, and ordering claims without held responses), mutates production code or waits to confirm each test fails when the covered behaviour is removed, then reports High/Medium/Low findings without editing tests permanently.

CLAUDE.md gains a convention that fixes/features must have tests that fail without the change, with a link to the subagent. No application or test runtime code changes.

Reviewed by Cursor Bugbot for commit 7afcd28. Bugbot is set up for automated code reviews on this repo. Configure here.

Reviews a branch's new and changed tests against the testing rules in
CLAUDE.md and proves each one fails when the behaviour it covers is
removed. Adds the matching rule to CLAUDE.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Deploy preview for adp-cost-calculator ready!

Project:adp-cost-calculator
Status: ✅  Deploy successful!
Preview URL:https://adp-cost-calculator-fznhccyvf-remotecom.vercel.app
Latest Commit:7afcd28

Deployed with vercel-action

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Deploy preview for remote-flows ready!

Project:remote-flows
Status: ✅  Deploy successful!
Preview URL:https://remote-flows-gtbev8ge0-remotecom.vercel.app
Latest Commit:7afcd28

Deployed with vercel-action

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

📊 Coverage Report

✅ Coverage increased! 🎉

Metric Current Previous Change Status
Lines 89.58% 89.53% +0.05% 🟢
Statements 89.45% 89.40% +0.05% 🟢
Functions 88.47% 88.41% +0.06% 🟢
Branches 81.20% 81.14% +0.06% 🟢

Detailed Breakdown

Lines Coverage
  • Covered: 5389 / 6016
  • Coverage: 89.58%
  • Change: +0.05% (4 lines)
Statements Coverage
  • Covered: 5454 / 6097
  • Coverage: 89.45%
  • Change: +0.05% (4 statements)
Functions Coverage
  • Covered: 1427 / 1613
  • Coverage: 88.47%
  • Change: +0.06% (1 functions)
Branches Coverage
  • Covered: 3343 / 4117
  • Coverage: 81.20%
  • Change: +0.06% (4 branches)

✅ Coverage check passed

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread .claude/agents/test-reviewer.md Outdated
gabrielseco and others added 3 commits October 12, 2026 14:03
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7d1d4af. Configure here.

Comment thread .claude/agents/test-reviewer.md Outdated
gabrielseco and others added 2 commits October 12, 2026 22:05
…he wait mutation

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@gabrielseco

Copy link
Copy Markdown
Collaborator Author

Closing: keeping this subagent and the testing rule local instead of shipping them in the repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant