Skip to content

docs(agents): replace the Review record with a Reviewer Brief - #778

Open
vishal-bala wants to merge 1 commit into
mainfrom
feat/reviewer-brief
Open

vishal-bala wants to merge 1 commit into
mainfrom
feat/reviewer-brief

Conversation

@vishal-bala

@vishal-bala vishal-bala commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

The Review record introduced in #767 turned out to be an audit log of the author's review: what was fixed, what was rejected, how many findings were raised and why each lens ran. On the first PRs that carried one, it ran to 190–560 words, and the parts a reviewer acts on were buried in the middle. A reviewer sees the change as it is now and never saw the problems the review fixed, so most of the record cost reading time without changing what they did. This PR replaces it with a Reviewer Brief that points the reviewer at the current code.

Changes

The Reviewer Brief

references/review-record.md becomes references/reviewer-brief.md. The brief has a fixed order:

  • Start here: the one file or function the rest of the change hangs off.
  • Decisions for you: questions only a maintainer can settle, or a rejected suggestion a reviewer might side with.
  • Where it could fail: up to three spots in the final diff, each with a file and line.
  • Not verified: what wasn't run or exercised.
  • Beyond CI: evidence CI can't show.
  • Footer: one line naming the reviewed commit, the agent and model, and the lenses that ran.

It deliberately leaves out fixed findings, rejected findings, counts and lens reasons. The reference also adds short guidance on writing for a reviewer who never saw the review: lead with the consequence, use names the reviewer can see in the diff, and cut anything the diff or CI already says. Its example is adapted from a real PR's brief.

Six lenses by default

SKILL.md now runs all six lenses by default. The trigger table is kept as guidance for authors who want to save tokens. The diff step also covers uncommitted and untracked work. Fixes for review comments that change behaviour now need a fresh review rather than a one-line note.

Helper scripts

Two steps that agents kept doing by hand are now scripts that use only the Python standard library and gh:

  • scripts/update_brief.py replaces only the brief section of a PR description. It keeps Bugbot's summary, replaces an older ## Review record heading, and copes with Windows line endings, which some PR descriptions use. tests/unit/test_pre_pr_review_scripts.py pins those cases.
  • scripts/open_feedback.py lists unresolved review threads through GraphQL, which is the only place resolution state is exposed. It also lists review summaries and top-level comments.

Smaller changes

  • evals/evals.json holds three prompts for comparing runs with and without the skill.
  • AGENTS.md, CONTRIBUTING.md, the PR template and .cursor/BUGBOT.md now describe the Reviewer Brief.

Reviewer Brief

Start here: .agents/skills/pre-pr-review/references/reviewer-brief.md. It defines what agents write for you, and SKILL.md, AGENTS.md, CONTRIBUTING.md, the PR template and BUGBOT.md all point at it.

Decisions for you

Where it could fail

  • .agents/skills/pre-pr-review/scripts/update_brief.py:22–25: the section ends at the next ## heading or at Bugbot's summary. A description with a ## line inside a code block below the brief would lose everything from the brief down to that line.
  • .agents/skills/pre-pr-review/scripts/open_feedback.py:106: threads GitHub marks outdated now count as open, so agents re-check them. Review summaries and top-level comments are read from the first 100 only.
  • .agents/skills/pre-pr-review/references/reviewer-brief.md:53–54: these two rules decide when a brief still applies after a push. Agents will follow them literally.

Not verified: open_feedback.py on a PR with unresolved threads (none were available); the evals were not run; agents other than Claude Code.

Beyond CI: a separate agent ran the revised skill on this change, and its five kept findings are fixed in this commit. The Windows line-ending test fails against the script without the fix. This brief was written into the description by update_brief.py.

Reviewed the working tree on 8d620ca, fixes in e2a3777 · Claude Code (claude-opus-5-5) · lenses: correctness, security, design, tests, docs, Redis behaviour


Note

Low Risk
Changes are limited to agent skill docs, contributor guidance, and stdlib gh helpers with unit tests; no RedisVL library runtime behavior changes.

Overview
Replaces the pre-PR Review record (audit log of fixes, rejections, and lens rationale) with a Reviewer Brief aimed at human approvers: where to start, decisions, up to three failure spots, what was not verified, and optional beyond-CI evidence—without replaying problems already fixed in the branch.

The skill workflow in SKILL.md now defaults to all six review lenses, broadens diff scoping to uncommitted/untracked work, and documents update_brief.py (surgical PR description updates, including legacy ## Review record headings and Bugbot blocks) and open_feedback.py (GraphQL-based open review thread listing). references/review-record.md is removed in favor of references/reviewer-brief.md; lens references rename “Not checked” to Not verified and adjust challenger guidance so rejections surface in the brief only when a reviewer might disagree.

AGENTS.md, CONTRIBUTING.md, the PR template, and BUGBOT.md are aligned on the new section name and how to read it. Adds evals/evals.json for skill regression prompts and unit tests for replace_brief() in tests/unit/test_pre_pr_review_scripts.py.

Reviewed by Cursor Bugbot for commit e2a3777. Bugbot is set up for automated code reviews on this repo. Configure here.

The Review record read as an audit log of the author's review: what was
fixed, what was rejected, finding counts and lens reasons. A reviewer
sees the change as it is now, so most of that cost reading time without
changing what they do.

The Reviewer Brief points the reviewer at the current code instead:
where to start reading, what they need to decide, up to three places
the change could fail, what was not verified, and evidence CI cannot
show. A one-line footer names the reviewed commit, the agent and model,
and the lenses that ran. The reference adds short guidance on writing
for a reviewer who never saw the review.

All six lenses now run by default, with the triggers kept as a way to
save tokens. Two helper scripts handle steps agents kept repeating:
update_brief.py replaces only the brief section of a PR description,
keeping Bugbot's summary and coping with Windows line endings, and
open_feedback.py lists unresolved review threads through GraphQL. An
evals file holds three prompts for comparing runs with and without the
skill.
@vishal-bala vishal-bala added the auto:documentation Changes only affect the documentation label Oct 20, 2026
@vishal-bala
vishal-bala marked this pull request as ready for review October 20, 2026 15:29

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto:documentation Changes only affect the documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant