Skip to content

docs: add Factory code review recipe - #897

Merged
rachaelrenk merged 6 commits into
factory/factory-use-cases-structurefrom
factory/factory-use-case-code-review
Oct 21, 2026
Merged

rachaelrenk merged 6 commits into
factory/factory-use-cases-structurefrom
factory/factory-use-case-code-review

Conversation

@warp-agent-staging

@warp-agent-staging warp-agent-staging Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

What this feature does

A review-only automation asks the factory's foreman to delegate code review and publish the review agent's report. Its prompt excludes implementation, approval, and merging.

Summary

Replaces the code-review scaffold with a review-only recipe using provider-specific triggers, the default foreman and review roles, and an explicit review-only automation prompt.

Changes

src/content/docs/factories/use-cases/code-review.mdx

  • Configures a foreman-targeted review-only automation: the review agent returns a report and the foreman publishes it.
  • Adds concise prerequisites, provider-specific initial and update events, and definition-as-code examples.
  • Adds a six-step setup and test procedure.
  • Documents re-review, independent model or harness selection, duplicate-review prevention, and human merge authority.

Content design plan

Audience and JTBD: An engineering team that wants consistent independent review on ready pull requests without giving an agent merge authority.

Problem: Existing docs describe agents and automations but do not show a focused review-only configuration or its safety boundaries.

Goals:

  • The reader can route ready pull requests through the foreman for delegated review and publication.
  • The reader can constrain findings to actionable defects and unmet requirements.
  • The reader can test that review does not mutate the branch.

Purpose and value: The recipe adapts a source-reviewed configuration pattern and clarifies how automated review complements repository protection.

Content type: Feature documentation (combined) — the reader needs both decision guidance and setup steps.

Skill and template: draft_feature_doc / .agents/templates/feature-doc.md.

High-impact scenarios:

  • Covers: initial review from provider-specific ready or creation events, draft-request checks, and optional re-review after updates.
  • Excludes: automatic approval, branch mutation, and merging.

Source files consulted

Checked on 2026-10-21:

  • Review-agent output contract — corrected. The default review agent reports only to its orchestrator and forbids direct PR posts. The foreman delegates review and is the communicator. The recipe now targets agent: foreman, requests a report from the review agent, and explicitly instructs the foreman to publish it and stop without implementation. Each re-review event is an explicit request for a fresh review, rather than a claim that the default lifecycle re-reviews automatically.
  • GitHub opened-ready finding — not supported by implementation. At warp-server@084e0e425af93139c6e95a78e2981dce55557c31, provider_github.go emits pull_request_ready for both non-draft opened and ready_for_review deliveries. classifier_test.go asserts both cases and excludes draft opens from the derived ready event. The recipe therefore retains the single ready trigger rather than adding an unnecessary opened trigger.
  • GitLab and Azure DevOps mappings and filters. provider_gitlab.go, provider_azure_devops.go, and the filter catalogue support the initial/update event mappings, GitLab action filters, and Azure DevOps repository UUID matching. These are implementation contracts, not proof of provider delivery in a particular workspace.
  • Definition application. factory_code_browser.go, the save handler, production_trigger.go, and webhook_predicate.go cover managed source saves and synchronization from the configured production branch. The procedure requires confirmation that changes are applied before testing.
  • Bot-policy reference — corrected. The recipe now links to the live GitHub bot-event troubleshooting section, verified on 2026-10-21. It explains accepted values, the default, and loop risks. No policy explanation was duplicated into the recipe or a stale reference.

Production-configuration and live contract evidence

  • Used the unchanged server_rollout.py helper from docs@9fe3255523eda5c858f3daefd9a15a970a7ea3be against an isolated checkout of warp-server@084e0e425af93139c6e95a78e2981dce55557c31.
  • Term checks covered pull_request_ready, pull_request_synchronized, merge_request, pull_request_created, pull_request_updated, and factoryfile. The broad definition-file search also matched unrelated experimental surfaces, so it was not used as a blanket availability verdict. A targeted follow-up check of the nine source-identified routing, provider, save, and apply gates returned exit 0: all nine enabled in the checked-in production configuration, no unknown names. Private flag names and raw output are intentionally not published here.
  • Retrieved the production automation JSON Schema on 2026-10-21; its title identifies v1alpha1. Response SHA-256: 655d676dbc436f7b89edfe2c37ef9f383c29b459a22b51c87fcda172a6ccde8e. Production schema publication establishes a live configuration contract, not a provider round trip.
  • The corrected foreman-targeted examples and re-review variants passed validate_factory_files with valid: true, no diagnostics, and no deferred resolutions. Provider access and other state-dependent checks were not exercised.

Unverified claims

  • No live end-to-end GitHub, GitLab, or Azure DevOps run has exercised the foreman-to-review-to-PR publishing workflow, draft/exclusion checks, or re-review instructions for this recipe. Confirm with a matching test event and inspection of the published report before relying on the workflow.
  • The exact inspected server revision has not been matched to a running production deployment. The rollout results above concern checked-in production configuration; schema and published documentation were checked live.
  • The original October 7 stable-release assertion was removed because the cited changelog does not establish all capabilities described here. No replacement release-date claim is made.

Documentation risk

Risk: engineering-review-required
Rationale: Adds new public Factory workflow guidance and configuration examples that require product review.
Docs override: none

PR stack

Dependency: #895 is the shared base and targets main. PRs #896–#901 each independently target factory/factory-use-cases-structure; no guide PR depends on another guide PR.

Human review focus

  • Confirm the recommended initial and re-review events.
  • Confirm that the output and permission boundaries match the intended review-only product path.

Co-Authored-By: Oz oz-agent@warp.dev

Co-Authored-By: Warp agent@warp.dev

@cla-bot cla-bot Bot added the cla-signed label Oct 10, 2026
@vercel

vercel Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 21, 2026 9:59pm UTC

Request Review

@warp-agent-staging warp-agent-staging Bot added the warpy-factory Opened by the Warp factory agents label Oct 10, 2026
@warp-agent-staging

Copy link
Copy Markdown
Contributor Author

This PR was generated with Warp.

Comment @warp-staging-factory on this PR to send it follow-up work.

View run View conversation View on Slack

Co-Authored-By: Warp <agent@warp.dev>
@rachaelrenk
rachaelrenk marked this pull request as ready for review October 21, 2026 20:36
@warp-for-oss

warp-for-oss Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

@rachaelrenk

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /warp-agent-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

@warp-for-oss warp-for-oss Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overview

This PR replaces the Factory code-review use-case scaffold with a full recipe for routing pull requests and merge requests to a review-only factory agent.

Concerns

  • The GitHub automation example promises review on pull requests opened ready for review, but the snippet only subscribes to pull_request_ready; opened non-draft pull requests require pull_request_opened or the prose and test instructions need to narrow scope.
  • No approved spec context was provided. The security pass found no separate security issue beyond keeping the recipe's trigger behavior aligned with its review-only claims.

Verdict

Found: 1 critical, 0 important, 0 suggestions

Request changes

Comment /warp-agent-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

Comment thread src/content/docs/factories/use-cases/code-review.mdx

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

The independent agent completed its review for this commit.

Findings

  • src/content/docs/factories/use-cases/code-review.mdx:34 — [CRITICAL] Keeping the default review agent's shared instructions intact conflicts with asking it to post findings directly: examples/00-warp-default-agents/agents/review/agent.md in warpdotdev/warp-factory-examples at 84e7c952f1506c4bd2784d96fd40a6f4d6398cad requires reports only to the orchestrator and explicitly forbids PR posts. Requested resolution: route through a communicator that publishes the review report, or configure a dedicated review-only agent with explicit posting instructions, then align the automation target, setup steps, and expected result.
  • src/content/docs/factories/use-cases/code-review.mdx:45 — [IMPORTANT] The new GitHub ready-event behavior, GitLab and Azure DevOps event mappings and filters, step 6 re-review behavior, and step 4 definition application lack recorded production evidence; the cited October 7 stable changelog does not establish these capabilities, no implementation sources are cited, and warp-server is unavailable for a rollout check. Requested resolution: record claim-specific sources and production evidence from a named-revision server_rollout.py check, an applicable released OpenAPI or stable changelog entry, or explicit engineering confirmation; otherwise defer the claims or mark them VERIFY and account for them under Unverified claims. Schema acceptance and existing docs are not release evidence.
  • src/content/docs/factories/use-cases/code-review.mdx:68 — [SUGGESTION] The trigger filter reference resolves but contains no skip_bots policy, accepted values, or default, leaving the bot-authored review use case unexplained. Requested resolution: link to an existing explanation that actually owns the policy, or add a source- and rollout-verified explanation to the canonical reference and link to it without duplicating the policy across recipes.

Verdict

Request changes

Co-Authored-By: Warp <agent@warp.dev>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

The independent agent completed its review for this commit.

Findings

  • No findings.

Verdict

Approve

Co-Authored-By: Warp <agent@warp.dev>
@rachaelrenk
rachaelrenk merged commit 4a2577e into factory/factory-use-cases-structure Oct 21, 2026
7 of 8 checks passed
@rachaelrenk
rachaelrenk deleted the factory/factory-use-case-code-review branch October 21, 2026 21:59

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

The independent agent completed its review for this commit.

Findings

  • No findings.

Verdict

Approve

This branch was successfully deployed

1 active deployment
Preview — 71659bbe Deployed Oct 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed warpy-factory Opened by the Warp factory agents

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant