Skip to content

Implement auto-verification for plural-native workflows - #3921

Open
michaeljguarino wants to merge 1 commit into
masterfrom
verification-loops
Open

Implement auto-verification for plural-native workflows#3921
michaeljguarino wants to merge 1 commit into
masterfrom
verification-loops

Conversation

@michaeljguarino

Copy link
Copy Markdown
Member
  • If we know a stack run is spawned by a workbench, auto-verify once it was successful
  • figure out appropriate service equivalent.

Test Plan

Test environment: https://console.your-env.onplural.sh/

Checklist

  • I have added a meaningful title and summary to convey the impact of this PR to a user.
  • If required, I have updated the Plural documentation accordingly.
  • I have added tests to cover my changes.
  • I have deployed the agent to a test environment and verified that it works as expected (required only when changing agent code).

Plural Flow: console

* If we know a stack run is spawned by a workbench, auto-verify once it was successful
* figure out appropriate service equivalent.
@michaeljguarino
michaeljguarino requested a review from a team July 28, 2026 03:34
@michaeljguarino michaeljguarino added the enhancement New feature or request label Jul 28, 2026
@soffi-ai

soffi-ai Bot commented Jul 28, 2026

Copy link
Copy Markdown

Soffi AI Summary

This PR implements auto-verification for plural-native (workbench-driven) workflows — specifically for stack runs that are spawned by a workbench via an AI agent session.

Motivation: When a workbench AI agent creates a pull request that triggers an infrastructure stack run, there was no mechanism to automatically loop back to the workbench once the run completed. This PR closes that feedback loop: if a stack run succeeds and was initiated by a workbench job (via a PR), the system now automatically sends a verification prompt back to the workbench, asking it to confirm the expected infrastructure changes were applied correctly.

Key changes:

  1. PR → StackRun association: A new stack_run_id foreign key is added to the pull_requests table (via migration), allowing a PR to be directly linked to the stack run it triggered. The for_stack_run/1 query helper is added to both PullRequest and WorkbenchJob schemas.

  2. PubSub wiring: In the PullRequestUpdated event handler, after Stacks.poll/1 succeeds, the resulting stack run ID is now written back onto the pull request record. In the StackRunCompleted event handler, Workbenches.kick_workbench/1 is called so that workbench-linked runs automatically trigger the verification step.

  3. Workbench kick logic: Workbenches.kick_workbench/1 looks up the workbench job associated with the completed stack run (via its PR), then posts a user-role message using a new EEx prompt template (priv/prompts/workbench/stack_run_verification.md.eex) that instructs the agent to do a verification pass on the completed changes.

  4. Agent version bump: AGENT_VERSION bumped from v0.6.53 to v0.6.54.

Commits

Commit Summary
68a8e5c Implements the full auto-verification loop for workbench-driven stack runs: adds a stack_run_id FK to pull requests (with migration and schema query helpers), wires the completed stack run ID back onto its originating PR in the PubSub PullRequestUpdated handler, and introduces Workbenches.kick_workbench/1 which — on successful run completion — finds the associated workbench job and posts a verification prompt back to the AI agent. Adds tests covering both the PR→StackRun linkage and the workbench kick-on-completion path. Also bumps the agent version to v0.6.54.

Deploy in Soffi


Updated: 2026-07-28 03:34 UTC

@greptile-apps

greptile-apps Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Implements automatic verification for successful workbench-originated stack runs.

  • Records the stack run created when a merged stack pull request is polled.
  • Sends a verification prompt to the associated workbench after successful completion.
  • Adds the pull-request-to-stack-run schema relationship and migration.
  • Updates the bundled deployment agent to v0.6.54.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
lib/console/deployments/pubsub/recurse.ex Records merged pull requests' generated stack runs and invokes workbench verification on successful stack-queue runs.
lib/console/deployments/workbenches.ex Adds the successful-stack-run lookup and verification-message creation flow.
lib/console/schema/pull_request.ex Adds the stack-run association, query helper, and changeset field.
lib/console/schema/workbench_job.ex Adds a query for locating a workbench job through a pull request's stack run.
priv/repo/migrations/20260727224601_add_wb_verification_fields.exs Adds the nullable indexed stack_run_id foreign key to pull requests.
test/console/deployments/pubsub/recurse_test.exs Covers recording generated stack runs and creating verification activity after successful completion.

Reviews (2): Last reviewed commit: "Implement auto-verification for plural-n..." | Re-trigger Greptile

Comment thread lib/console/deployments/pubsub/recurse.ex
Comment thread lib/console/deployments/workbenches.ex
@michaeljguarino

Copy link
Copy Markdown
Member Author

@greptileai

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant