fix(flows): stop long agent steps at their FLOW_TIME allowance (#138) - #139
Merged
Merged
Conversation
Pass the FLOW_TIME allowances as hard f.agent timeouts (relayflows 2.0.40, AgentWorkforce/flows#606) on check-repair, the adversary reviews, the fixer and check-discovery, and handle completionReason "timeout" explicitly on each: a repair is a failed attempt (re-check, no further repair), a review is unresolved (never clean), a fixer's work is kept and checked, and a discovery falls back to the ecosystem default. Only the cloud target states limits; the local kit's pinned 2.0.26 refuses the option. The budget sweep now runs the long agents past their limits and charges each exactly its limit. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Session-Id: 27242e9e-7fb5-448f-a698-8fa8e9644610
Contributor
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
All reported issues were addressed across 4 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Contributor
|
Preview deployed!
This is a Cloudflare Workers preview version of this PR's build. |
…nt timeouts in tests Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Session-Id: 27242e9e-7fb5-448f-a698-8fa8e9644610
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #138
What
FLOW_TIMEdecided whether a long agent step could start, but nothing stopped it at its allowance (a check-repair agent ran 38m / $54). The generated cloud flow now passes each allowance as a hardf.agenttimeout(relayflows 2.0.40, AgentWorkforce/flows#606), and handlescompletionReason === "timeout"explicitly. A timeout is never treated as success.check-repair-NrepairMinutes(45m)adversary-NreviewMinutes(20m)review.cleanis not read, even if the stopped reviewer left one. Same path as an unresolved review: fix round (traditional, round 1), or the PR goes to draft and the report is posted. That report now says the review was stopped at its limit (review_timeout=yes).fixerfixerMinutes(45m)check-discoverydiscoveryMinutes(15m; measured 6m and 9m24s).relayflow/check.shand falls back to the ecosystem default.Each branch prints a clear
console.errorline.Left unbounded, with the reason in a comment: planner, plan-reviewer, implementer, prototypes and comparator. They are the mandatory path that produces the change. A hard stop there leaves nothing worth publishing, and no measured run overran on them. The header's wallclock still bounds them.
Budget arithmetic
Each allowance is ≤
agentLimitMaxMinutes(60, the runtime ceiling). TherepairStartMinutes/reviewStartMinutes/fixRoundStartMinutesarithmetic is unchanged, because it already assumed each step ran its full allowance. Now the runtime enforces that. The budget sim now models the runtime: an agent that would run past itstimeoutis charged exactly the limit and resolves withcompletionReason: "timeout". The edge-of-every-guard sweep runs repair, review and fixer at 10× their allowances. It asserts that each one timed out and was charged exactly its limit, and that publishing still fits (more than 100 timed-out steps checked).Local target (item 4)
Only the cloud target emits
timeout. The local kit pinsRELAYFLOWS_VERSION2.0.26, and that version'sAgentOptionshas notimeout: a cloud source failstscthere withTS2353 'timeout' does not exist in type 'AgentOptions'. The local flow keeps the sametimedOut()branches. They are typed againstunknown, so they compile against 2.0.26'sAgentResult, which has nocompletionReason, and they never fire. A test fails once the pin reaches 2.0.40, as a reminder to turn the limits on locally.Checked by hand: every generated cloud source typechecks against
@relayflows/surface@2.0.40, and every local source typechecks against@relayflows/surface@2.0.26(tsc --strict, exit 0).Tests (red → green)
flow-workflows.test.ts:f.agentcall carries exactly itsFLOW_TIMElimit (or none).FLOW_REVIEW_BLOCKED_COMMANDwithreview_timeout=yesgives the timeout note in the posted report.flow-budget.test.ts: behaviour tests for repair (fails re-check, and passes re-check), adversary (with a stalereview.clean), first-review timeout leading to a fix round, fixer, discovery and the local target, plus the extended sweep.flow-onboarding.test.ts: updated for thereview_timeout=no;prefix.Before the change, 18 of the new tests failed. Now they all pass, and
tsc --noEmitis clean. Full webnpx vitest runlocally: 378 of 379 pass. The one failure isflow-agent-settings.test.ts › gives every generated preset agent an explicit supported CLI/model pair("Test timed out in 5000ms" under full-suite load). It fails the same way on cleanmainand passes when run alone.Do not redeploy: the orchestrator rolls this onto the live Gardens after merge.
🤖 Generated with Claude Code
Note
Medium Risk
Changes production cloud flow orchestration for repairs, reviews, and check discovery; behavior is well-tested but mis-handling timeouts could draft PRs or skip repairs incorrectly.
Overview
Cloud-generated factory flows now pass hard
f.agenttimeouts (relayflows 2.0.40) for optional long steps—check-discovery,check-repair, adversary reviews, and the fixer—using newFLOW_TIMEallowances including 15m discovery and a 60m runtime ceiling. Steps that hit the limit resolve withcompletionReason: "timeout"via a sharedtimedOut()helper; the flow never treats that as success.Repair: a timed-out repair is one failed attempt (re-check, no second repair), then the existing draft/report path if checks still fail. Review: timeouts are always unresolved (no
review.clean, stalereview.mdcleared between rounds); blocked PR text can includereview_timeout=yes. Discovery: drops a partial.relayflow/check.shand falls back to the ecosystem default. Fixer: keeps partial work and continues check/push/re-review. Planner, implementer, and similar mandatory agents stay unbounded on cloud; local generated flows omittimeoutuntil the kit pins relayflows ≥ 2.0.40.Budget simulation and workflow tests were extended to charge agents at their limits and assert each timeout branch.
Reviewed by Cursor Bugbot for commit 2d639df. Bugbot is set up for automated code reviews on this repo. Configure here.
Summary by cubic
Stops long agent steps at their
FLOW_TIMEallowance as hardf.agenttimeouts (relayflows 2.0.40), so a step that once started only when its allowance fit but then ran on past it (a check-repair agent ran 38m / $54) is now stopped there. A timeout is handled explicitly per step and never treated as success.review.clean. Each reviewer now starts from a clearedreview.md, so a stopped one can never pass off a previous round's findings; the posted report says the review was stopped at its limit.check.shand falls back to the ecosystem default.Only the cloud target states limits; the local kit pins
RELAYFLOWS_VERSION2.0.26, which has no agenttimeout, so its flow keeps the same branches that never fire. The budget sim now charges a step that would run past its limit exactly the limit and resolves it withcompletionReason: "timeout", and a sweep asserts such steps still leave room to publish. Tests also flag any non-literal agenttimeoutso the no-limit checks cannot pass over one.Written for commit 2d639df. Summary will update on new commits.