Repository navigation
feat: adopt pi 1.0 codemode-era APIs in Dynamax, the workflow tool, and subagents - #60
Merged
Merged
Conversation
Resolve the host-provided peers to pi 1.0.4 so the codemode-era extension APIs (tool exposure, isError results, system prompt sections, executeTool) typecheck. ToolInfo fixtures gain the now-required exposure, and the OAuth session test picks any catalog Codex model instead of the retired gpt-5.4.
Returning systemPrompt from before_agent_start forces pi 1.0 to replace the whole prompt head for the run. A <dynamax> section is diffed and appended as a patch instead. The reminder shrinks to the opt-in itself: the authoring rules it repeated already reach the model as the workflow tool's guidelines, and the one-off authoring prompt no longer repeats them a third time.
…odel-only - Synchronous workflow tool results carry the run's subagent usage, which pi adds to the session cost and footer; that spend was missing there before. - Invocation, compile, unknown-workflow and background-start failures return isError: true, so the model sees a failed call while the renderer keeps details. - The tool registers with exposure model-only, like codemode itself: scripts cannot start workflows, and codemode.mode "only" no longer hides it.
Subagents are SDK sessions, which load none of pi's built-in extensions. - Agents that allowlist "codemode" get the codemode extension (mode on), so a stage can batch and filter tool calls in one script. final_answer is now model-only, so a script cannot answer for the agent. - In synchronous workflow tool runs, subagents reach host MCP tools through ctx.executeTool over the host's connections and permission handlers: tools named in the allowlist, or open-world research tools for the external-search hint. The matcher no longer rejects MCP tools for their built-in source and honours MCP readOnly/openWorld annotations. Runs without a bridge log the MCP tools they could not reach. - Tool-result usage inside a subagent (codemode classifier calls) now counts toward the run's usage and budget.
api.classify(context, opts?) answers typed choice/score/bool questions about JSON state through ctx.modelRegistry.classify(), with the host's credentials and no chat session. It defaults to the first credentialed classifier, shares the run's concurrency cap and budget, records usage like an agent, and rejects with a recoverable error when no classifier is configured. It is not journaled.
A missing external-search capability now notes that MCP tools reach subagents only in runs started by the workflow tool, both in the hint error and in the research workflow's next step, instead of implying nothing is installed.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Tool capability filtering can expose unintended tools, and cancellation or workflow failures can omit usage accounting.
Review effort: Balanced
Findings: 1
Open (3)
What changed in this PR
Migrates the workflow engine to pi 1.0 APIs, integrating host orchestration with expanded subagent capabilities.
Changes:
- Uses prompt sections, model-only tool exposure, and explicit error results.
- Adds opt-in codemode, host MCP bridging, and classifier calls.
- Extends usage accounting and updates compatibility documentation and tests.
| File | Description |
|---|---|
| USAGE.md | Documents new APIs and requirements. |
| tests/workflow-tool.test.ts | Checks exposure, errors, and guidance. |
| tests/workflow-options.test.ts | Updates authoring-prompt expectations. |
| tests/workflow-extension-fixtures.ts | Captures tool exposure. |
| tests/usage.test.ts | Tests tool usage accounting. |
| tests/research-workflow.test.ts | Checks MCP capability guidance. |
| tests/pi-compat.test.ts | Tests pi 1.0 minimum version. |
| tests/inline-workflow.test.ts | Adds classifier fixture. |
| tests/host-tools.test.ts | Tests host bridge forwarding. |
| tests/dynamax.test.ts | Tests prompt-section integration. |
| tests/classify.test.ts | Tests classifier selection and accounting. |
| tests/builtin-workflows.test.ts | Adds classifier fixture. |
| tests/agent-session-services.test.ts | Tests opt-in codemode sessions. |
| tests/agent-runner-workspace-tools.test.ts | Tests MCP selection and proxies. |
| tests/agent-runner-replay.test.ts | Updates tool metadata fixtures. |
| tests/agent-runner-fixtures.ts | Adds tool exposure metadata. |
| tests/advisory-provenance.test.ts | Adds classifier fixture. |
| README.md | Updates minimum pi version. |
| bun.lock | Refreshes dependencies for pi 1.0.4. |
| AGENTS.md | Documents integration architecture. |
| .pi/extensions/pi-workflow-engine/src/usage.ts | Aggregates and exports tool usage. |
| .pi/extensions/pi-workflow-engine/src/types.ts | Defines classifier and bridge contracts. |
| .pi/extensions/pi-workflow-engine/src/tool-capabilities.ts | Adds MCP-aware capability matching. |
| .pi/extensions/pi-workflow-engine/src/research-evidence.ts | Clarifies unavailable research capabilities. |
| .pi/extensions/pi-workflow-engine/src/pi-compat.ts | Requires pi 1.0.0. |
| .pi/extensions/pi-workflow-engine/src/host-tools.ts | Implements host MCP proxies. |
| .pi/extensions/pi-workflow-engine/src/engine.ts | Wires classifiers and host tools. |
| .pi/extensions/pi-workflow-engine/src/dynamax.ts | Uses Dynamax prompt sections. |
| .pi/extensions/pi-workflow-engine/src/classify.ts | Implements budgeted classifier calls. |
| .pi/extensions/pi-workflow-engine/src/background-workflow-tool.ts | Flags background-start errors. |
| .pi/extensions/pi-workflow-engine/src/agent-session.ts | Registers codemode and MCP proxies. |
| .pi/extensions/pi-workflow-engine/src/agent-runner-types.ts | Adds host bridge context. |
| .pi/extensions/pi-workflow-engine/index.ts | Integrates host-facing pi 1.0 APIs. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| if (isMutationLikeToolName(tool.name) || tool.sourceInfo.source === "builtin") return false; | ||
| if (isMutationLikeToolName(tool.name) || isPiLocalTool(tool)) return false; | ||
| // MCP servers declare these hints: a tool that changes its environment or stays in a closed domain is not web research. | ||
| if (tool.annotations?.readOnlyHint === false || tool.annotations?.openWorldHint === false) return false; |
| content: [{ type: "text", text: formatWorkflowResultForContext(envelope) }], | ||
| details: envelope, | ||
| // pi counts tool-result usage toward the session cost; subagent spend was invisible there before. | ||
| usage: workflowUsageForPi(envelope.usage), |
Comment on lines
+48
to
+54
| throwIfAborted(rc.signal); | ||
| // One classifier response is one model response: recording it like one keeps per-model usage and the budget whole. | ||
| rc.usage.recordAgentSession({ | ||
| label, | ||
| phase: opts.phase, | ||
| messages: [{ role: "assistant", provider: result.provider, model: result.model, usage: result.usage }], | ||
| }); |
- Dynamax appends to a prompt an earlier extension forced; pi renders no sections then, so the opt-in was lost while the status said active. - The workflow tool's guidelines carry the per-agent profile rule that the Dynamax reminder and authoring prompt no longer repeat. - api.classify() is bounded by the per-agent timeout, records usage before the abort check, and parses model refs like agent(); test registries gain classifier stubs instead of a production branch. - An agent that names an MCP tool the run cannot reach fails before prompting, and pi's MCP resource tools count as MCP tools. - A failed synchronous run returns an isError result carrying the usage its agents spent instead of throwing it away. - Document that bridged tools act in the host session, including for isolated worktree agents.
…sing out of agent-session
api.classify({ model: [...] }) uses the first listed classifier with working
credentials. parseAgentModelRef moves to its own module so classifier code no
longer depends on the session bridge (raised by review runs on this branch).
… verification When a TypeSafe Jev classifier is reachable, the shared lens pipeline asks it whether each same-file pair of candidates is one defect and merges pairs at >= 0.8, keeping every source id, location, impact, and piece of evidence. Without Jev credentials it skips after one probe call. Measured with GPT-6.1 Sol on two code-review benchmarks with planted bugs (42 runs): verifier agents and cost fell ~40% (29 -> 18 agents, $0.60 -> $0.37; 22 -> 14 agents, $1.74 -> $0.89), and in every instrumented run each planted bug a finder proposed reached the final report. diagnose kept the true root cause ranked first in all runs. Codemode for finders and a classifier triage gate were also measured and left out: no recall gain, and no cost gain on top of the merge.
- Complete linkage: a candidate joins a group only when it matches every member, so a vague candidate cannot chain two distinct defects together. - Folded wordings stay visible to the verifier as "Also reported as" evidence. - Any failed probe (bad key, rate limit) skips the merge after one call, with a log line; aborts still propagate. - Pair candidates by their changed-line anchor and repository-relative path. - Resolve each classifier route once per run instead of once per call, and log how many pairs are compared. Re-measured with GPT-6.1 Sol: 8/8 at 17 agents ($0.33) on the small benchmark, 7/8 at 15 agents ($1.10) on the large one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adopts the pi 1.0 extension APIs (codemode, tool exposure, prompt sections,
executeTool, classifiers) across Dynamax, theworkflowtool, and subagents. Requires pi 1.0.0 or newer.Host prompt and tool
<dynamax>system prompt section. ReturningsystemPromptfrombefore_agent_startmade pi 1.0 replace the whole prompt head for that run. Section edits are appended as patches, so the cached head is kept. The reminder is cut down to the opt-in itself; its authoring rules duplicated the tool's guidelines, and the one-off authoring prompt repeated them a third time./session. A failed run now returns anisErrorresult carrying what it spent, instead of throwing that usage away.isError: true; the renderer still readsdetails.workflowismodel-only. Like codemode itself, it is an orchestrator: scripts can't start runs (which would lose the tool row, hit the script output cap, and multiply the concurrency cap), andcodemode.mode: "only"no longer hides it.Subagents
Subagents are SDK sessions, so pi's built-in extensions don't load in them.
"codemode"intoolsget the extension (modeon). Scripts can call only that agent's other allowed tools.final_answeris nowmodel-only, so a script can't answer for the agent.workflowtool runs, subagents reach host MCP tools throughctx.executeTool, over the host's connections and permission handlers: tools named intools, or open-world research tools fortoolHints: ["external-search"]. The matcher no longer rejects MCP tools for their built-in source, and it honours MCPreadOnlyHint/openWorldHint. Bridged tools act in the host session, including for isolated worktree agents (documented)./workflowcommands and background runs outlive the tool call that makes this possible. An agent there that names an MCP tool fails before prompting, andresearchexplains this in its next step. pi's MCP resource tools count as MCP tools.api.classify()Answers typed
choice/score/boolquestions about JSON state with a pi classifier model, without a chat session. It defaults to the first credentialed classifier, parses model refs likeagent(), and shares the run's concurrency cap, budget, and per-agent timeout. It rejects with a recoverable error when no classifier is configured, and is not journaled.Duplicate-candidate merge (measured)
code-review,diagnose,perf-review, andrefactor-scoutnow merge same-file candidates that a TypeSafe Jev classifier calls one defect before verification. A candidate joins a group only when it matches every member at p >= 0.8, and folded wordings stay visible to the verifier. If the first comparison fails (no Jev credentials, a bad key, a rate limit), the merge is skipped.I measured codemode and classify opt-ins for the advisory pipeline with GPT-6.1 Sol: 42 runs over two code-review benchmarks with planted bugs, plus 8 diagnose runs.
lost=none). The merge's lower mean on the large diff is finder variance (p ~ 0.17), not merges.diagnose, the true root cause ranked first in 8/8 runs with and without it.Verification
bun run typecheckandbun run test(603 pass) against pi 1.0.4 types.{dynamax: null}and<dynamax>patches.isError: trueand the TUI kept the✗line.tools.workflowwas rejected (does not exist), whileworkflowstayed declared.tools.final_answerwas rejected. The external-search subagent gotmcp__fakeweb__web_searchbut not thereadOnlyHint: falsetool. The named-tool subagent got exactly its tool. Both MCP calls went over the host's connection.api.classifycalls (explicit and default model) were recorded in run usage./sessionshowed the workflow's subagent tokens under Tools/summaries.researchvia the tool got past its capability check through the bridge; via/workflowit reported the new MCP guidance./sessionstill showed its subagent and classifier usage under Tools/summaries.Review
A
/code-reviewpass found 10 candidates. Nine are addressed in 374f1c0: eight with code changes, and the host-cwd caveat for bridged tools in isolated agents with docs. One needed no change: tool-result usage carries the agent's chat model label, but the only per-agent view sums usage by label and nothing splits cost per model, so no spend is shown under the wrong model.A second pass over the merge commits found 9 more. I fixed 6 of them in the last commit (complete linkage, folded wordings, probe failures, anchor-based pairing, per-run route cache, progress log). I left 3 unchanged: