diff --git a/.agents/skills/release/SKILL.md b/.agents/skills/release/SKILL.md index 8d2f9c0..7bd20e5 100644 --- a/.agents/skills/release/SKILL.md +++ b/.agents/skills/release/SKILL.md @@ -258,7 +258,7 @@ node --input-type=module -e "import('@onkernel/cua-ai').then((m) => { if (typeof ``` For `@onkernel/cua-agent`, install `@onkernel/cua-agent@` the same -way and check `typeof m.CuaAgent === "function"`. For the CLI, install it in a +way and check `typeof m.attach === "function"`. For the CLI, install it in a fresh directory and verify `./node_modules/.bin/cua --help` prints `Usage:`. If a workflow fails after a tag is pushed, do not reuse the same package diff --git a/README.md b/README.md index 7970f58..d030e31 100644 --- a/README.md +++ b/README.md @@ -41,8 +41,8 @@ packages/ ``` **Building your own agent? Start here:** [`packages/agent`](packages/agent) -(`@onkernel/cua-agent`) — `CuaAgent`/`CuaAgentHarness` run the full -computer-use loop against a Kernel browser. It sits on +(`@onkernel/cua-agent`) — `attach()` binds a Kernel browser and compiles a +(model, tools) pair into plain pi objects you drive yourself. It sits on [`packages/ai`](packages/ai) (`@onkernel/cua-ai`), the model layer with the pi-ai model catalog, canonical tool schemas, and per-provider adapters on top of pi-ai; reach for cua-ai directly only when you bring your @@ -148,7 +148,7 @@ cua -p -o jsonl "open example.com and tell me the heading" `pi-agent-core`'s `Agent`/`AgentHarness`. It materializes the caller's exact catalog over one shared resource pool and executes canonical actions through Kernel's computer API or a raw-CDP browser executor. -3. **CLI** — `@onkernel/cua-cli` assembles a `CuaAgentHarness` from +3. **CLI** — `@onkernel/cua-cli` assembles a pi `AgentHarness` from command-line flags, env-var-based API keys, a `JsonlSessionRepo` for transcripts, and pi skills; renders the result either as plain text (`--print`), JSONL events (`-o jsonl`), or an interactive pi-tui diff --git a/docs/agent-tool-configuration-spec.md b/docs/agent-tool-configuration-spec.md index 4125edb..53d96bc 100644 --- a/docs/agent-tool-configuration-spec.md +++ b/docs/agent-tool-configuration-spec.md @@ -1,6 +1,11 @@ # Agent Tool Configuration -**Status:** Implemented +**Status:** Superseded by `attach()` (2026-08-14). The tool-array-as-single-source-of-truth +rule it establishes still holds; what changed is where the array lives. `CuaAgent` and +`CuaAgentHarness` are gone, and a caller now compiles a (model, tools) pair through +`attach()` and owns the selection itself, so the `setTools()`/`setModel()` mutation surface +and its in-tool `executionMode: "sequential"` guard described below no longer exist. +Retained as the record of why the tool array is explicit and required. **Scope:** `@onkernel/cua-agent` and the tool-building surface in `@onkernel/cua-ai` **Compatibility:** Not a goal; these packages are alpha and may make breaking API changes. diff --git a/docs/architecture.md b/docs/architecture.md index 66dcde9..8cbd5d6 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -82,15 +82,17 @@ before a model request. ## Dynamic catalogs -`CuaAgent` and `CuaAgentHarness` use composition around pi and expose: +`attach()` returns a handle; `compile()` turns a (model, tools) pair into plain +pi objects, and `apply()` swaps a running harness onto a new pair: ```ts -agent.getTools(); -agent.setTools(nextTools); -agent.setModel(nextModel); +const kb = attach({ browser, client }); +const compiled = kb.compile({ model, tools }); +await compiled.apply(harness); ``` -`setTools()` recompiles atomically before mutating pi state. Existing tool +Nothing mutates in place: a change compiles a new pair, and `compile()` throws +before anything reaches pi. Existing tool identity with a changed schema, executor, or coordinates counts as a real replacement. Additions made from inside a running tool are recorded in pi's Anthropic-compatible `addedToolNames` marker only when that provider/model can @@ -200,10 +202,14 @@ derives a transport that the rest of the selection must be compatible with. Callers rebuild the menu after each staged change rather than caching a per-tool verdict. -`CuaAgent` and `CuaAgentHarness` push the compiled `catalog.model` into pi on -every construction and on every `setTools()`/`setModel()`, so the derived -transport applies uniformly regardless of which mutation path selected the -tools. +`apply()` pushes the compiled `catalog.model` into pi alongside its tools, and +only when the derived transport actually moved, so a tools-only change records +no model change while a transport-moving change records exactly one. + +pi fixes a harness's `models` at construction, but the headers, payload +transforms, and incoming tool plan it applies are per-catalog. The handle +therefore owns one `Models` collection that serves whichever pair was last +activated; `activate()` is what redirects it, and `apply()` calls it. Generated payload processing has fixed order: model preparation, tool serialization, provider fields, then the caller's `onPayload` hook. @@ -221,7 +227,9 @@ serialization, provider fields, then the caller's `onPayload` hook. - Anthropic's native browser tool when the model supports it; - Google's native browser action set; 3. creates and retains its own application-level coding-tool list; -4. passes the complete list to `CuaAgentHarness`; +4. compiles the complete list through the handle and hands the result to a + stock pi `AgentHarness`, retaining the selection in `CuaCliCatalog` so + `/model` and `/tools` can recompile it; 5. builds a caller-owned prompt from loaded skills and context files; 6. uses one `Session` for transcript persistence and resume; 7. exposes `cua act ''` as a model-free path to the same `browser_act` @@ -257,14 +265,14 @@ switch — run through that one queue, because each suspends across several `setTools()`/`setModel()` calls. Without it an apply could land between a switch's `setModel()` and its final `setTools()` and fail its compile against the wrong provider. Selectors also refuse to open mid-turn: the agent's -execution-scope guard only covers mutation from inside a tool's `execute`, so -this TUI-side check is what protects a streaming request. +compiled pair is immutable, so this TUI-side check is what keeps a swap from +landing mid-request. ## Per-turn flow ```text user prompt - -> CuaAgentHarness / pi agent loop + -> pi agent loop -> active identity-keyed catalog -> generated headers and payload transforms -> caller onPayload diff --git a/docs/cua-cli-harness-migration.md b/docs/cua-cli-harness-migration.md index a45c055..f53029b 100644 --- a/docs/cua-cli-harness-migration.md +++ b/docs/cua-cli-harness-migration.md @@ -1,8 +1,10 @@ # CUA CLI Harness Migration -**Status:** Completed and superseded. +**Status:** Historical (2026-08-14). The CLI now composes a stock pi `AgentHarness` from +`attach()` rather than `CuaAgentHarness`, which no longer exists. Retained as the record of +the print/action/interactive consolidation this describes. -The CLI now uses the shared `CuaAgentHarness` composition path for print, +The CLI uses one shared composition path for print, action, and TUI flows. Its current architecture—including explicit tool-list selection, coding-tool composition, sessions, skills, and rendering—is documented in [`architecture.md`](architecture.md#cli-composition). diff --git a/packages/agent/CHANGELOG.md b/packages/agent/CHANGELOG.md index 188fc2d..34b8c22 100644 --- a/packages/agent/CHANGELOG.md +++ b/packages/agent/CHANGELOG.md @@ -2,52 +2,56 @@ ## Unreleased +Breaking: `CuaAgent` and `CuaAgentHarness` are removed. cua-agent hands back +plain pi objects; the caller constructs the agent. + - Add `attach({ browser, client })`, returning a handle that compiles - (model, tools) pairs into plain pi objects: the model carrying the transport - its tools derive, executables materialized against the handle's browser pool, - a `Models` collection adding provider retry, required headers, the catalog's - payload transforms and the tool-result image bound, and an `install(harness)` - for the behaviors that are pi event handlers rather than constructor options. - The handle owns what actually persists — the Kernel client and browser, the - translator, the raw-CDP executor, ref and frame state — so a spec materializes - once across repeat compiles. -- `CuaAgent` and `CuaAgentHarness` are unchanged and now share their internals - with `attach()` rather than owning private copies. They are slated to retire - in favor of the handle. -- Add `CuaAgentHarness.setModelAndTools()`. A model switch that also swaps - interaction tools has to compile as one pair now that the selected tools - decide the transport: staging the two in sequence produces an intermediate - catalog whose derived transport differs from both the old and the new one, and - records a model change for a transport nothing ever streamed with. -- `CuaAgentHarness` no longer refuses a model ref that is absent from its - supplied `Models` collection: it falls back to the registry, and an id the - registry lacks is synthesized. -- The model streamed for a Google model now depends on which tools `CuaAgent` / - `CuaAgentHarness` were constructed or mutated with: selecting Google's native - browser toolset still compiles to the CUA-owned Interactions API, but a Google - model selected with only CDP browser tools now streams through pi's builtin - Google transport instead of always carrying the CUA-owned api. This applies - uniformly across construction, `setTools()`, and `setModel()`, since all three - feed the same compiled `catalog.model` into pi. -- Fix `setTools()` recompiling from the previously *compiled* model instead of - the caller's model selection: dropping a native toolset that had derived a - tool-selection-dependent api (e.g. Google's Interactions API) left subsequent - tools-only recompiles stuck on that api even though the new selection no - longer required it. `CuaToolManager` now recompiles tools-only changes from - the model input the caller last selected. -- `responseThreading` (`CuaAgentOptions`/`CuaAgentHarnessOptions`) no longer - affects OpenAI models: OpenAI now streams through pi-ai's builtin Responses - transport and its automatic prompt caching regardless of this flag. The option - still governs Google's `previous_response_id`-style continuation. + (model, tools) pairs into plain pi objects: `model` carrying the transport its + tools derive, `tools` / `agentTools` materialized against the handle's browser + pool, `models` adding provider retry, required headers, the catalog's payload + transforms and the tool-result image bound, `activate(harness)` for the + behaviors that are pi event handlers rather than constructor options, and + `apply(harness)` to swap a running harness onto a new pair. The handle owns + what actually persists — the Kernel client and browser, the translator, the + raw-CDP executor, ref and frame state — so a spec materializes once across + repeat compiles. +- `getTools()`, `setTools()`, `setModel()`, and `setModelAndTools()` are gone + with the classes. A change compiles a new pair and applies it, so the current + selection belongs to the caller and there is no second copy of it to drift. + `compile()` throws before anything reaches pi, and `apply()` restores the + previous pair if pi rejects the new one, so the atomicity those methods + provided is preserved. `apply()` sets the model only when the derived + transport actually moved, so a tools-only change records no model change. +- `CuaToolManager` is now immutable: one compiled pair per instance, with + `prepareTools`/`prepareModel`/`prepareModelAndTools`/`commit`/`getTools` and + the async-local execution scope removed. Removing the execution scope also + removes cache-preserving deferred tool addition: a tool that added tools mid + execution used to have those names recorded on its result as + `addedToolNames`, letting pi extend an OpenAI request without invalidating the + prompt-cache prefix. Nothing produced them outside that mutation path. The + transport still consumes `addedToolNames` on a transcript that carries them. +- The tool-result image bound, payload transforms, and required headers now + follow whichever pair is active rather than the one a harness was built with. + pi fixes `models` at construction while those are per-catalog, so one + collection per handle is what makes a swap possible at all. +- A model ref absent from a supplied `Models` collection falls back to the + registry, and an id the registry lacks is synthesized. +- The model streamed for a Google model depends on which tools it was compiled + with: selecting Google's native browser toolset compiles to the CUA-owned + Interactions API, while a Google model selected with only CDP browser tools + streams through pi's builtin Google transport. +- `responseThreading` (`CuaAttachOptions`) no longer affects OpenAI models: + OpenAI streams through pi-ai's builtin Responses transport and its automatic + prompt caching regardless of this flag. The option still governs Google's + `previous_response_id`-style continuation. - Exempt OpenAI's native computer tool from the tool-result image replay limit. Its `computer_call_output` items must each carry a screenshot, and stateless replay no longer leaves them in provider-stored state. Breaking: Tzafon and Yutori support is removed. -- Constructing a `CuaAgent` or `CuaAgentHarness` with a Tzafon or Yutori model - ref now fails to resolve the model, and `cua.providers.tzafon` / - `cua.providers.yutori` no longer exist. +- Compiling a Tzafon or Yutori model ref now fails to resolve the model, and + `cua.providers.tzafon` / `cua.providers.yutori` no longer exist. - Remove `CuaExecutionResources.viewport`. It only fed the removed catalog viewport option; the same value is still on `resources.browser.viewport`. diff --git a/packages/agent/README.md b/packages/agent/README.md index a020bfc..8f94c08 100644 --- a/packages/agent/README.md +++ b/packages/agent/README.md @@ -12,22 +12,31 @@ npm install @onkernel/cua-agent @onkernel/cua-ai @onkernel/sdk Requires Node 22.19 or newer, `KERNEL_API_KEY`, and the selected model provider's API key. -## `CuaAgent` +## `attach()` + +`attach()` binds a Kernel browser to CUA's execution resources and returns a +handle. `compile()` turns a (model, tools) pair into plain pi objects; you +construct whatever pi agent you want with them. There is no CUA agent class. ```ts import Kernel from "@onkernel/sdk"; import { cua } from "@onkernel/cua-ai"; -import { CuaAgent } from "@onkernel/cua-agent"; +import { Agent, attach } from "@onkernel/cua-agent"; const client = new Kernel({ apiKey: process.env.KERNEL_API_KEY! }); const browser = await client.browsers.create({ stealth: true }); +const kb = attach({ client, browser }); -const agent = new CuaAgent({ - client, - browser, +const { model, agentTools, models } = kb.compile({ + model: "anthropic:claude-opus-5", tools: cua.toolsets.browser(), +}); + +const agent = new Agent({ + streamFn: (selected, context, options) => models.streamSimple(selected, context, options), initialState: { - model: "anthropic:claude-opus-5", + model, + tools: [...agentTools], systemPrompt: "Inspect and interact with the page using the requested tools.", }, }); @@ -35,36 +44,54 @@ const agent = new CuaAgent({ try { await agent.prompt("Open example.com and report the heading."); } finally { + await kb.dispose(); await client.browsers.deleteByID(browser.session_id); } ``` -## `CuaAgentHarness` +The compiled `model` carries the transport its tools derive: selecting a +provider-native browser or computer surface can change `model.api`, so the pair +has to reach pi together. -Use the harness for session-backed transcripts, skills, prompt templates, -compaction, steering, and follow-ups: +## With pi's `AgentHarness` + +Use pi's harness for session-backed transcripts, skills, prompt templates, +compaction, steering, and follow-ups. `activate()` registers the behaviors CUA +owns that are pi event handlers rather than constructor options, and points the +handle's `models` at this catalog: ```ts -import { - CuaAgentHarness, - InMemorySessionRepo, -} from "@onkernel/cua-agent"; +import { AgentHarness, attach, InMemorySessionRepo } from "@onkernel/cua-agent"; import { cua } from "@onkernel/cua-ai"; -const repo = new InMemorySessionRepo(); -const session = await repo.create(); -const harness = new CuaAgentHarness({ - client, - browser, +const session = await new InMemorySessionRepo().create(); +const kb = attach({ client, browser }); +const compiled = kb.compile({ model: "openai:gpt-5.6-sol", tools: cua.toolsets.browser() }); + +const harness = new AgentHarness({ session, - model: "openai:gpt-5.6-sol", - tools: cua.toolsets.browser(), + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), systemPrompt: "Use the supplied browser tools.", }); +compiled.activate(harness); await harness.prompt("Find the pricing page."); ``` +To change the model or the tool list on a running harness, compile the new pair +and apply it: + +```ts +await kb.compile({ model: "google:gemini-3.6-flash", tools: cua.providers.google.toolsets.browser() }).apply(harness); +``` + +`apply()` moves the model and tools together, sets the model only when the +derived transport actually moved, and restores the previous pair if pi rejects +the new one. + The package re-exports pi-agent-core session, skill, prompt-template, compaction, and execution-environment primitives used with the harness. @@ -77,27 +104,34 @@ to every tool call: ```ts import { - CuaAgentHarness, + AgentHarness, + attach, NodeExecutionEnv, createBashTool, createReadTool, type ExecutionToolContext, } from "@onkernel/cua-agent"; -const harness = new CuaAgentHarness({ - client, - browser, - session, +const compiled = kb.compile({ model: "openai:gpt-5.6-sol", tools: [createReadTool(), createBashTool(), ...cua.toolsets.browser()], +}); +const harness = new AgentHarness({ + session, + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), toolContext: { env: new NodeExecutionEnv({ cwd: process.cwd() }) }, systemPrompt: "Use the supplied tools.", }); +compiled.activate(harness); ``` -CUA specs and plain pi `AgentTool`s are accepted too — they simply ignore the -context. The low-level `CuaAgent` stays context-free: its tools are ordinary -pi `AgentTool`s (`CuaAgentTool`). +Compile for the same context the harness delivers, so a later swap stays +type-compatible. CUA specs and plain pi `AgentTool`s are accepted too — they +simply ignore the context. `compiled.agentTools` is the context-free view for +the low-level `Agent`. ## Choosing tools @@ -150,24 +184,23 @@ the active credential cannot access `browser_20260701`. ## Dynamic catalogs -Both classes expose the same catalog controls: +Changing the model or the tool list compiles a new pair; nothing mutates in +place: ```ts -agent.getTools(); // copy of the exact requested inputs -agent.setTools(next); // atomic compile + replace -agent.setModel(nextModel); +const next = kb.compile({ model: nextModel, tools: nextTools }); +await next.apply(harness); ``` -The requested catalog is recompiled on every `setTools()` or `setModel()`. Duplicate identities, caller-visible name collisions, provider-normalized name -collisions, and incompatible model/tool combinations fail before state changes. +collisions, and incompatible model/tool combinations fail in `compile()`, before +anything reaches pi. `apply()` then moves the model and tools together and +restores the previous pair if pi rejects the new one. -Tools may call `setTools()` or `setModel()` while they execute, but only if they -declare `executionMode: "sequential"`; mutating the catalog from a tool that can -run in parallel is rejected. Eligible ordinary function-tool additions are -recorded for pi's deferred-loading protocol; additions outside a tool execution -are eager. Provider-native tools are always eager. Replacing an existing -identity's schema, executor, or coordinates is a real replacement. +The handle holds the current selection only in the sense that `models` serves +whichever pair was last activated; the *selection* itself belongs to the caller, +which is what removes the class of bug where a compiled pair and the live one +disagree. One shared execution-resource pool survives all catalog/model changes, so browser refs, tabs, connections, and translator state are not reset. @@ -222,7 +255,7 @@ const lookup = { }, }; -agent.setTools([lookup, ...cua.toolsets.browser()]); +kb.compile({ model, tools: [lookup, ...cua.toolsets.browser()] }); ``` Caller tools receive identity `caller.` through cua-ai's canonical @@ -230,17 +263,15 @@ Caller tools receive identity `caller.` through cua-ai's canonical fingerprint rules. `CuaAgentTool` is defined and exported by this package: cua-ai compiles declaration-only catalogs and never sees executors, while cua-agent projects caller `AgentTool`s into fresh declarations, joins compiled -entries back by identity, materializes each CUA spec exactly once per shared -execution-resource pool, and owns implementation identity for replacement -detection (a reused `execute` function keeps its identity across wrappers; a -new `execute` or freshly created spec object is a conservative replacement). +entries back by identity, and materializes each CUA spec exactly once per shared +execution-resource pool, so repeat compiles hand pi a stable implementation. ## Events and state -`CuaAgent` delegates pi's prompt/continue/steer/follow-up/abort lifecycle and -subscriptions. `CuaAgentHarness` delegates harness events and session APIs. -`CuaAgent.state.tools` is the active materialized list; use `getTools()` for a -copy of the exact requested catalog. +The agent is pi's, so its lifecycle, events, and session APIs are pi's too. +What CUA adds on top is `activate()`: failed tool results are marked, a turn's +remaining calls are blocked after one fails, and an empty successful response +can be followed up. It returns a release. ## Development diff --git a/packages/agent/src/agent.ts b/packages/agent/src/agent.ts deleted file mode 100644 index 6e38b5f..0000000 --- a/packages/agent/src/agent.ts +++ /dev/null @@ -1,495 +0,0 @@ -import { - Agent, - AgentHarness, - type AgentEvent, - type AgentHarnessEvent, - type AgentHarnessEventResultMap, - type AgentHarnessOptions, - type AgentHarnessOwnEvent, - type AgentHarnessToolContextSource, - type AgentHarnessResources, - type AgentHarnessStreamOptions, - type AgentHarnessTool, - type AgentMessage, - type AgentOptions, - type AgentTool, - type NavigateTreeResult, - type PromptTemplate, - type QueueMode, - type Session, - type Skill, - type StreamFn, - type ThinkingLevel, -} from "@earendil-works/pi-agent-core"; -import { - type Api, - type Context, - cuaModels, - type CuaIncomingToolPlan, - type CuaModelRef, - getCuaModel, - parseCuaModelRef, - type CuaSimpleStreamOptions, - getCuaEnvApiKey, - type ImageContent, - type Model, - type Models, - type SimpleStreamOptions, -} from "@onkernel/cua-ai"; -import type Kernel from "@onkernel/sdk"; -import { - resolveProviderRetryPolicy, - type CuaRetryOptions, - withProviderRetry, - withProviderRetryModels, -} from "./provider-retry"; -import { CuaExecutionResources, type CuaExecutionDetails } from "./resources"; -import { CuaToolManager, type CuaAgentTool, type CuaHarnessTool } from "./tool-manager"; -import { - type CuaEmptyResponseRecoveryOptions, - type CuaModelInput, - defaultCuaStream, - hasExecutionError, - isEmptyAssistantResponse, - modelTransportChanged, - projectToolResultImages, - requiredImageToolNames, - resolveEmptyResponseRecovery, - resolveModelFromCollection, - resolveResponseThreading, - resolveToolResultImageReplayLimit, - type ToolResultImageReplayLimit, - turnFailureStopMessage, - withCatalogModels, -} from "./attach"; -import type { KernelBrowser } from "./translator/translator"; - -/** Mutable conversation state exposed by {@link CuaAgent}. */ -export interface CuaAgentState { - systemPrompt: string; - model: Model | CuaModelRef; - thinkingLevel: ThinkingLevel; - messages: AgentMessage[]; - readonly tools: readonly AgentTool[]; - readonly isStreaming: boolean; - readonly streamingMessage?: AgentMessage; - readonly pendingToolCalls: ReadonlySet; - readonly errorMessage?: string; -} - -type CuaAgentInitialState = Omit, "model" | "tools"> & { - model: CuaModelInput; -}; - -/** Construction options for {@link CuaAgent}, including its exact caller-owned tool catalog. */ -export type CuaAgentOptions = Omit & { - browser: KernelBrowser; - client: Kernel; - tools: readonly CuaAgentTool[]; - initialState: CuaAgentInitialState; - /** Defaults to streaming through the shared {@link cuaModels} collection. */ - streamFn?: AgentOptions["streamFn"]; - emptyResponseRecovery?: CuaEmptyResponseRecoveryOptions; - toolResultImageReplayLimit?: ToolResultImageReplayLimit; - /** Governs Google's `previous_response_id`-style continuation. Every other provider streams through pi's transports and their automatic prompt caching regardless of this flag. Defaults to `true`. */ - responseThreading?: boolean; - retry?: CuaRetryOptions; -}; - -type CuaAgentHarnessOptionsBase< - TContext extends object | undefined, - TSkill extends Skill, - TPromptTemplate extends PromptTemplate, -> = Omit< - AgentHarnessOptions>, - "activeToolNames" | "model" | "models" | "tools" | "toolContext" | "retry" -> & { - browser: KernelBrowser; - client: Kernel; - model: CuaModelInput; - models?: Models; - tools: readonly CuaHarnessTool[]; - onPayload?: SimpleStreamOptions["onPayload"]; - emptyResponseRecovery?: CuaEmptyResponseRecoveryOptions; - toolResultImageReplayLimit?: ToolResultImageReplayLimit; - /** Governs Google's `previous_response_id`-style continuation. Every other provider streams through pi's transports and their automatic prompt caching regardless of this flag. Defaults to `true`. */ - responseThreading?: boolean; - retry?: CuaRetryOptions; -}; - -/** - * Construction options for {@link CuaAgentHarness}, including its exact - * caller-owned tool catalog. Mirrors pi's `AgentHarnessOptions` generic order: - * the tool context first, then skill and prompt-template resource types. The - * supplied `toolContext` is forwarded to pi untouched, and every executable - * tool receives it on each call. - */ -export type CuaAgentHarnessOptions< - TContext extends object | undefined = undefined, - TSkill extends Skill = Skill, - TPromptTemplate extends PromptTemplate = PromptTemplate, -> = CuaAgentHarnessOptionsBase & ([TContext] extends [undefined] ? { - /** Context-free harnesses do not need a tool context. */ - toolContext?: undefined; -} : { - /** Static context or zero-argument context provider resolved for each turn snapshot. */ - toolContext: AgentHarnessToolContextSource; -}); - -/** Pi Agent behavior with an explicit, identity-keyed CUA tool catalog. */ -export class CuaAgent { - private readonly coreAgent: Agent; - private readonly tools: CuaToolManager; - private runtimeDirty = false; - private readonly stateView: CuaAgentState; - private emptyResponseRecoveryAttempts = 0; - - constructor(options: CuaAgentOptions) { - const { - browser, - client, - tools: requestedTools, - initialState, - onPayload, - streamFn, - prepareNextTurn, - prepareNextTurnWithContext, - transformContext, - afterToolCall, - beforeToolCall, - emptyResponseRecovery, - toolResultImageReplayLimit, - responseThreading, - retry, - ...agentOptions - } = options; - const recovery = resolveEmptyResponseRecovery(emptyResponseRecovery); - const imageReplayLimit = resolveToolResultImageReplayLimit(toolResultImageReplayLimit); - const useResponseThreading = resolveResponseThreading(responseThreading); - const resources = new CuaExecutionResources({ browser, client }); - const manager = new CuaToolManager(resources, initialState.model, requestedTools); - const retryingStream = withProviderRetry(streamFn ?? defaultCuaStream, resolveProviderRetryPolicy(retry)); - const streamWithCatalog: StreamFn = (model, context, streamOptions) => { - const catalog = manager.catalog; - const generatedOnPayload = async (payload: unknown, selectedModel: Model) => { - const generated = await catalog.payload.apply(payload, selectedModel); - return onPayload ? (await onPayload(generated, selectedModel)) ?? generated : generated; - }; - const cuaOptions: CuaSimpleStreamOptions = { - ...streamOptions, - headers: catalog.headers.merge(streamOptions?.headers), - onPayload: generatedOnPayload, - disableResponseThreading: !useResponseThreading, - cuaIncomingToolPlan: catalog.incoming, - }; - return retryingStream(model, context, cuaOptions); - }; - const failedTurns = new WeakSet(); - let core!: Agent; - core = new Agent({ - ...agentOptions, - getApiKey: agentOptions.getApiKey ?? getCuaEnvApiKey, - streamFn: streamWithCatalog, - beforeToolCall: async (context, signal) => { - const stopMessage = turnFailureStopMessage(manager); - if (stopMessage && failedTurns.has(context.assistantMessage)) return { block: true, reason: stopMessage }; - return beforeToolCall?.(context, signal); - }, - afterToolCall: async (context, signal) => { - const result = await afterToolCall?.(context, signal); - const forcedError = hasExecutionError(result?.details ?? context.result.details); - if ((result?.isError ?? context.isError) || forcedError) failedTurns.add(context.assistantMessage); - return forcedError ? { ...result, isError: true } : result; - }, - transformContext: async (messages, signal) => projectToolResultImages( - transformContext ? await transformContext(messages, signal) : messages, - imageReplayLimit, - requiredImageToolNames(manager.catalog.incoming), - ), - prepareNextTurnWithContext: async (context, signal) => { - const update = prepareNextTurnWithContext - ? await prepareNextTurnWithContext(context, signal) - : await prepareNextTurn?.(signal); - if (update?.model) this.setModel(update.model as CuaModelInput); - if (!update && !this.runtimeDirty) return undefined; - this.runtimeDirty = false; - return { - ...update, - model: core.state.model, - context: { - ...(update?.context ?? context.context), - tools: core.state.tools.slice(), - }, - }; - }, - initialState: { - ...initialState, - model: manager.catalog.model, - tools: manager.agentTools(), - systemPrompt: initialState.systemPrompt ?? "", - }, - }); - this.coreAgent = core; - this.tools = manager; - this.stateView = this.createStateView(); - - if (recovery && recovery.maxAttempts > 0) { - this.subscribe((event, signal) => { - if (event.type === "agent_start") { - this.emptyResponseRecoveryAttempts = 0; - return; - } - if (event.type !== "turn_end" || !isEmptyAssistantResponse(event.message)) return; - if (signal.aborted || this.emptyResponseRecoveryAttempts >= recovery.maxAttempts || this.hasQueuedMessages()) return; - this.followUp({ role: "user", content: [{ type: "text", text: recovery.followUp }], timestamp: Date.now() }); - this.emptyResponseRecoveryAttempts += 1; - }); - } - } - - get state(): CuaAgentState { - return this.stateView; - } - - getTools(): readonly CuaAgentTool[] { - return this.tools.getTools(); - } - - setTools(tools: readonly CuaAgentTool[]): void { - const prepared = this.tools.prepareTools(tools); - this.coreAgent.state.model = prepared.catalog.model; - this.coreAgent.state.tools = this.tools.agentTools(prepared); - this.tools.commit(prepared); - this.runtimeDirty = true; - } - - getModel(): Model { - return this.tools.catalog.model; - } - - setModel(model: CuaModelInput): void { - const prepared = this.tools.prepareModel(model); - this.coreAgent.state.model = prepared.catalog.model; - this.coreAgent.state.tools = this.tools.agentTools(prepared); - this.tools.commit(prepared); - this.runtimeDirty = true; - } - - prompt(...args: Parameters): Promise { - return this.coreAgent.prompt(...args); - } - - continue(): Promise { return this.coreAgent.continue(); } - steer(message: AgentMessage): void { this.coreAgent.steer(message); } - followUp(message: AgentMessage): void { this.coreAgent.followUp(message); } - clearSteeringQueue(): void { this.coreAgent.clearSteeringQueue(); } - clearFollowUpQueue(): void { this.coreAgent.clearFollowUpQueue(); } - clearAllQueues(): void { this.coreAgent.clearAllQueues(); } - hasQueuedMessages(): boolean { return this.coreAgent.hasQueuedMessages(); } - abort(): void { this.coreAgent.abort(); } - waitForIdle(): Promise { return this.coreAgent.waitForIdle(); } - reset(): void { this.coreAgent.reset(); } - subscribe(listener: (event: AgentEvent, signal: AbortSignal) => Promise | void): () => void { return this.coreAgent.subscribe(listener); } - get signal(): AbortSignal | undefined { return this.coreAgent.signal; } - get steeringMode(): QueueMode { return this.coreAgent.steeringMode; } - set steeringMode(mode: QueueMode) { this.coreAgent.steeringMode = mode; } - get followUpMode(): QueueMode { return this.coreAgent.followUpMode; } - set followUpMode(mode: QueueMode) { this.coreAgent.followUpMode = mode; } - - async dispose(): Promise { - this.abort(); - await this.waitForIdle(); - await this.tools.resources.dispose(); - } - - private createStateView(): CuaAgentState { - const owner = this; - return Object.defineProperties({}, { - systemPrompt: { enumerable: true, get: () => owner.coreAgent.state.systemPrompt, set: (value: string) => { owner.coreAgent.state.systemPrompt = value; } }, - model: { enumerable: true, get: () => owner.getModel(), set: (value: CuaModelInput) => owner.setModel(value) }, - thinkingLevel: { enumerable: true, get: () => owner.coreAgent.state.thinkingLevel, set: (value: ThinkingLevel) => { owner.coreAgent.state.thinkingLevel = value; } }, - messages: { enumerable: true, get: () => owner.coreAgent.state.messages, set: (value: AgentMessage[]) => { owner.coreAgent.state.messages = value; } }, - tools: { enumerable: true, get: () => owner.coreAgent.state.tools.slice() }, - isStreaming: { enumerable: true, get: () => owner.coreAgent.state.isStreaming }, - streamingMessage: { enumerable: true, get: () => owner.coreAgent.state.streamingMessage }, - pendingToolCalls: { enumerable: true, get: () => owner.coreAgent.state.pendingToolCalls }, - errorMessage: { enumerable: true, get: () => owner.coreAgent.state.errorMessage }, - }) as CuaAgentState; - } -} - -/** Pi AgentHarness behavior through composition, without inherited active-tool APIs. */ -export class CuaAgentHarness< - TContext extends object | undefined = undefined, - TSkill extends Skill = Skill, - TPromptTemplate extends PromptTemplate = PromptTemplate, -> { - readonly models: Models; - private readonly coreHarness: AgentHarness>; - private readonly tools: CuaToolManager>; - private emptyResponseRecoveryAttempts = 0; - private hasPendingQueue = false; - private toolTurnFailed = false; - - constructor(options: CuaAgentHarnessOptions) { - const { - browser, - client, - model, - models, - tools: requestedTools, - onPayload, - emptyResponseRecovery, - toolResultImageReplayLimit, - responseThreading, - retry, - ...harnessOptions - } = options; - const recovery = resolveEmptyResponseRecovery(emptyResponseRecovery); - const imageReplayLimit = resolveToolResultImageReplayLimit(toolResultImageReplayLimit); - const useResponseThreading = resolveResponseThreading(responseThreading); - const resources = new CuaExecutionResources({ browser, client }); - const retrying = withProviderRetryModels(models ?? cuaModels(), resolveProviderRetryPolicy(retry)); - const manager = new CuaToolManager>(resources, model, requestedTools, (ref) => resolveModelFromCollection(ref, retrying)); - const catalogModels = withCatalogModels(retrying, manager, imageReplayLimit, useResponseThreading); - const materialized = manager.harnessTools(); - // A generic TContext leaves pi's conditional toolContext unverifiable - // here; the spread forwards the caller's toolContext verbatim. - const core = new AgentHarness>({ - ...harnessOptions, - model: manager.catalog.model, - models: catalogModels, - tools: materialized, - activeToolNames: materialized.map((tool) => tool.name), - } as AgentHarnessOptions>); - this.coreHarness = core; - this.tools = manager; - this.models = core.models; - - core.on("tool_result", (event) => hasExecutionError(event.details) ? { isError: true } : undefined); - core.on("tool_call", () => this.toolTurnFailed && turnFailureStopMessage(manager) - ? { block: true, reason: turnFailureStopMessage(manager) } - : undefined); - core.subscribe((event) => { - if (event.type === "message_end" && event.message.role === "assistant") this.toolTurnFailed = false; - else if (event.type === "tool_execution_end" && event.isError) this.toolTurnFailed = true; - else if (event.type === "queue_update") this.hasPendingQueue = event.steer.length > 0 || event.followUp.length > 0; - }); - core.on("before_agent_start", () => { - this.toolTurnFailed = false; - return undefined; - }); - if (onPayload) { - core.on("before_provider_payload", async ({ model: selectedModel, payload }) => ({ - payload: (await onPayload(payload, selectedModel)) ?? payload, - })); - } - if (recovery && recovery.maxAttempts > 0) { - core.on("before_agent_start", () => { - this.emptyResponseRecoveryAttempts = 0; - this.hasPendingQueue = false; - return undefined; - }); - core.subscribe(async (event, signal) => { - if (event.type !== "turn_end" || !isEmptyAssistantResponse(event.message)) return; - if (signal?.aborted || this.emptyResponseRecoveryAttempts >= recovery.maxAttempts || this.hasPendingQueue) return; - await core.followUp(recovery.followUp); - this.emptyResponseRecoveryAttempts += 1; - }); - } - } - - getTools(): readonly CuaHarnessTool[] { return this.tools.getTools(); } - - async setTools(tools: readonly CuaHarnessTool[]): Promise { - const previousModel = this.tools.catalog.model; - const previousTools = this.tools.harnessTools(); - const prepared = this.tools.prepareTools(tools); - const materialized = this.tools.harnessTools(prepared); - const transportChanged = modelTransportChanged(previousModel, prepared.catalog.model); - try { - if (transportChanged) await this.coreHarness.setModel(prepared.catalog.model); - await this.coreHarness.setTools(materialized, materialized.map((tool) => tool.name)); - } catch (error) { - if (transportChanged) await this.coreHarness.setModel(previousModel); - await this.coreHarness.setTools(previousTools, previousTools.map((tool) => tool.name)); - throw error; - } - this.tools.commit(prepared); - } - - getModel(): Model { return this.tools.catalog.model; } - - async setModel(model: CuaModelInput): Promise { - const previousModel = this.tools.catalog.model; - const previousTools = this.tools.harnessTools(); - const prepared = this.tools.prepareModel(model); - const materialized = this.tools.harnessTools(prepared); - try { - await this.coreHarness.setModel(prepared.catalog.model); - await this.coreHarness.setTools(materialized, materialized.map((tool) => tool.name)); - } catch (error) { - await this.coreHarness.setModel(previousModel); - await this.coreHarness.setTools(previousTools, previousTools.map((tool) => tool.name)); - throw error; - } - this.tools.commit(prepared); - } - - /** - * Select a model and its tool list in one compile. A model switch that also - * swaps interaction tools would otherwise stage through an intermediate - * catalog whose derived transport differs from both the old and the new one, - * recording a `model_change` for a transport nothing ever streamed with. - */ - async setModelAndTools(model: CuaModelInput, tools: readonly CuaHarnessTool[]): Promise { - const previousModel = this.tools.catalog.model; - const previousTools = this.tools.harnessTools(); - const prepared = this.tools.prepareModelAndTools(model, tools); - const materialized = this.tools.harnessTools(prepared); - try { - await this.coreHarness.setModel(prepared.catalog.model); - await this.coreHarness.setTools(materialized, materialized.map((tool) => tool.name)); - } catch (error) { - await this.coreHarness.setModel(previousModel); - await this.coreHarness.setTools(previousTools, previousTools.map((tool) => tool.name)); - throw error; - } - this.tools.commit(prepared); - } - - prompt(text: string, options?: { images?: ImageContent[] }) { return this.coreHarness.prompt(text, options); } - skill(name: string, additionalInstructions?: string) { return this.coreHarness.skill(name, additionalInstructions); } - promptFromTemplate(name: string, args?: string[]) { return this.coreHarness.promptFromTemplate(name, args); } - steer(text: string, options?: { images?: ImageContent[] }) { return this.coreHarness.steer(text, options); } - followUp(text: string, options?: { images?: ImageContent[] }) { return this.coreHarness.followUp(text, options); } - nextTurn(text: string, options?: { images?: ImageContent[] }) { return this.coreHarness.nextTurn(text, options); } - appendMessage(message: AgentMessage) { return this.coreHarness.appendMessage(message); } - compact(customInstructions?: string) { return this.coreHarness.compact(customInstructions); } - navigateTree(targetId: string, options?: Parameters[1]): Promise { return this.coreHarness.navigateTree(targetId, options); } - getThinkingLevel(): ThinkingLevel { return this.coreHarness.getThinkingLevel(); } - setThinkingLevel(level: ThinkingLevel): Promise { return this.coreHarness.setThinkingLevel(level); } - getSteeringMode(): QueueMode { return this.coreHarness.getSteeringMode(); } - setSteeringMode(mode: QueueMode): Promise { return this.coreHarness.setSteeringMode(mode); } - getFollowUpMode(): QueueMode { return this.coreHarness.getFollowUpMode(); } - setFollowUpMode(mode: QueueMode): Promise { return this.coreHarness.setFollowUpMode(mode); } - getResources(): AgentHarnessResources { return this.coreHarness.getResources(); } - setResources(resources: AgentHarnessResources): Promise { return this.coreHarness.setResources(resources); } - getStreamOptions(): AgentHarnessStreamOptions { return this.coreHarness.getStreamOptions(); } - setStreamOptions(options: AgentHarnessStreamOptions): Promise { return this.coreHarness.setStreamOptions(options); } - abort() { return this.coreHarness.abort(); } - waitForIdle(): Promise { return this.coreHarness.waitForIdle(); } - subscribe(listener: (event: AgentHarnessEvent, signal?: AbortSignal) => Promise | void) { return this.coreHarness.subscribe(listener); } - - on( - type: TType, - handler: (event: Extract) => Promise | AgentHarnessEventResultMap[TType], - ): () => void { - return this.coreHarness.on(type, handler); - } - - async dispose(): Promise { - await this.abort(); - await this.tools.resources.dispose(); - } -} diff --git a/packages/agent/src/attach.ts b/packages/agent/src/attach.ts index faefc04..e632115 100644 --- a/packages/agent/src/attach.ts +++ b/packages/agent/src/attach.ts @@ -63,17 +63,30 @@ export interface CuaCompiled { /** Same tools viewed as pi `AgentTool`s, for the low-level `Agent`. */ readonly agentTools: readonly AgentTool[]; /** - * A `Models` collection that adds what CUA owns per request: provider retry, - * required headers, the catalog's payload transforms, and the tool-result - * image bound. + * The handle's `Models` collection, adding what CUA owns per request: + * provider retry, required headers, the catalog's payload transforms, and + * the tool-result image bound. Shared by every compile from this handle, + * because pi fixes `models` at construction while those transforms are + * per-catalog; it serves whichever pair is currently active. */ readonly models: Models; /** - * Register the behaviors that are pi event handlers rather than constructor - * options: marking failed tool results, blocking a turn's remaining calls - * after one fails, and empty-response recovery. Returns an unsubscribe. + * Make this the handle's live pair on a harness: register the behaviors that + * are pi event handlers rather than constructor options (marking failed tool + * results, blocking a turn's remaining calls after one fails, empty-response + * recovery), and point `models` at this catalog. Returns a release that undoes + * both. Activating another pair releases this one. */ - install(harness: AgentHarness): () => void; + activate(harness: AgentHarness): () => void; + /** + * Swap a running harness onto this pair, then activate it. Model and tools + * move together because the transport is derived from both: setting one + * without the other leaves pi streaming a combination that never compiled. + * The model is only set when the derived transport actually moved, so a + * tools-only change records no model change. A failure restores the harness's + * previous pair before rethrowing, leaving the old pair active. + */ + apply(harness: AgentHarness>): Promise; } /** @@ -93,6 +106,8 @@ export interface CuaBrowserHandle { }): CuaCompiled; /** The shared execution pool, for callers that need it directly. */ readonly resources: CuaExecutionResources; + /** Same collection every {@link CuaCompiled.models} returns; see the note there. */ + readonly models: Models; dispose(): Promise; } @@ -113,8 +128,28 @@ export function attach(options: CuaAttachOptions): CuaBrowserHandle { const recovery = resolveEmptyResponseRecovery(options.emptyResponseRecovery); const retrying = withProviderRetryModels(options.models ?? cuaModels(), resolveProviderRetryPolicy(options.retry)); + // pi fixes `models` at construction, but the headers, payload transforms and + // incoming tool plan it applies are per-catalog. One collection per handle, + // reading whichever pair is live, is what lets a caller swap the pair on a + // running harness at all. + let live: CuaToolManager | undefined; + let lastCompiled: CuaToolManager | undefined; + let release: (() => void) | undefined; + const models = withCatalogModels( + retrying, + () => { + const manager = live ?? lastCompiled; + if (!manager) throw new Error("cua: compile a (model, tools) pair before streaming"); + return manager; + }, + imageReplayLimit, + useResponseThreading, + options.onPayload, + ); + return { resources, + models, dispose: () => resources.dispose(), compile(request: { model: CuaModelInput; @@ -126,18 +161,56 @@ export function attach(options: CuaAttachOptions): CuaBrowserHandle { request.tools, (ref) => resolveModelFromCollection(ref, retrying), ); - const models = withCatalogModels(retrying, manager, imageReplayLimit, useResponseThreading, options.onPayload); + lastCompiled = manager; + const model = manager.catalog.model; + const tools = manager.harnessTools() as readonly AgentHarnessTool[]; + const activate = (harness: AgentHarness): (() => void) => { + release?.(); + const uninstall = installCuaBehaviors(harness, manager, recovery); + live = manager; + // Identity-checked so calling a stale release cannot clear a newer + // activation: only the pair still live releases anything. + const releaseThis = (): void => { + uninstall(); + if (live === manager) live = undefined; + if (release === releaseThis) release = undefined; + }; + release = releaseThis; + return releaseThis; + }; return { - model: manager.catalog.model, - tools: manager.harnessTools() as readonly AgentHarnessTool[], + model, + tools, agentTools: manager.agentTools(), models, - install: (harness) => installCuaBehaviors(harness, manager, recovery), + activate, + async apply(harness) { + await applyCompiled(harness, model, tools); + activate(harness); + }, }; }, }; } +async function applyCompiled( + harness: AgentHarness>, + model: Model, + tools: readonly AgentHarnessTool[], +): Promise { + const previousModel = harness.getModel(); + const previousTools = harness.getTools(); + const transportChanged = modelTransportChanged(previousModel, model); + try { + if (transportChanged) await harness.setModel(model); + await harness.setTools([...tools], tools.map((tool) => tool.name)); + } catch (error) { + if (transportChanged) await harness.setModel(previousModel); + await harness.setTools(previousTools, previousTools.map((tool) => tool.name)); + throw error; + } +} + /** * Wire the pi event handlers CUA owns. Kept separate from `compile()` because * they are handlers on a constructed harness, not constructor options. @@ -179,25 +252,25 @@ export function installCuaBehaviors( }; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export const defaultCuaStream: StreamFn = (model, context, options) => cuaModels().streamSimple(model, context, options); -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function resolveModelFromCollection(ref: CuaModelRef, models: Models): Model { const { provider, model: id } = parseCuaModelRef(ref); return models.getModel(provider, id) ?? getCuaModel(ref); } /** Whether a tools-only recompile actually changed the model pi streams with, so `setTools()` only pushes `setModel()` (and its session/event side effects) when the derived transport moved. */ -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function modelTransportChanged(previous: Model, next: Model): boolean { return previous.provider !== next.provider || previous.id !== next.id || previous.api !== next.api; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function withCatalogModels( models: Models, - manager: CuaToolManager, + liveManager: () => CuaToolManager, imageReplayLimit: ToolResultImageReplayLimit, responseThreading: boolean, handleOnPayload?: SimpleStreamOptions["onPayload"], @@ -205,10 +278,10 @@ export function withCatalogModels( const contextFor = (context: Context) => projectModelContext( context, imageReplayLimit, - requiredImageToolNames(manager.catalog.incoming), + requiredImageToolNames(liveManager().catalog.incoming), ); const optionsFor = (options: T): T => { - const catalog = manager.catalog; + const catalog = liveManager().catalog; const callerOnPayload = options?.onPayload ?? handleOnPayload; return { ...options, @@ -239,7 +312,7 @@ export function withCatalogModels( }; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function resolveToolResultImageReplayLimit(limit: ToolResultImageReplayLimit | undefined): ToolResultImageReplayLimit { if (limit === undefined) return DEFAULT_TOOL_RESULT_IMAGE_REPLAY_LIMIT; if (limit !== false && (!Number.isFinite(limit) || !Number.isInteger(limit) || limit < 0)) { @@ -249,12 +322,12 @@ export function resolveToolResultImageReplayLimit(limit: ToolResultImageReplayLi } /** Native computer tool names whose screenshot history the provider protocol requires in full, regardless of the image replay limit. */ -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function requiredImageToolNames(incoming: CuaIncomingToolPlan): ReadonlySet { return new Set(incoming.openaiComputerName ? [incoming.openaiComputerName] : []); } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function projectToolResultImages( messages: TMessage[], limit: ToolResultImageReplayLimit, @@ -299,7 +372,7 @@ function projectModelContext( return messages === context.messages ? context : { ...context, messages }; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function resolveEmptyResponseRecovery(options: CuaEmptyResponseRecoveryOptions | undefined): CuaEmptyResponseRecoveryOptions | undefined { if (!options) return undefined; if (options.followUp.trim().length === 0) throw new Error("emptyResponseRecovery.followUp must not be blank"); @@ -307,23 +380,23 @@ export function resolveEmptyResponseRecovery(options: CuaEmptyResponseRecoveryOp return { followUp: options.followUp, maxAttempts: options.maxAttempts }; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function resolveResponseThreading(value: boolean | undefined): boolean { if (value !== undefined && typeof value !== "boolean") throw new TypeError("responseThreading must be a boolean"); return value ?? true; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function isEmptyAssistantResponse(message: AgentMessage): boolean { return message.role === "assistant" && message.stopReason === "stop" && message.content.length === 0; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function hasExecutionError(details: unknown): boolean { return Boolean(details && typeof details === "object" && (details as CuaExecutionDetails).isError === true); } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function turnFailureStopMessage(manager: CuaToolManager): string | undefined { for (const entry of manager.catalog.entries) { const execution = manager.specFor(entry.identity)?.execution; diff --git a/packages/agent/src/index.ts b/packages/agent/src/index.ts index d890993..aa78a66 100644 --- a/packages/agent/src/index.ts +++ b/packages/agent/src/index.ts @@ -31,7 +31,6 @@ export type { BrowserWaitForResult, BrowserWaitReason, } from "./translator/types"; -export { CuaAgent, CuaAgentHarness } from "./agent"; export { attach } from "./attach"; export type { CuaAttachOptions, @@ -41,5 +40,4 @@ export type { CuaModelInput, ToolResultImageReplayLimit, } from "./attach"; -export type { CuaAgentHarnessOptions, CuaAgentOptions, CuaAgentState } from "./agent"; export type { CuaRetryOptions } from "./provider-retry"; diff --git a/packages/agent/src/resources.ts b/packages/agent/src/resources.ts index 390a2bb..a6cc07f 100644 --- a/packages/agent/src/resources.ts +++ b/packages/agent/src/resources.ts @@ -17,7 +17,7 @@ export interface CuaExecutionDetails { readResults?: Array>; skippedActions?: number; failedActionIndex?: number; - /** Internal marker consumed by CuaAgent/CuaAgentHarness to set ToolResultMessage.isError. */ + /** Internal marker consumed by the behaviors `activate()` installs, to set ToolResultMessage.isError. */ isError?: boolean; result?: unknown; stdout?: string; diff --git a/packages/agent/src/tool-manager.ts b/packages/agent/src/tool-manager.ts index ec81e03..12d9fc1 100644 --- a/packages/agent/src/tool-manager.ts +++ b/packages/agent/src/tool-manager.ts @@ -1,12 +1,10 @@ -import { AsyncLocalStorage } from "node:async_hooks"; -import type { AgentHarnessTool, AgentTool, AgentToolResult } from "@earendil-works/pi-agent-core"; +import type { AgentHarnessTool, AgentTool } from "@earendil-works/pi-agent-core"; import type { Api, Model } from "@earendil-works/pi-ai"; import { callerToolIdentity, compileCuaToolCatalog, getCuaModel, isCuaToolSpec, - modelSupportsDeferredTools, type CuaCatalogToolInput, type CuaModelRef, type CuaToolCatalog, @@ -15,244 +13,84 @@ import { import { CuaExecutionResources } from "./resources"; /** - * Caller-owned tool for {@link CuaAgent}: a declarative CUA spec materialized - * by this package, or an already executable pi `AgentTool`. Defined here - * because cua-agent is the only package that holds both halves; cua-ai - * compiles declaration-only catalogs. + * Caller-owned tool: a declarative CUA spec materialized by this package, or an + * already executable pi `AgentTool`. Defined here because cua-agent is the only + * package that holds both halves; cua-ai compiles declaration-only catalogs. */ export type CuaAgentTool = CuaToolSpec | AgentTool; /** - * Caller-owned tool for {@link CuaAgentHarness}: a declarative CUA spec - * materialized by this package, or an executable pi `AgentHarnessTool` that - * receives the harness's tool context on every call. A plain `AgentTool` is - * assignable (it simply ignores the context), but the two APIs are kept - * distinct: `CuaAgent` takes `CuaAgentTool`, `CuaAgentHarness` takes this. + * Caller-owned tool for a harness: a declarative CUA spec, or an executable pi + * `AgentHarnessTool` that receives the harness's tool context on every call. A + * plain `AgentTool` is assignable, since it simply ignores the context. */ export type CuaHarnessTool = CuaToolSpec | AgentHarnessTool; /** - * One atomically committable tools state. `requested` is the sole caller-owned - * source of truth; `catalog` is its pure declarative projection; `tools` and - * `harnessTools` are the wrapped executables joined back by identity after - * compilation, viewed as pi `AgentTool`s (for `CuaAgent`) or as - * context-delivering `AgentHarnessTool`s (for `CuaAgentHarness`). + * One compiled (model, tools) pair: the caller's list joined back to its + * declarative catalog by compiled identity, with each spec materialized against + * the shared browser resources. + * + * Immutable by construction. Changing the model or the tool list compiles a new + * one, which is what lets a caller hand pi a fresh pair instead of mutating a + * live catalog underneath it. */ -export interface PreparedCuaTools = CuaAgentTool> { - readonly requested: readonly TRequested[]; - /** - * The model input this state was compiled from, before catalog compilation - * derived a tool-selection-dependent transport onto it. A tools-only - * recompile reuses this (not `catalog.model`, which may carry a transport - * the new tool selection no longer requires) so the derivation re-runs from - * a clean model every time. - */ - readonly modelSelection: CuaModelRef | Model; - readonly catalog: CuaToolCatalog; - readonly tools: readonly AgentTool[]; - readonly harnessTools: readonly AgentHarnessTool[]; - /** Identity → CUA spec, for execution metadata (e.g. stop-on-failure policy). */ - readonly specs: ReadonlyMap; - /** Declaration fingerprint composed with implementation identity, in entry order. */ - readonly fingerprints: readonly string[]; -} - -interface ToolExecutionScope { - readonly toolName: string; - readonly executionMode: AgentTool["executionMode"]; - readonly baseline: PreparedCuaTools; -} - -/** - * Implementation identity, owned by cua-agent: keyed on a caller tool's - * `execute` function (a new wrapper reusing the same function retains - * identity) and on the spec object itself (a freshly created spec is - * conservatively a replacement, the same object stays stable). - */ -const implementationIds = new WeakMap(); -let nextImplementationId = 1; - -function implementationId(key: object): number { - let id = implementationIds.get(key); - if (id === undefined) { - id = nextImplementationId++; - implementationIds.set(key, id); - } - return id; -} - -/** Owns the caller's requested list and its compiled catalog while browser resources live independently. */ export class CuaToolManager = CuaAgentTool> { - private readonly execution = new AsyncLocalStorage(); - private current: PreparedCuaTools; + readonly catalog: CuaToolCatalog; + private readonly executables: readonly (AgentTool | AgentHarnessTool)[]; + private readonly specs: ReadonlyMap; constructor( readonly resources: CuaExecutionResources, model: CuaModelRef | Model, requestedTools: readonly TRequested[], - private readonly resolveModel: (model: CuaModelRef) => Model = getCuaModel, + resolveModel: (model: CuaModelRef) => Model = getCuaModel, ) { - this.current = this.prepare(model, requestedTools); - } - - get catalog(): CuaToolCatalog { - return this.current.catalog; - } - - getTools(): TRequested[] { - return [...this.current.requested]; - } - - /** Wrapped pi `AgentTool` view of a prepared (or the committed) state, in entry order. */ - agentTools(prepared: PreparedCuaTools = this.current): AgentTool[] { - return [...prepared.tools]; - } - - /** Wrapped pi `AgentHarnessTool` view of a prepared (or the committed) state, in entry order. */ - harnessTools(prepared: PreparedCuaTools = this.current): AgentHarnessTool[] { - return [...prepared.harnessTools]; - } - - /** Execution metadata for one committed catalog identity. */ - specFor(identity: string): CuaToolSpec | undefined { - return this.current.specs.get(identity); - } - - prepareTools(tools: readonly TRequested[]): PreparedCuaTools { - this.assertMutationScope("setTools"); - return this.prepare(this.current.modelSelection, tools); - } - - prepareModel(model: CuaModelRef | Model): PreparedCuaTools { - this.assertMutationScope("setModel"); - return this.prepare(model, this.current.requested); - } - - prepareModelAndTools(model: CuaModelRef | Model, tools: readonly TRequested[]): PreparedCuaTools { - this.assertMutationScope("setModelAndTools"); - return this.prepare(model, tools); - } - - commit(prepared: PreparedCuaTools): void { - this.current = prepared; - } - - /** - * Compile and materialize without touching committed state. Ordinary - * AgentTools are projected into fresh declaration-only objects before - * compilation, then joined back strictly by compiled identity — never by - * position. Any failure leaves the committed state untouched. - */ - private prepare(model: CuaModelRef | Model, tools: readonly TRequested[]): PreparedCuaTools { - const requested = Object.freeze([...tools]); const inputs: CuaCatalogToolInput[] = []; const executables = new Map(); - const implementations = new Map(); const specs = new Map(); - for (const tool of requested) { + for (const tool of requestedTools) { if (isCuaToolSpec(tool)) { inputs.push(tool); executables.set(tool.identity, tool); - implementations.set(tool.identity, tool); specs.set(tool.identity, tool); } else { - const identity = callerToolIdentity(tool.name); // Fresh declaration-only projection: execute, label, prepareArguments, // and executionMode never cross into cua-ai. inputs.push({ name: tool.name, description: tool.description, parameters: tool.parameters }); - executables.set(identity, tool); - implementations.set(identity, tool.execute); + executables.set(callerToolIdentity(tool.name), tool); } } - const catalog = compileCuaToolCatalog({ - model: typeof model === "string" ? this.resolveModel(model) : model, + this.catalog = compileCuaToolCatalog({ + model: typeof model === "string" ? resolveModel(model) : model, requestedTools: inputs, }); - const fingerprints: string[] = []; - const joined: Array> = catalog.entries.map((entry) => { + // Joined strictly by compiled identity, never by position. + this.executables = this.catalog.entries.map((entry) => { const executable = executables.get(entry.identity); - const implementation = implementations.get(entry.identity); - if (!executable || !implementation) { - throw new Error(`compiled catalog entry "${entry.identity}" has no matching requested tool`); - } + if (!executable) throw new Error(`compiled catalog entry "${entry.identity}" has no matching requested tool`); executables.delete(entry.identity); - fingerprints.push(`${entry.fingerprint}#impl-${implementationId(implementation)}`); - return isCuaToolSpec(executable) ? this.resources.materialize(executable) : (executable as AgentHarnessTool); + return isCuaToolSpec(executable) ? resources.materialize(executable) : (executable as AgentHarnessTool); }); if (executables.size > 0) { throw new Error(`requested tool(s) ${[...executables.keys()].join(", ")} missing from the compiled catalog`); } - - return Object.freeze({ - requested, - modelSelection: model, - catalog, - tools: Object.freeze(joined.map((tool) => this.wrapAgentExecutable(tool as AgentTool))), - harnessTools: Object.freeze(joined.map((tool) => this.wrapHarnessExecutable(tool))), - specs, - fingerprints: Object.freeze(fingerprints), - }); - } - - /** Low-level `AgentTool` view for {@link CuaAgent}; caller tools there never declare a harness context. */ - private wrapAgentExecutable(tool: AgentTool): AgentTool { - return { - ...tool, - execute: (toolCallId, input, signal, onUpdate) => - this.executeWithScope(tool, () => tool.execute(toolCallId, input, signal, onUpdate)), - }; + this.specs = specs; } - /** Context-delivering `AgentHarnessTool` view for {@link CuaAgentHarness}. */ - private wrapHarnessExecutable(tool: AgentHarnessTool): AgentHarnessTool { - return { - ...tool, - execute: (toolCallId, params, signal, onUpdate, context) => - this.executeWithScope(tool, () => tool.execute(toolCallId, params, signal, onUpdate, context)), - }; + /** Pi `AgentTool` view, in catalog entry order. */ + agentTools(): AgentTool[] { + return [...(this.executables as readonly AgentTool[])]; } - private async executeWithScope( - tool: { readonly name: string; readonly executionMode?: AgentTool["executionMode"] }, - call: () => Promise>, - ): Promise> { - const scope: ToolExecutionScope = { - toolName: tool.name, - executionMode: tool.executionMode, - baseline: this.current, - }; - return this.execution.run(scope, async () => { - const result = await call(); - return mergeAddedToolNames(result, cachePreservingAdditions(scope.baseline, this.current) ?? []); - }); + /** Context-delivering pi `AgentHarnessTool` view, in catalog entry order. */ + harnessTools(): AgentHarnessTool[] { + return [...(this.executables as readonly AgentHarnessTool[])]; } - private assertMutationScope(api: "setTools" | "setModel" | "setModelAndTools"): void { - const scope = this.execution.getStore(); - if (scope && scope.executionMode !== "sequential") { - throw new Error(`tool "${scope.toolName}" must declare executionMode: "sequential" before calling ${api}() during execution`); - } - } -} - -function cachePreservingAdditions(previous: PreparedCuaTools, next: PreparedCuaTools): string[] | undefined { - const previousModel = previous.catalog.model; - const nextModel = next.catalog.model; - if (previousModel.provider !== nextModel.provider || previousModel.id !== nextModel.id || previousModel.api !== nextModel.api) return undefined; - if (next.fingerprints.length <= previous.fingerprints.length) return undefined; - for (let index = 0; index < previous.fingerprints.length; index += 1) { - if (previous.fingerprints[index] !== next.fingerprints[index]) return undefined; + /** Execution metadata for one catalog identity. */ + specFor(identity: string): CuaToolSpec | undefined { + return this.specs.get(identity); } - const added = next.catalog.entries.slice(previous.fingerprints.length); - if (!modelSupportsDeferredTools(nextModel) || added.some((entry) => entry.dynamicLoading !== "eligible")) return undefined; - return added.map((entry) => entry.name); -} - -function mergeAddedToolNames(result: AgentToolResult, names: readonly string[]): AgentToolResult { - if (names.length === 0) return result; - return { - ...result, - addedToolNames: [...new Set([...(result.addedToolNames ?? []), ...names])], - }; } diff --git a/packages/agent/test/agent.test.ts b/packages/agent/test/agent.test.ts deleted file mode 100644 index 742ecf5..0000000 --- a/packages/agent/test/agent.test.ts +++ /dev/null @@ -1,634 +0,0 @@ -import { describe, expect, it, vi } from "vitest"; -import { - createAssistantMessageEventStream, - createCuaModels, - getCuaModel, - cua, - GOOGLE_CUA_INTERACTIONS_API, - isCuaToolSpec, - type AssistantMessage, - type Context, - type Model, -} from "@onkernel/cua-ai"; -import type Kernel from "@onkernel/sdk"; -import { - Agent, - AgentHarness, - CuaAgent, - CuaAgentHarness, - InMemorySessionRepo, - type AgentMessage, - type AgentTool, - type KernelBrowser, - type StreamFn, -} from "../src/index"; - -const browser = { session_id: "browser_123", viewport: { width: 1440, height: 900 } } as KernelBrowser; -const client = {} as Kernel; - -function assistant(model: Model, content: AssistantMessage["content"] = [], stopReason: AssistantMessage["stopReason"] = "stop"): AssistantMessage { - return { - role: "assistant", - content, - api: model.api, - provider: model.provider, - model: model.id, - usage: { - input: 0, - output: 0, - cacheRead: 0, - cacheWrite: 0, - totalTokens: 0, - cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 }, - }, - stopReason, - timestamp: Date.now(), - }; -} - -function scriptedStream( - turns: Array<(model: Model) => AssistantMessage>, - contexts: Context[] = [], -): StreamFn { - let call = 0; - return (model, context) => { - contexts.push({ ...context, messages: structuredClone(context.messages), tools: context.tools?.slice() }); - const stream = createAssistantMessageEventStream(); - const message = turns[call++]?.(model) ?? assistant(model); - stream.push({ type: "start", partial: message }); - stream.push({ type: "done", reason: message.stopReason as "stop" | "length" | "toolUse", message }); - stream.end(message); - return stream; - }; -} - -function callerTool(name: string, execute?: AgentTool["execute"], executionMode?: AgentTool["executionMode"]): AgentTool { - return { - name, - label: name, - description: `${name} tool`, - parameters: { type: "object", properties: {}, additionalProperties: false } as never, - ...(executionMode ? { executionMode } : {}), - execute: execute ?? (async () => ({ content: [{ type: "text", text: "ok" }], details: {} })), - }; -} - -async function harnessServices() { - const repo = new InMemorySessionRepo(); - return { - session: await repo.create(), - }; -} - -function modelsFromStream(streamFn: StreamFn, provider = "openai") { - const models = createCuaModels(); - models.setProvider({ - id: provider, - name: "scripted", - auth: { apiKey: { name: "test", resolve: async () => ({ auth: { apiKey: "test" } }) } }, - getModels: () => [], - stream: streamFn, - streamSimple: streamFn, - } as never); - return models; -} - -describe("CuaAgent explicit tools", () => { - it("uses composition and accepts tools: [] without a prompt or implicit tool", () => { - const agent = new CuaAgent({ browser, client, tools: [], initialState: { model: "openai:gpt-5.5" } }); - expect(agent).not.toBeInstanceOf(Agent); - expect(agent.getTools()).toEqual([]); - expect("inspectTools" in agent).toBe(false); - expect(agent.state.tools).toEqual([]); - expect(agent.state.systemPrompt).toBe(""); - expect("setMode" in agent).toBe(false); - expect("getMode" in agent).toBe(false); - }); - - it("retains protocol-required OpenAI native computer screenshot results outside the image replay limit", async () => { - const contexts: Context[] = []; - const model = getCuaModel("openai:gpt-5.5"); - const nativeComputer = cua.providers.openai.tools.computer(); - const ordinaryTool = callerTool("ordinary"); - const image = { type: "image" as const, data: "c2NyZWVuc2hvdA==", mimeType: "image/png" }; - const messages: AgentMessage[] = [ - assistant(model, [{ type: "toolCall", id: "native-shot", name: "computer", arguments: { action: { type: "screenshot" } } }], "toolUse"), - { role: "toolResult", toolCallId: "native-shot", toolName: "computer", content: [image], isError: false, timestamp: 1 }, - assistant(model, [{ type: "toolCall", id: "ordinary-shot", name: "ordinary", arguments: {} }], "toolUse"), - { role: "toolResult", toolCallId: "ordinary-shot", toolName: "ordinary", content: [image], isError: false, timestamp: 2 }, - ]; - const agent = new CuaAgent({ - browser, - client, - tools: [nativeComputer, ordinaryTool], - streamFn: scriptedStream([(selectedModel) => assistant(selectedModel)], contexts), - toolResultImageReplayLimit: 0, - initialState: { model, messages }, - }); - - await agent.prompt("continue"); - - const results = contexts[0]!.messages.filter((message) => message.role === "toolResult"); - expect(results[0]!.content).toEqual([image]); - expect(results[1]!.content).toEqual([{ type: "text", text: "[stale tool-result images omitted]" }]); - }); - - it("installs exact native-only, native-plus-CUA, Playwright-only, and browser-act-only catalogs", () => { - const custom = callerTool("customer_lookup"); - const cases = [ - { - model: "anthropic:claude-opus-5" as const, - tools: [cua.providers.anthropic.tools.browser(), custom], - names: ["browser", "customer_lookup"], - }, - { - model: "openai:gpt-5.5" as const, - tools: [cua.providers.openai.tools.computer(), cua.tools.browser.snapshot(), cua.tools.browser.act()], - names: ["computer", "browser_snapshot", "browser_act"], - }, - { model: "openai:gpt-5.5" as const, tools: [cua.tools.playwright()], names: ["playwright_execute"] }, - { model: "openai:gpt-5.5" as const, tools: [cua.tools.browser.act()], names: ["browser_act"] }, - ]; - for (const entry of cases) { - const agent = new CuaAgent({ browser, client, tools: entry.tools, initialState: { model: entry.model } }); - expect(agent.state.tools.map((tool) => tool.name)).toEqual(entry.names); - expect(agent.getTools()).toEqual(entry.tools); - } - }); - - it("returns a copy of the exact requested specs", () => { - const requested = [cua.tools.browser.snapshot(), callerTool("customer_lookup")]; - const agent = new CuaAgent({ browser, client, tools: requested, initialState: { model: "anthropic:claude-opus-5" } }); - expect(agent.getTools()).toEqual(requested); - expect(agent.getTools()).not.toBe(requested); - expect(agent.state.tools.map((tool) => tool.name)).toEqual(["browser_snapshot", "customer_lookup"]); - }); - - it("keeps the caller system prompt stable across setTools", () => { - const agent = new CuaAgent({ - browser, - client, - tools: [], - initialState: { model: "openai:gpt-5.5", systemPrompt: "caller-owned" }, - }); - agent.setTools([cua.tools.playwright()]); - expect(agent.state.systemPrompt).toBe("caller-owned"); - expect(agent.getTools().map((tool) => tool.name)).toEqual(["playwright_execute"]); - }); - - it("marks a sequential in-tool prefix addition for deferred loading", async () => { - const contexts: Context[] = []; - let agent!: CuaAgent; - const added = callerTool("added"); - const loader = callerTool("loader", async () => { - agent.setTools([...agent.getTools(), added]); - return { content: [{ type: "text", text: "loaded" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [loader], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load-1", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model, [{ type: "text", text: "done" }]), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - - await agent.prompt("load it"); - - expect(agent.getTools()).toEqual([loader, added]); - expect(contexts[1]?.tools?.map((tool) => tool.name)).toEqual(["loader", "added"]); - expect(contexts[1]?.messages.find((message) => message.role === "toolResult")).toMatchObject({ addedToolNames: ["added"] }); - }); - - it("computes deferred additions from the final in-tool catalog", async () => { - const contexts: Context[] = []; - let agent!: CuaAgent; - const added = callerTool("temporary"); - const loader = callerTool("loader", async () => { - agent.setTools([...agent.getTools(), added]); - agent.setTools([loader]); - return { content: [{ type: "text", text: "unchanged" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [loader], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("load"); - expect(contexts[1]?.messages.find((message) => message.role === "toolResult")).not.toHaveProperty("addedToolNames"); - }); - - it("uses eager fallback for removals and replacements", async () => { - const run = async (replacement: "remove" | "replace") => { - const contexts: Context[] = []; - let agent!: CuaAgent; - const loader = callerTool("loader", async () => { - agent.setTools(replacement === "remove" ? [] : [callerTool("loader")]); - return { content: [{ type: "text", text: replacement }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [loader], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "change", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("change"); - return contexts[1]?.messages.find((message) => message.role === "toolResult"); - }; - expect(await run("remove")).not.toHaveProperty("addedToolNames"); - expect(await run("replace")).not.toHaveProperty("addedToolNames"); - }); - - it("keeps deferred additions when a replacement wrapper reuses the same execute function", async () => { - const contexts: Context[] = []; - let agent!: CuaAgent; - const sharedExecute: AgentTool["execute"] = async () => ({ content: [{ type: "text", text: "ok" }], details: {} }); - const original = callerTool("original", sharedExecute); - const added = callerTool("added"); - const loader = callerTool("loader", async () => { - const rewrapped = { ...original }; - expect(rewrapped).not.toBe(original); - agent.setTools([rewrapped, loader, added]); - return { content: [{ type: "text", text: "loaded" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [original, loader], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("load"); - expect(contexts[1]?.messages.find((message) => message.role === "toolResult")).toMatchObject({ addedToolNames: ["added"] }); - }); - - it("installs and invokes a replacement executor with an identical name and schema", async () => { - const contexts: Context[] = []; - const calls: string[] = []; - let agent!: CuaAgent; - const workerV1 = callerTool("worker", async () => { - calls.push("v1"); - return { content: [{ type: "text", text: "v1" }], details: {} }; - }); - const loader = callerTool("loader", async () => { - const workerV2 = callerTool("worker", async () => { - calls.push("v2"); - return { content: [{ type: "text", text: "v2" }], details: {} }; - }); - agent.setTools([loader, workerV2]); - return { content: [{ type: "text", text: "replaced" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [loader, workerV1], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model, [{ type: "toolCall", id: "work", name: "worker", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("replace then work"); - expect(calls).toEqual(["v2"]); - expect(contexts[1]?.messages.find((message) => message.role === "toolResult")).not.toHaveProperty("addedToolNames"); - }); - - it("treats a freshly created spec object as a replacement but the same object as stable", async () => { - const run = async (reuseSpec: boolean) => { - const contexts: Context[] = []; - let agent!: CuaAgent; - const spec = cua.tools.browser.snapshot(); - const added = callerTool("added"); - const loader = callerTool("loader", async () => { - agent.setTools([reuseSpec ? spec : cua.tools.browser.snapshot(), loader, added]); - return { content: [{ type: "text", text: "loaded" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [spec, loader], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("load"); - return contexts[1]?.messages.find((message) => message.role === "toolResult"); - }; - expect(await run(true)).toMatchObject({ addedToolNames: ["added"] }); - expect(await run(false)).not.toHaveProperty("addedToolNames"); - }); - - it("leaves model, requested tools, and installed executables untouched when setTools fails", () => { - const spec = cua.tools.browser.snapshot(); - const keep = callerTool("keep"); - const agent = new CuaAgent({ browser, client, tools: [spec, keep], initialState: { model: "openai:gpt-5.5" } }); - const installed = agent.state.tools; - expect(() => agent.setTools([cua.providers.anthropic.tools.browser()])).toThrow(/requires a anthropic model/); - expect(() => agent.setTools([callerTool("keep"), callerTool("keep")])).toThrow(/caller\.keep/); - expect(agent.getModel().id).toBe("gpt-5.5"); - expect(agent.getTools()).toEqual([spec, keep]); - expect(agent.state.tools).toEqual(installed); - agent.state.tools.forEach((tool, index) => expect(tool).toBe(installed[index])); - }); - - it("leaves model, requested tools, and installed executables untouched when setModel fails", () => { - const spec = cua.providers.anthropic.tools.browser(); - const agent = new CuaAgent({ browser, client, tools: [spec], initialState: { model: "anthropic:claude-opus-5" } }); - const installed = agent.state.tools; - expect(() => agent.setModel("openai:gpt-5.5")).toThrow(/requires a anthropic model/); - expect(agent.getModel().provider).toBe("anthropic"); - expect(agent.getTools()).toEqual([spec]); - agent.state.tools.forEach((tool, index) => expect(tool).toBe(installed[index])); - }); - - it("rejects in-tool mutation from a non-sequential caller tool", async () => { - let agent!: CuaAgent; - const loader = callerTool("loader", async () => { - agent.setTools([cua.tools.playwright()]); - return { content: [{ type: "text", text: "unexpected" }], details: {} }; - }); - agent = new CuaAgent({ - browser, - client, - tools: [loader], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model), - ]), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("load"); - const result = agent.state.messages.find((message) => message.role === "toolResult"); - expect(result).toMatchObject({ isError: true }); - expect(result?.content).toEqual([expect.objectContaining({ text: expect.stringContaining('executionMode: "sequential"') })]); - }); - - it("rejects in-tool model switching from a non-sequential caller tool", async () => { - let agent!: CuaAgent; - const switcher = callerTool("switcher", async () => { - agent.setModel("anthropic:claude-opus-5"); - return { content: [{ type: "text", text: "unexpected" }], details: {} }; - }); - agent = new CuaAgent({ - browser, - client, - tools: [switcher], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "switch", name: "switcher", arguments: {} }], "toolUse"), - (model) => assistant(model), - ]), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("switch"); - const result = agent.state.messages.find((message) => message.role === "toolResult"); - expect(result).toMatchObject({ isError: true }); - expect(result?.content).toEqual([expect.objectContaining({ text: expect.stringContaining('before calling setModel()') })]); - expect(agent.getModel().provider).toBe("openai"); - }); - - it("allows in-tool model switching from a sequential caller tool", async () => { - const contexts: Context[] = []; - let agent!: CuaAgent; - const switcher = callerTool("switcher", async () => { - agent.setModel("anthropic:claude-opus-5"); - return { content: [{ type: "text", text: "switched" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [switcher], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "switch", name: "switcher", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("switch"); - const result = agent.state.messages.find((message) => message.role === "toolResult"); - expect(result).not.toMatchObject({ isError: true }); - expect(agent.getModel().provider).toBe("anthropic"); - expect(agent.getTools()).toEqual([switcher]); - expect(contexts[1]?.messages.find((message) => message.role === "toolResult")).not.toHaveProperty("addedToolNames"); - }); - - it("streams with the transport setTools derives, not just the catalog it compiles", async () => { - const streamedApis: string[] = []; - const agent = new CuaAgent({ - browser, - client, - tools: [cua.tools.browser.snapshot()], - streamFn: (model, context, options) => { - streamedApis.push(model.api); - return scriptedStream([(selectedModel) => assistant(selectedModel)])(model, context, options); - }, - initialState: { model: "google:gemini-3.6-flash" }, - }); - expect(agent.getModel().api).toBe("google-generative-ai"); - - agent.setTools(cua.providers.google.toolsets.browser()); - expect(agent.getModel().api).toBe(GOOGLE_CUA_INTERACTIONS_API); - - await agent.prompt("go"); - expect(streamedApis).toEqual([GOOGLE_CUA_INTERACTIONS_API]); - }); -}); - -describe("CuaAgentHarness explicit tools", () => { - it("resolves refs from supplied models for construction and setModel", async () => { - const models = createCuaModels(); - const openai = models.getProvider("openai")!; - const first = { ...getCuaModel("openai:gpt-5.5"), baseUrl: "https://first.example" }; - const second = { ...getCuaModel("openai:gpt-5.6-sol"), baseUrl: "https://second.example" }; - models.setProvider({ ...openai, getModels: () => [first, second] }); - const services = await harnessServices(); - const harness = new CuaAgentHarness({ ...services, browser, client, models, model: "openai:gpt-5.5", tools: [] }); - expect(harness.getModel()).toBe(first); - await harness.setModel("openai:gpt-5.6-sol"); - expect(harness.getModel()).toBe(second); - - // A ref the supplied collection does not carry falls back to the registry - // rather than being refused: the provider decides what exists. - const fallback = new CuaAgentHarness({ ...services, browser, client, models, model: "openai:gpt-5.4", tools: [] }); - expect(fallback.getModel().id).toBe("gpt-5.4"); - }); - - it("uses composition, hides active-tool APIs, and supports an empty catalog", async () => { - const harness = new CuaAgentHarness({ ...(await harnessServices()), browser, client, model: "openai:gpt-5.5", tools: [] }); - expect(harness).not.toBeInstanceOf(AgentHarness); - expect(harness.getTools()).toEqual([]); - expect("inspectTools" in harness).toBe(false); - expect("getActiveTools" in harness).toBe(false); - expect("setActiveTools" in harness).toBe(false); - expect("setMode" in harness).toBe(false); - }); - - it("preserves requested tools on compatible model changes and rejects incompatible native tools", async () => { - const custom = callerTool("custom"); - const harness = new CuaAgentHarness({ ...(await harnessServices()), browser, client, model: "openai:gpt-5.5", tools: [custom] }); - await harness.setModel("anthropic:claude-opus-5"); - expect(harness.getTools()).toEqual([custom]); - expect(harness.getModel().provider).toBe("anthropic"); - - await harness.setTools([cua.providers.anthropic.tools.browser()]); - await expect(harness.setModel("openai:gpt-5.5")).rejects.toThrow(/requires a anthropic model/); - expect(harness.getModel().provider).toBe("anthropic"); - const installed = harness.getTools()[0]; - expect(installed && isCuaToolSpec(installed) ? installed.identity : undefined).toBe("provider.anthropic.native.browser.20260701"); - }); - - it("keeps the harness catalog and executors unchanged when setTools fails", async () => { - const keep = callerTool("keep"); - const harness = new CuaAgentHarness({ ...(await harnessServices()), browser, client, model: "openai:gpt-5.5", tools: [keep] }); - await expect(harness.setTools([cua.providers.anthropic.tools.browser()])).rejects.toThrow(/requires a anthropic model/); - expect(harness.getModel().id).toBe("gpt-5.5"); - expect(harness.getTools()).toEqual([keep]); - }); - - it("persists exact tool-name changes and anchors additive in-tool loading", async () => { - const contexts: Context[] = []; - const services = await harnessServices(); - let harness!: CuaAgentHarness; - const added = callerTool("added"); - const loader = callerTool("loader", async () => { - await harness.setTools([...harness.getTools(), added]); - return { content: [{ type: "text", text: "loaded" }], details: {} }; - }, "sequential"); - harness = new CuaAgentHarness({ - ...services, - browser, - client, - model: "openai:gpt-5.5", - models: modelsFromStream(scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts)), - tools: [loader], - systemPrompt: "stable", - }); - - await harness.prompt("load"); - - expect(contexts[0]?.systemPrompt).toBe("stable"); - expect(contexts[1]?.systemPrompt).toBe("stable"); - expect(contexts[1]?.messages.find((message) => message.role === "toolResult")).toMatchObject({ addedToolNames: ["added"] }); - const changes = (await services.session.getBranch()).filter((entry) => entry.type === "active_tools_change"); - expect(changes.at(-1)).toMatchObject({ activeToolNames: ["loader", "added"] }); - }); - - it("does not emit an artificial tool result for idle additions", async () => { - const contexts: Context[] = []; - const harness = new CuaAgentHarness({ - ...(await harnessServices()), - browser, - client, - model: "openai:gpt-5.5", - models: modelsFromStream(scriptedStream([(model) => assistant(model)], contexts)), - tools: [], - }); - await harness.setTools([callerTool("added")]); - await harness.prompt("next"); - expect(contexts[0]?.tools?.map((tool) => tool.name)).toEqual(["added"]); - expect(contexts[0]?.messages.some((message) => message.role === "toolResult")).toBe(false); - }); - - it("clears failed-turn state before the next prompt", async () => { - let successfulCalls = 0; - const failing = callerTool("failing", async () => { - throw new Error("expected failure"); - }); - const succeeding = callerTool("succeeding", async () => { - successfulCalls += 1; - return { content: [{ type: "text", text: "ok" }], details: {} }; - }); - const harness = new CuaAgentHarness({ - ...(await harnessServices()), - browser, - client, - model: "anthropic:claude-opus-5", - models: modelsFromStream(scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "fail", name: "failing", arguments: {} }], "toolUse"), - (model) => assistant(model, [{ type: "toolCall", id: "succeed", name: "succeeding", arguments: {} }], "toolUse"), - (model) => assistant(model, [{ type: "text", text: "done" }]), - ]), "anthropic"), - tools: [cua.providers.anthropic.tools.browser(), failing, succeeding], - }); - harness.on("tool_result", (event) => event.toolName === "failing" ? { terminate: true } : undefined); - - await harness.prompt("fail once"); - await harness.prompt("try again"); - - expect(successfulCalls).toBe(1); - }); - - it("streams with the transport setTools derives, not just the catalog it compiles", async () => { - const streamedApis: string[] = []; - const script = scriptedStream([(selectedModel) => assistant(selectedModel)]); - const harness = new CuaAgentHarness({ - ...(await harnessServices()), - browser, - client, - model: "google:gemini-3.6-flash", - models: modelsFromStream((model, context, options) => { - streamedApis.push(model.api); - return script(model, context, options); - }, "google"), - tools: [cua.tools.browser.snapshot()], - }); - expect(harness.getModel().api).toBe("google-generative-ai"); - - await harness.setTools(cua.providers.google.toolsets.browser()); - expect(harness.getModel().api).toBe(GOOGLE_CUA_INTERACTIONS_API); - - await harness.prompt("go"); - expect(streamedApis).toEqual([GOOGLE_CUA_INTERACTIONS_API]); - }); - - it("does not record a model change for a setTools() call that leaves the derived transport unchanged", async () => { - const services = await harnessServices(); - const harness = new CuaAgentHarness({ - ...services, - browser, - client, - model: "openai:gpt-5.5", - tools: [callerTool("first")], - }); - await harness.setTools([callerTool("second")]); - const modelChanges = (await services.session.getBranch()).filter((entry) => entry.type === "model_change"); - expect(modelChanges).toEqual([]); - }); - - it("records one model change for a switch that changes both the model and its derived transport", async () => { - const services = await harnessServices(); - const harness = new CuaAgentHarness({ - ...services, - browser, - client, - model: "google:gemini-3.6-flash", - tools: cua.providers.google.toolsets.browser(), - }); - expect(harness.getModel().api).toBe(GOOGLE_CUA_INTERACTIONS_API); - - await harness.setModelAndTools("openai:gpt-5.5", [cua.tools.browser.snapshot()]); - expect(harness.getModel().api).toBe("openai-responses"); - - const modelChanges = (await services.session.getBranch()).filter((entry) => entry.type === "model_change"); - expect(modelChanges).toHaveLength(1); - }); -}); diff --git a/packages/agent/test/attach-session.test.ts b/packages/agent/test/attach-session.test.ts new file mode 100644 index 0000000..2f8da1f --- /dev/null +++ b/packages/agent/test/attach-session.test.ts @@ -0,0 +1,313 @@ +import { describe, expect, it } from "vitest"; +import { + createAssistantMessageEventStream, + createCuaModels, + cua, + getCuaModel, + GOOGLE_CUA_INTERACTIONS_API, + type AssistantMessage, + type Context, + type Model, + type Models, +} from "@onkernel/cua-ai"; +import type Kernel from "@onkernel/sdk"; +import { + Agent, + AgentHarness, + attach, + InMemorySessionRepo, + type AgentMessage, + type AgentTool, + type CuaHarnessTool, + type CuaModelInput, + type KernelBrowser, + type Session, + type StreamFn, +} from "../src/index"; + +const browser = { session_id: "browser_123", viewport: { width: 1440, height: 900 } } as KernelBrowser; +const client = {} as Kernel; + +function assistant(model: Model, content: AssistantMessage["content"] = [], stopReason: AssistantMessage["stopReason"] = "stop"): AssistantMessage { + return { + role: "assistant", + content, + api: model.api, + provider: model.provider, + model: model.id, + usage: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, totalTokens: 0, cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 } }, + stopReason, + timestamp: Date.now(), + }; +} + +function scriptedStream(turns: Array<(model: Model) => AssistantMessage>, contexts: Context[] = []): StreamFn { + let call = 0; + return (model, context) => { + contexts.push({ ...context, messages: structuredClone(context.messages), tools: context.tools?.slice() }); + const stream = createAssistantMessageEventStream(); + const message = turns[call++]?.(model) ?? assistant(model); + stream.push({ type: "start", partial: message }); + stream.push({ type: "done", reason: message.stopReason as "stop" | "length" | "toolUse", message }); + stream.end(message); + return stream; + }; +} + +function callerTool(name: string, execute?: AgentTool["execute"]): AgentTool { + return { + name, + label: name, + description: `${name} tool`, + parameters: { type: "object", properties: {}, additionalProperties: false } as never, + execute: execute ?? (async () => ({ content: [{ type: "text", text: "ok" }], details: {} })), + }; +} + +function modelsFromStream(streamFn: StreamFn, provider = "openai"): Models { + const models = createCuaModels(); + models.setProvider({ + id: provider, + name: "scripted", + auth: { apiKey: { name: "test", resolve: async () => ({ auth: { apiKey: "test" } }) } }, + getModels: () => [], + stream: streamFn, + streamSimple: streamFn, + } as never); + return models; +} + +/** + * What a consumer does with a handle: compile a pair, hand it to a stock pi + * harness, and recompile-then-apply to change it. The CLI's `CuaCliCatalog` is + * this same shape. + */ +async function openSession(options: { + model: CuaModelInput; + tools: readonly CuaHarnessTool[]; + models?: Models; +}): Promise<{ + harness: AgentHarness; + session: Session; + select: (model: CuaModelInput, tools: readonly CuaHarnessTool[]) => Promise; +}> { + const session = await new InMemorySessionRepo().create(); + const handle = attach({ browser, client, models: options.models }); + const compiled = handle.compile({ model: options.model, tools: options.tools }); + const harness = new AgentHarness({ + session, + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), + }); + compiled.activate(harness); + return { + harness, + session, + select: async (model, tools) => { + const next = handle.compile({ model, tools }); + await next.apply(harness); + }, + }; +} + +describe("compiling a pair", () => { + it("compiles exact native-only, native-plus-CUA, Playwright-only, and browser-act-only catalogs", () => { + const custom = callerTool("customer_lookup"); + const cases = [ + { + model: "anthropic:claude-opus-5" as const, + tools: [cua.providers.anthropic.tools.browser(), custom], + names: ["browser", "customer_lookup"], + }, + { + model: "openai:gpt-5.5" as const, + tools: [cua.providers.openai.tools.computer(), cua.tools.browser.snapshot(), cua.tools.browser.act()], + names: ["computer", "browser_snapshot", "browser_act"], + }, + { model: "openai:gpt-5.5" as const, tools: [cua.tools.playwright()], names: ["playwright_execute"] }, + { model: "openai:gpt-5.5" as const, tools: [cua.tools.browser.act()], names: ["browser_act"] }, + ]; + const handle = attach({ browser, client }); + for (const entry of cases) { + expect(handle.compile({ model: entry.model, tools: entry.tools }).tools.map((tool) => tool.name)).toEqual(entry.names); + } + }); + + it("resolves a ref through the supplied collection, falling back to the registry", () => { + const models = createCuaModels(); + const openai = models.getProvider("openai")!; + const first = { ...getCuaModel("openai:gpt-5.5"), baseUrl: "https://first.example" }; + const second = { ...getCuaModel("openai:gpt-5.6-sol"), baseUrl: "https://second.example" }; + models.setProvider({ ...openai, getModels: () => [first, second] }); + const handle = attach({ browser, client, models }); + + expect(handle.compile({ model: "openai:gpt-5.5", tools: [] }).model).toBe(first); + expect(handle.compile({ model: "openai:gpt-5.6-sol", tools: [] }).model).toBe(second); + // A ref the supplied collection does not carry falls back to the registry + // rather than being refused: the provider decides what exists. + expect(handle.compile({ model: "openai:gpt-5.4", tools: [] }).model.id).toBe("gpt-5.4"); + }); + + it("refuses a native tool the model cannot take, before anything is applied", async () => { + const keep = callerTool("keep"); + const { harness, select } = await openSession({ model: "openai:gpt-5.5", tools: [keep] }); + + await expect(select("openai:gpt-5.5", [cua.providers.anthropic.tools.browser()])).rejects.toThrow(/requires a anthropic model/); + expect(harness.getModel().id).toBe("gpt-5.5"); + expect(harness.getTools().map((tool) => tool.name)).toEqual(["keep"]); + }); + + it("retains protocol-required OpenAI native computer screenshot results outside the image replay limit", async () => { + const contexts: Context[] = []; + const model = getCuaModel("openai:gpt-5.5"); + const image = { type: "image" as const, data: "c2NyZWVuc2hvdA==", mimeType: "image/png" }; + const messages: AgentMessage[] = [ + assistant(model, [{ type: "toolCall", id: "native-shot", name: "computer", arguments: { action: { type: "screenshot" } } }], "toolUse"), + { role: "toolResult", toolCallId: "native-shot", toolName: "computer", content: [image], isError: false, timestamp: 1 }, + assistant(model, [{ type: "toolCall", id: "ordinary-shot", name: "ordinary", arguments: {} }], "toolUse"), + { role: "toolResult", toolCallId: "ordinary-shot", toolName: "ordinary", content: [image], isError: false, timestamp: 2 }, + ]; + const handle = attach({ + browser, + client, + toolResultImageReplayLimit: 0, + models: modelsFromStream(scriptedStream([(selected) => assistant(selected)], contexts)), + }); + const compiled = handle.compile({ model, tools: [cua.providers.openai.tools.computer(), callerTool("ordinary")] }); + const agent = new Agent({ + streamFn: (selected, context, options) => compiled.models.streamSimple(selected, context, options), + initialState: { model: compiled.model, tools: [...compiled.agentTools], messages }, + }); + + await agent.prompt("continue"); + + const results = contexts[0]!.messages.filter((message) => message.role === "toolResult"); + expect(results[0]!.content).toEqual([image]); + expect(results[1]!.content).toEqual([{ type: "text", text: "[stale tool-result images omitted]" }]); + }); +}); + +describe("applying a pair to a live harness", () => { + it("streams the transport the selection derives", async () => { + const streamedApis: string[] = []; + const script = scriptedStream([(selected) => assistant(selected)]); + const { harness, select } = await openSession({ + model: "google:gemini-3.6-flash", + tools: [cua.tools.browser.snapshot()], + models: modelsFromStream((model, context, options) => { + streamedApis.push(model.api); + return script(model, context, options); + }, "google"), + }); + expect(harness.getModel().api).toBe("google-generative-ai"); + + await select("google:gemini-3.6-flash", cua.providers.google.toolsets.browser()); + expect(harness.getModel().api).toBe(GOOGLE_CUA_INTERACTIONS_API); + + await harness.prompt("go"); + expect(streamedApis).toEqual([GOOGLE_CUA_INTERACTIONS_API]); + }); + + it("records no model change when the derived transport is unchanged", async () => { + const { session, select } = await openSession({ model: "openai:gpt-5.5", tools: [callerTool("first")] }); + + await select("openai:gpt-5.5", [callerTool("second")]); + + expect((await session.getBranch()).filter((entry) => entry.type === "model_change")).toEqual([]); + }); + + it("records one model change for a switch that moves both the model and its transport", async () => { + const { harness, session, select } = await openSession({ + model: "google:gemini-3.6-flash", + tools: cua.providers.google.toolsets.browser(), + }); + expect(harness.getModel().api).toBe(GOOGLE_CUA_INTERACTIONS_API); + + await select("openai:gpt-5.5", [cua.tools.browser.snapshot()]); + expect(harness.getModel().api).toBe("openai-responses"); + + expect((await session.getBranch()).filter((entry) => entry.type === "model_change")).toHaveLength(1); + }); + + it("ignores a release from a pair that is no longer live", async () => { + const seen: Model[] = []; + const script = scriptedStream([(selected) => assistant(selected)]); + const models = modelsFromStream((model, context, options) => { + seen.push(model); + return script(model, context, options); + }, "google"); + const session = await new InMemorySessionRepo().create(); + const handle = attach({ browser, client, models }); + const first = handle.compile({ model: "google:gemini-3.6-flash", tools: [cua.tools.browser.snapshot()] }); + const harness = new AgentHarness({ + session, + model: first.model, + models: first.models, + tools: [...first.tools], + activeToolNames: first.tools.map((tool) => tool.name), + }); + const releaseFirst = first.activate(harness); + + // Count what each activation registers on the harness and what it takes + // back, so a leaked pair of handlers is visible. + let live = 0; + const on = harness.on.bind(harness); + harness.on = ((type: never, handler: never) => { + live += 1; + const off = on(type, handler); + return () => { + live -= 1; + off(); + }; + }) as typeof harness.on; + const subscribe = harness.subscribe.bind(harness); + harness.subscribe = ((listener: never) => { + live += 1; + const off = subscribe(listener); + return () => { + live -= 1; + off(); + }; + }) as typeof harness.subscribe; + + await handle.compile({ model: "google:gemini-3.6-flash", tools: cua.providers.google.toolsets.browser() }).apply(harness); + const afterSwap = live; + // The caller still holds the first pair's release. Calling it must neither + // strand `models` with no live catalog nor drop the handle's grip on the + // pair that *is* live — otherwise the next activation cannot release it. + releaseFirst(); + await handle.compile({ model: "google:gemini-3.6-flash", tools: [cua.tools.browser.snapshot()] }).apply(harness); + expect(live).toBe(afterSwap); + + await harness.prompt("go"); + expect(seen.map((model) => model.api)).toEqual(["google-generative-ai"]); + }); + + it("clears failed-turn state before the next prompt", async () => { + let successfulCalls = 0; + const failing = callerTool("failing", async () => { + throw new Error("expected failure"); + }); + const succeeding = callerTool("succeeding", async () => { + successfulCalls += 1; + return { content: [{ type: "text", text: "ok" }], details: {} }; + }); + const { harness } = await openSession({ + model: "anthropic:claude-opus-5", + tools: [cua.providers.anthropic.tools.browser(), failing, succeeding], + models: modelsFromStream(scriptedStream([ + (model) => assistant(model, [{ type: "toolCall", id: "fail", name: "failing", arguments: {} }], "toolUse"), + (model) => assistant(model, [{ type: "toolCall", id: "succeed", name: "succeeding", arguments: {} }], "toolUse"), + (model) => assistant(model, [{ type: "text", text: "done" }]), + ]), "anthropic"), + }); + harness.on("tool_result", (event) => (event.toolName === "failing" ? { terminate: true } : undefined)); + + await harness.prompt("fail once"); + await harness.prompt("try again"); + + expect(successfulCalls).toBe(1); + }); +}); diff --git a/packages/agent/test/attach.test.ts b/packages/agent/test/attach.test.ts index fbffca7..bb5a71b 100644 --- a/packages/agent/test/attach.test.ts +++ b/packages/agent/test/attach.test.ts @@ -4,8 +4,10 @@ import { createCuaModels, cua, GOOGLE_CUA_INTERACTIONS_API, + OPENAI_CUA_COMPUTER_API, type AssistantMessage, type Context, + type CuaSimpleStreamOptions, type Model, } from "@onkernel/cua-ai"; import type Kernel from "@onkernel/sdk"; @@ -28,9 +30,9 @@ function assistant(model: Model): AssistantMessage { }; } -function recordingStream(seen: { model: Model; context: Context }[]): StreamFn { - return (model, context) => { - seen.push({ model, context: { ...context, tools: context.tools?.slice() } }); +function recordingStream(seen: { model: Model; context: Context; options?: CuaSimpleStreamOptions }[]): StreamFn { + return (model, context, options) => { + seen.push({ model, context: { ...context, tools: context.tools?.slice() }, options: options as CuaSimpleStreamOptions }); const stream = createAssistantMessageEventStream(); const message = assistant(model); stream.push({ type: "start", partial: message }); @@ -77,7 +79,7 @@ describe("attach", () => { }); it("drives a plain pi Agent with no CUA agent class", async () => { - const seen: { model: Model; context: Context }[] = []; + const seen: { model: Model; context: Context; options?: CuaSimpleStreamOptions }[] = []; const handle = attach({ browser, client }); const compiled = handle.compile({ model: "openai:gpt-5.5", tools: [cua.tools.browser.snapshot()] }); @@ -93,7 +95,7 @@ describe("attach", () => { }); it("drives a plain pi AgentHarness, with CUA's behaviors installed", async () => { - const seen: { model: Model; context: Context }[] = []; + const seen: { model: Model; context: Context; options?: CuaSimpleStreamOptions }[] = []; const handle = attach({ browser, client, models: modelsFromStream(recordingStream(seen)) }); const compiled = handle.compile({ model: "openai:gpt-5.5", tools: [cua.tools.browser.snapshot()] }); const session = await new InMemorySessionRepo().create(); @@ -104,14 +106,41 @@ describe("attach", () => { models: compiled.models, tools: [...compiled.tools], activeToolNames: compiled.tools.map((tool) => tool.name), - } as never); - const uninstall = compiled.install(harness); + }); + const release = compiled.activate(harness); await harness.prompt("go"); expect(seen).toHaveLength(1); expect(seen[0]!.context.tools?.map((tool) => tool.name)).toEqual(["browser_snapshot"]); - expect(typeof uninstall).toBe("function"); - uninstall(); + expect(typeof release).toBe("function"); + release(); + }); + + it("streams the live catalog's tool plan after a swap, not the one the harness was built with", async () => { + const seen: { model: Model; context: Context; options?: CuaSimpleStreamOptions }[] = []; + const handle = attach({ browser, client, models: modelsFromStream(recordingStream(seen)) }); + const first = handle.compile({ model: "openai:gpt-5.5", tools: [cua.tools.browser.snapshot()] }); + const session = await new InMemorySessionRepo().create(); + + const harness = new AgentHarness({ + session, + model: first.model, + models: first.models, + tools: [...first.tools], + activeToolNames: first.tools.map((tool) => tool.name), + }); + first.activate(harness); + + const second = handle.compile({ model: "openai:gpt-5.5", tools: [cua.providers.openai.tools.computer()] }); + await second.apply(harness); + await harness.prompt("go"); + + // pi fixes `models` at construction while the headers, payload transforms + // and tool plan it carries are per-catalog, so a per-compile collection + // would keep sending the first catalog's plan for the rest of the session. + expect(seen).toHaveLength(1); + expect(seen[0]!.options?.cuaIncomingToolPlan?.openaiComputerName).toBe("computer"); + expect(seen[0]!.model.api).toBe(OPENAI_CUA_COMPUTER_API); }); it("spends an empty-response retry only when the follow-up is queued", async () => { diff --git a/packages/agent/test/e2e.live.test.ts b/packages/agent/test/e2e.live.test.ts index 80b1638..2b9355e 100644 --- a/packages/agent/test/e2e.live.test.ts +++ b/packages/agent/test/e2e.live.test.ts @@ -1,8 +1,9 @@ import Kernel from "@onkernel/sdk"; import { describe, expect, it } from "vitest"; import { - CuaAgent, - CuaAgentHarness, + Agent, + AgentHarness, + attach, cua, InMemorySessionRepo, type AgentEvent, @@ -259,18 +260,17 @@ describe("Cua live e2e", () => { const test = shouldRunCase(c) ? it : it.skip; test( - `${c.name}: CuaAgent executes browser steps`, + `${c.name}: a plain pi Agent executes browser steps`, async () => { await withBrowser(async (client, browser) => { const stats = createRunStats(); - const agent = new CuaAgent({ - browser, - client, - getApiKey: () => apiKeyForCase(c), + const compiled = attach({ browser, client }).compile({ model: c.modelRef, tools: toolsForCase(c) }); + const agent = new Agent({ + streamFn: (model, context, options) => compiled.models.streamSimple(model, context, options), afterToolCall: async () => ({ terminate: true }), - tools: toolsForCase(c), initialState: { - model: c.modelRef, + model: compiled.model, + tools: [...compiled.agentTools], systemPrompt: systemPromptForCase(c), }, }); @@ -286,18 +286,20 @@ describe("Cua live e2e", () => { ); test( - `${c.name}: CuaAgentHarness executes browser steps`, + `${c.name}: a plain pi AgentHarness executes browser steps`, async () => { await withBrowser(async (client, browser) => { const stats = createRunStats(); - const harness = new CuaAgentHarness({ + const compiled = attach({ browser, client }).compile({ model: c.modelRef, tools: toolsForCase(c) }); + const harness = new AgentHarness({ ...(await createHarnessServices(`${c.name}-harness`)), - browser, - client, - model: c.modelRef, - tools: toolsForCase(c), + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), systemPrompt: systemPromptForCase(c), }); + compiled.activate(harness); harness.on("tool_result", () => ({ terminate: true })); harness.subscribe((event) => { @@ -316,21 +318,17 @@ describe("Cua live e2e", () => { const test = shouldRunSwitchCase(c) ? it : it.skip; test( - `${c.name}: CuaAgent switches models after a turn`, + `${c.name}: a plain pi Agent switches models after a turn`, async () => { await withBrowser(async (client, browser) => { let stats = createRunStats(); - const agent = new CuaAgent({ - browser, - client, - getApiKey: (provider) => { - if (provider === c.from.modelRef.split(":")[0]) return apiKeyForCase(c.from); - if (provider === c.to.modelRef.split(":")[0]) return apiKeyForCase(c.to); - return undefined; - }, - tools: modelSwitchTools(), + const handle = attach({ browser, client }); + const from = handle.compile({ model: c.from.modelRef, tools: modelSwitchTools() }); + const agent = new Agent({ + streamFn: (model, context, options) => handle.models.streamSimple(model, context, options), initialState: { - model: c.from.modelRef, + model: from.model, + tools: [...from.agentTools], systemPrompt: "Use only the explicitly selected screenshot tool.", }, }); @@ -341,8 +339,12 @@ describe("Cua live e2e", () => { await agent.prompt(modelSwitchPrompt); assertStats(stats, c.from, "agent"); + // A switch recompiles: the new model carries the transport its + // tools derive, and its executables replace the old pair. stats = createRunStats(); - agent.state.model = c.to.modelRef; + const to = handle.compile({ model: c.to.modelRef, tools: modelSwitchTools() }); + agent.state.model = to.model; + agent.state.tools = [...to.agentTools]; await agent.prompt(modelSwitchPrompt); assertStats(stats, c.to, "agent"); }); @@ -351,18 +353,21 @@ describe("Cua live e2e", () => { ); test( - `${c.name}: CuaAgentHarness switches models after a turn`, + `${c.name}: a plain pi AgentHarness switches models after a turn`, async () => { await withBrowser(async (client, browser) => { let stats = createRunStats(); - const harness = new CuaAgentHarness({ + const handle = attach({ browser, client }); + const from = handle.compile({ model: c.from.modelRef, tools: modelSwitchTools() }); + const harness = new AgentHarness({ ...(await createHarnessServices(`${c.name}-harness-switch`)), - browser, - client, - model: c.from.modelRef, - tools: modelSwitchTools(), + model: from.model, + models: from.models, + tools: [...from.tools], + activeToolNames: from.tools.map((tool) => tool.name), systemPrompt: "Use only the explicitly selected screenshot tool.", }); + from.activate(harness); harness.subscribe((event) => { recordRunEvent(stats, event); }); @@ -371,7 +376,7 @@ describe("Cua live e2e", () => { assertStats(stats, c.from, "harness"); stats = createRunStats(); - await harness.setModel(c.to.modelRef); + await handle.compile({ model: c.to.modelRef, tools: modelSwitchTools() }).apply(harness); await harness.prompt(modelSwitchPrompt); assertStats(stats, c.to, "harness"); }); diff --git a/packages/agent/test/harness-context.test.ts b/packages/agent/test/harness-context.test.ts index fa6ae63..89e3f06 100644 --- a/packages/agent/test/harness-context.test.ts +++ b/packages/agent/test/harness-context.test.ts @@ -14,7 +14,8 @@ import { createEditTool, createReadTool, createWriteTool, - CuaAgentHarness, + AgentHarness, + attach, InMemorySessionRepo, NodeExecutionEnv, type AgentHarnessTool, @@ -71,11 +72,27 @@ function modelsFromStream(streamFn: StreamFn, provider = "openai") { return models; } -async function harnessSession() { - return new InMemorySessionRepo().create(); +/** Build a stock pi harness from a handle, exactly as a consumer does. */ +async function openHarness(options: { + models: ReturnType; + tools: readonly AgentHarnessTool[]; + toolContext: TContext; +}): Promise> { + const handle = attach({ browser, client, models: options.models }); + const compiled = handle.compile({ model: "openai:gpt-5.5", tools: options.tools }); + const harness = new AgentHarness({ + session: await new InMemorySessionRepo().create(), + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), + toolContext: options.toolContext, + }); + compiled.activate(harness); + return harness; } -describe("CuaAgentHarness tool context", () => { +describe("harness tool context", () => { it("delivers the exact supplied context object to custom harness tools", async () => { interface CustomContext { env: ExecutionToolContext["env"]; @@ -94,11 +111,7 @@ describe("CuaAgentHarness tool context", () => { return { content: [{ type: "text", text: "ok" }], details: {} }; }, }; - const harness = new CuaAgentHarness({ - browser, - client, - session: await harnessSession(), - model: "openai:gpt-5.5", + const harness = await openHarness({ models: modelsFromStream(scriptedStream([ (model) => assistant(model, [{ type: "toolCall", id: "call-1", name: "custom_context", arguments: {} }], "toolUse"), (model) => assistant(model, [{ type: "text", text: "done" }]), @@ -115,11 +128,7 @@ describe("CuaAgentHarness tool context", () => { it("runs pi's native read/write/edit/bash tools against the context's execution env", async () => { const cwd = mkdtempSync(join(tmpdir(), "cua-harness-tools-")); - const harness = new CuaAgentHarness({ - browser, - client, - session: await harnessSession(), - model: "openai:gpt-5.5", + const harness = await openHarness({ models: modelsFromStream(scriptedStream([ (model) => assistant(model, [{ type: "toolCall", id: "write-1", name: "write", arguments: { path: "notes.txt", content: "hello cua\n" } }], "toolUse"), (model) => assistant(model, [{ type: "toolCall", id: "edit-1", name: "edit", arguments: { path: "notes.txt", edits: [{ oldText: "hello", newText: "goodbye" }] } }], "toolUse"), diff --git a/packages/agent/test/openai-deferred-tools.test.ts b/packages/agent/test/openai-deferred-tools.test.ts deleted file mode 100644 index 962725d..0000000 --- a/packages/agent/test/openai-deferred-tools.test.ts +++ /dev/null @@ -1,109 +0,0 @@ -import type { AgentTool } from "@earendil-works/pi-agent-core"; -import type { ToolCall } from "@onkernel/cua-ai"; -import type Kernel from "@onkernel/sdk"; -import { describe, expect, it, vi } from "vitest"; -import { CuaAgent, type KernelBrowser } from "../src/index"; - -const { responsesCreate } = vi.hoisted(() => ({ responsesCreate: vi.fn() })); - -vi.mock("openai", () => ({ - default: class { - responses = { - create: (payload: unknown) => ({ - withResponse: async () => ({ - data: responseEvents(responsesCreate(payload)), - response: { status: 200, headers: new Headers() }, - }), - }), - }; - }, -})); - -async function* responseEvents(response: Record) { - yield { type: "response.created", response: { id: response.id } }; - for (const [outputIndex, item] of ((response.output as unknown[]) ?? []).entries()) { - yield { type: "response.output_item.added", output_index: outputIndex, item }; - yield { type: "response.output_item.done", output_index: outputIndex, item }; - } - yield { type: "response.completed", response }; -} - -function functionCall(id: string, callId: string, name: string, namespace?: string) { - return { - type: "function_call", - id, - call_id: callId, - name, - arguments: "{}", - status: "completed", - ...(namespace ? { namespace } : {}), - }; -} - -function response(id: string, output: unknown[]) { - return { - id, - status: "completed", - output, - usage: { input_tokens: 1, output_tokens: 1, total_tokens: 2 }, - }; -} - -function callerTool(name: string, execute: AgentTool["execute"], executionMode?: AgentTool["executionMode"]): AgentTool { - return { - name, - label: name, - description: `${name} tool`, - parameters: { type: "object", properties: {}, additionalProperties: false } as never, - execute, - ...(executionMode ? { executionMode } : {}), - }; -} - -describe("OpenAI deferred tool namespace continuation", () => { - it("executes a deferred-added call and replays its provider namespace", async () => { - responsesCreate - .mockReturnValueOnce(response("resp_loader", [functionCall("fc_loader", "call_loader", "loader")])) - .mockReturnValueOnce(response("resp_added", [functionCall("fc_added", "call_added", "added", "deferred_tools")])) - .mockReturnValueOnce(response("resp_done", [{ - type: "message", - id: "msg_done", - role: "assistant", - status: "completed", - content: [{ type: "output_text", text: "done", annotations: [] }], - }])); - - const addedExecute = vi.fn(async () => ({ content: [{ type: "text" as const, text: "added result" }], details: {} })); - const added = callerTool("added", addedExecute); - let agent!: CuaAgent; - const loader = callerTool("loader", async () => { - agent.setTools([...agent.getTools(), added]); - return { content: [{ type: "text", text: "loaded" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser: { session_id: "browser_123" } as KernelBrowser, - client: {} as Kernel, - tools: [loader], - initialState: { model: "openai:gpt-5.5" }, - getApiKey: () => "test", - }); - - await agent.prompt("load and run the added tool"); - - expect(addedExecute).toHaveBeenCalledTimes(1); - const deferredCall = agent.state.messages - .find((message) => message.role === "assistant" && message.content.some((part) => part.type === "toolCall" && part.name === "added")) - ?.content.find((part): part is ToolCall => part.type === "toolCall") as (ToolCall & { namespace?: string }) | undefined; - expect(deferredCall?.namespace).toBe("deferred_tools"); - - const secondPayload = responsesCreate.mock.calls[1]?.[0] as { input: Array> }; - expect(secondPayload.input).toContainEqual(expect.objectContaining({ type: "tool_search_output" })); - const replayPayload = responsesCreate.mock.calls[2]?.[0] as { input: Array> }; - expect(replayPayload.input).toContainEqual(expect.objectContaining({ - type: "function_call", - call_id: "call_added", - name: "added", - namespace: "deferred_tools", - })); - }); -}); diff --git a/packages/agent/test/published-declarations.test.ts b/packages/agent/test/published-declarations.test.ts index 2442aa4..f174c95 100644 --- a/packages/agent/test/published-declarations.test.ts +++ b/packages/agent/test/published-declarations.test.ts @@ -16,8 +16,9 @@ const repoRoot = resolve(agentRoot, "..", ".."); const CONSUMER = ` import { Type } from "typebox"; import { - CuaAgent, - CuaAgentHarness, + Agent, + AgentHarness, + attach, InMemorySessionRepo, NodeExecutionEnv, createBashTool, @@ -63,32 +64,31 @@ const agentTools: readonly CuaAgentTool[] = [...cua.toolsets.browser()]; async function build() { const session = await new InMemorySessionRepo().create(); - const harness = new CuaAgentHarness({ - browser, - client, + const handle = attach({ browser, client }); + + const compiled = handle.compile({ model: "openai:gpt-5.6-sol", tools: harnessTools }); + const harness = new AgentHarness({ session, - model: "openai:gpt-5.6-sol", - tools: harnessTools, + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), toolContext, }); - const contextFree = new CuaAgentHarness({ - browser, - client, - session, - model: getCuaModel("openai:gpt-5.6-sol"), - tools: [], - }); - new CuaAgentHarness({ - browser, - client, - session, - model: "openai:gpt-5.6-sol", - tools: [], - // @ts-expect-error env was removed; the tool context now carries the execution env - env: new NodeExecutionEnv({ cwd: process.cwd() }), + const release = compiled.activate(harness); + // A swap compiles for the same tool context the harness delivers. + await handle.compile({ model: getCuaModel("openai:gpt-5.6-sol"), tools: [] }).apply(harness); + + const lowLevel = handle.compile({ model: "openai:gpt-5.6-sol", tools: agentTools }); + const agent = new Agent({ + streamFn: (model, context, options) => lowLevel.models.streamSimple(model, context, options), + initialState: { model: lowLevel.model, tools: [...lowLevel.agentTools] }, }); - const agent = new CuaAgent({ browser, client, tools: agentTools, initialState: { model: "openai:gpt-5.6-sol" } }); - return { harness, contextFree, agent }; + + // @ts-expect-error the browser and client belong to attach(), not to a compile + handle.compile({ browser, model: "openai:gpt-5.6-sol", tools: [] }); + + return { harness, agent, release }; } void build; diff --git a/packages/agent/test/tool-manager.test.ts b/packages/agent/test/tool-manager.test.ts index c9d93d9..15cc28d 100644 --- a/packages/agent/test/tool-manager.test.ts +++ b/packages/agent/test/tool-manager.test.ts @@ -48,13 +48,6 @@ describe("CuaToolManager declaration projection", () => { expect("agentTools" in manager.catalog).toBe(false); }); - it("keeps the caller list as the sole owner of requested objects", () => { - const spec = cua.tools.browser.snapshot(); - const tool = callerTool("lookup"); - const manager = new CuaToolManager(setup(), "openai:gpt-5.5", [spec, tool]); - expect(manager.getTools()[0]).toBe(spec); - expect(manager.getTools()[1]).toBe(tool); - }); }); describe("CuaToolManager identity join", () => { @@ -93,63 +86,29 @@ describe("CuaToolManager identity join", () => { }); describe("CuaToolManager transport derivation", () => { - it("derives the compiled model's api from selected tools on construction and on both mutation paths", () => { - const manager = new CuaToolManager(setup(), "google:gemini-3.6-flash", [cua.tools.browser.snapshot()]); - expect(manager.catalog.model.api).toBe("google-generative-ai"); - - manager.commit(manager.prepareTools(cua.providers.google.toolsets.browser())); - expect(manager.catalog.model.api).toBe(GOOGLE_CUA_INTERACTIONS_API); - - manager.commit(manager.prepareModel("google:gemini-3.6-flash")); - expect(manager.catalog.model.api).toBe(GOOGLE_CUA_INTERACTIONS_API); + it("derives the compiled model's api from the tools selected with it", () => { + const resources = setup(); + const cdp = new CuaToolManager(resources, "google:gemini-3.6-flash", [cua.tools.browser.snapshot()]); + const native = new CuaToolManager(resources, "google:gemini-3.6-flash", cua.providers.google.toolsets.browser()); - manager.commit(manager.prepareTools([cua.tools.browser.snapshot()])); - expect(manager.catalog.model.api).toBe("google-generative-ai"); + expect(cdp.catalog.model.api).toBe("google-generative-ai"); + expect(native.catalog.model.api).toBe(GOOGLE_CUA_INTERACTIONS_API); }); }); -describe("CuaToolManager implementation identity", () => { - it("materializes each spec exactly once across model and tool recompilation", () => { +describe("CuaToolManager materialization", () => { + it("materializes each spec exactly once, however many pairs it is compiled into", () => { const resources = setup(); const spy = vi.spyOn(resources, "materialize"); const spec = cua.tools.browser.snapshot(); - const manager = new CuaToolManager(resources, "openai:gpt-5.5", [spec]); - manager.commit(manager.prepareModel("openai:gpt-5.6-sol")); - manager.commit(manager.prepareTools([...manager.getTools(), callerTool("added")])); + new CuaToolManager(resources, "openai:gpt-5.5", [spec]); + new CuaToolManager(resources, "openai:gpt-5.6-sol", [spec]); + new CuaToolManager(resources, "openai:gpt-5.6-sol", [spec, callerTool("added")]); - // Every materialization for the same spec object returned one identical executable. + // The executable is cached per pool and per spec object, so pi sees one + // stable implementation across every recompile. expect(spy.mock.calls.length).toBeGreaterThan(1); expect(new Set(spy.mock.results.map((result) => result.value)).size).toBe(1); }); - - it("keeps the same spec object stable across model recompilation", () => { - const manager = new CuaToolManager(setup(), "openai:gpt-5.5", [cua.tools.browser.snapshot()]); - const first = manager.prepareModel("openai:gpt-5.6-sol"); - const second = manager.prepareModel("openai:gpt-5.6-sol"); - expect(second.fingerprints).toEqual(first.fingerprints); - }); - - it("treats a freshly created spec object as a conservative replacement", () => { - const manager = new CuaToolManager(setup(), "openai:gpt-5.5", [cua.tools.browser.snapshot()]); - const stable = manager.prepareModel("openai:gpt-5.6-sol"); - const replaced = manager.prepareTools([cua.tools.browser.snapshot()]); - expect(replaced.catalog.entries[0]?.fingerprint).toBe(stable.catalog.entries[0]?.fingerprint); - expect(replaced.fingerprints[0]).not.toBe(stable.fingerprints[0]); - }); - - it("retains implementation identity for a new wrapper reusing the same execute function", () => { - const sharedExecute: AgentTool["execute"] = async () => ({ content: [{ type: "text", text: "ok" }], details: {} }); - const original = callerTool("worker", sharedExecute); - const manager = new CuaToolManager(setup(), "openai:gpt-5.5", [original]); - const baseline = manager.prepareModel("openai:gpt-5.6-sol"); - - const rewrapped = { ...original }; - expect(rewrapped).not.toBe(original); - expect(manager.prepareTools([rewrapped]).fingerprints).toEqual(baseline.fingerprints); - - const freshExecute = manager.prepareTools([callerTool("worker")]); - expect(freshExecute.catalog.entries[0]?.fingerprint).toBe(baseline.catalog.entries[0]?.fingerprint); - expect(freshExecute.fingerprints[0]).not.toBe(baseline.fingerprints[0]); - }); }); diff --git a/packages/ai/README.md b/packages/ai/README.md index 5d6ec5c..75c43d3 100644 --- a/packages/ai/README.md +++ b/packages/ai/README.md @@ -231,8 +231,8 @@ unioned and deduplicated; exact-value conflicts throw. Ordinary function tools are marked eligible only where pi 0.83.0 supports deferred loading. Provider-native tools are eager-only. The catalog itself does -not guess when tools were added; `CuaAgent`/`CuaAgentHarness` record in-tool -additions through pi's active-tool change entries. +not guess when tools were added; a caller that adds tools mid-turn records the +addition through pi's active-tool change entries. ## Provider behavior diff --git a/packages/ai/src/providers.ts b/packages/ai/src/providers.ts index 3b19e65..3e3fadb 100644 --- a/packages/ai/src/providers.ts +++ b/packages/ai/src/providers.ts @@ -60,7 +60,7 @@ let defaultCuaModels: MutableModels | undefined; /** * Shared default {@link createCuaModels} collection, created on first use. * - * `CuaAgent` and `CuaAgentHarness` stream through this instance unless given + * cua-agent streams through this instance unless given * another one. Auth resolves from the documented CUA env-var convention (see * `cuaApiKeyEnvVarsForProvider`); pass an explicit `options.apiKey` per * request to override. diff --git a/packages/cli/CHANGELOG.md b/packages/cli/CHANGELOG.md index a87266a..15d20c7 100644 --- a/packages/cli/CHANGELOG.md +++ b/packages/cli/CHANGELOG.md @@ -2,6 +2,11 @@ ## Unreleased +- The CLI now builds a stock pi `AgentHarness` from `attach()` instead of + `CuaAgentHarness`. `buildCuaHarness()` returns `{ harness, catalog }`: the + harness is pi's, and `CuaCliCatalog` holds the live (model, tools) selection + that `/model` and `/tools` recompile. Behavior is unchanged, including the + guarantee that a rejected selection leaves the session exactly as it was. - `/tools` now offers the model's whole tool menu instead of filtering the list the CLI composed. Tools the CLI did not choose — `playwright_execute`, the computer toolset, a provider-native surface — can be enabled, and tools the diff --git a/packages/cli/README.md b/packages/cli/README.md index c5ae1b1..2e3c9d2 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -1,7 +1,7 @@ # `@onkernel/cua-cli` The CLI / TUI binary for the [`cua`](../../README.md) monorepo. Wires -[`@onkernel/cua-agent`](../agent)'s `CuaAgentHarness` to +[`@onkernel/cua-agent`](../agent)'s `attach()` handle and a pi `AgentHarness` to [`pi-tui`](https://www.npmjs.com/package/@earendil-works/pi-tui) for an interactive front-end and to [`pi-coding-agent`](https://www.npmjs.com/package/@earendil-works/pi-coding-agent)'s diff --git a/packages/cli/src/cli-harness.ts b/packages/cli/src/cli-harness.ts index aad9178..f62cc81 100644 --- a/packages/cli/src/cli-harness.ts +++ b/packages/cli/src/cli-harness.ts @@ -23,7 +23,13 @@ import { type ModelActionType, } from "./action/prompts"; import { runAction, emitCompact } from "./action/harness-runner"; -import { buildCuaHarness, defaultApplicationTools, defaultInteractionTools } from "./harness"; +import { + buildCuaHarness, + type CuaCliCatalog, + type CuaCliHarness, + defaultApplicationTools, + defaultInteractionTools, +} from "./harness"; import { provisionBrowser } from "./harness-browser"; import { DEFAULT_CUA_MODEL_REF, listSupportedModels, resolveCuaModelRef } from "./harness-models"; import { @@ -472,7 +478,8 @@ interface HarnessRuntime { skills: Skill[]; contextFiles: ContextFile[]; applicationTools: ReturnType; - harness: ReturnType; + harness: CuaCliHarness; + catalog: CuaCliCatalog; provider: string; modelRef: CuaModelRef; } @@ -567,7 +574,7 @@ async function finishHarnessRuntime( const thinkingLevel = mapThinkingLevel(flags.thinking); const baseUrlOverride = providerBaseUrlOverride(provider); const applicationTools = defaultApplicationTools(); - const harness = buildCuaHarness({ + const { harness, catalog } = buildCuaHarness({ cwd, client: provisioned.handle.client, browser: provisioned.handle.browser, @@ -588,6 +595,7 @@ async function finishHarnessRuntime( contextFiles, applicationTools, harness, + catalog, provider, modelRef: auth.modelRef, }; @@ -672,6 +680,7 @@ export async function runInteractiveCommand( return await runInteractive({ cwd: process.cwd(), harness: runtime.harness, + catalog: runtime.catalog, browserHandle: runtime.handle, session: runtime.session, skills: runtime.skills, diff --git a/packages/cli/src/harness.ts b/packages/cli/src/harness.ts index a21a5de..78e4a3c 100644 --- a/packages/cli/src/harness.ts +++ b/packages/cli/src/harness.ts @@ -1,7 +1,10 @@ import { - CuaAgentHarness, - type CuaAgentHarnessOptions, + AgentHarness, + attach, + type CuaAttachOptions, + type CuaBrowserHandle, type CuaHarnessTool, + type CuaModelInput, formatSkillsForSystemPrompt, type KernelBrowser, type Session, @@ -21,20 +24,86 @@ import { } from "@onkernel/cua-ai"; import type Kernel from "@onkernel/sdk"; import { + type AgentHarnessTool, createBashTool, createEditTool, createReadTool, createWriteTool, type ExecutionToolContext, + type PromptTemplate, } from "@earendil-works/pi-agent-core"; import { NodeExecutionEnv } from "@earendil-works/pi-agent-core/node"; import type { ContextFile } from "./harness-skills"; -/** CLI harness: a CUA harness whose tool context carries the coding tools' execution environment. */ -export type CuaCliHarness = CuaAgentHarness; /** One tool in the CLI harness's caller-owned list. */ export type CuaCliTool = CuaHarnessTool; +/** + * The CLI's harness: stock pi, with no CUA class wrapping it. Its tool type is + * pi's own — {@link CuaCliTool} is the caller-owned input the catalog compiles + * from, and a CUA spec is not executable until the handle materializes it. + */ +export type CuaCliHarness = AgentHarness>; + +/** + * The live (model, tools) selection, and the compile-then-swap that changes it. + * + * `attach()` hands back an immutable compiled pair, so the current selection + * lives with whoever can change it — here, `/model` and `/tools`. Both steps + * fail safe: `compile()` throws before anything mutates, and `apply()` restores + * the previous pair if pi rejects the new one, so a rejected selection leaves + * the session exactly as it was. + */ +export class CuaCliCatalog { + private selection: CuaModelInput; + private requested: readonly CuaCliTool[]; + + constructor( + private readonly handle: CuaBrowserHandle, + private readonly harness: CuaCliHarness, + selection: CuaModelInput, + requested: readonly CuaCliTool[], + ) { + this.selection = selection; + this.requested = [...requested]; + } + + /** The caller-owned tool list, as selected — CUA specs, not materialized pi tools. */ + getTools(): readonly CuaCliTool[] { + return [...this.requested]; + } + + setTools(tools: readonly CuaCliTool[]): Promise { + return this.swap(this.selection, tools); + } + + setModel(model: CuaModelInput): Promise { + return this.swap(model, this.requested); + } + + /** + * Select a model and its tool list in one compile. Staging the two in + * sequence would compile an intermediate catalog whose derived transport + * differs from both the old and the new one. + */ + setModelAndTools(model: CuaModelInput, tools: readonly CuaCliTool[]): Promise { + return this.swap(model, tools); + } + + private async swap(model: CuaModelInput, tools: readonly CuaCliTool[]): Promise { + const compiled = this.handle.compile({ model, tools }); + await compiled.apply(this.harness); + this.selection = model; + this.requested = [...tools]; + } +} + +/** A CLI session: stock pi driving the agent, a CUA handle owning the browser. */ +export interface CuaCliSession { + readonly harness: CuaCliHarness; + readonly catalog: CuaCliCatalog; +} + export interface BuildCuaHarnessOptions { cwd: string; client: Kernel; @@ -47,14 +116,14 @@ export interface BuildCuaHarnessOptions { /** Override the CLI's explicit interaction + coding tool list. */ tools?: CuaCliTool[]; models?: Models; - toolResultImageReplayLimit?: CuaAgentHarnessOptions["toolResultImageReplayLimit"]; - responseThreading?: CuaAgentHarnessOptions["responseThreading"]; - retry?: CuaAgentHarnessOptions["retry"]; + toolResultImageReplayLimit?: CuaAttachOptions["toolResultImageReplayLimit"]; + responseThreading?: CuaAttachOptions["responseThreading"]; + retry?: CuaAttachOptions["retry"]; modelBaseUrl?: string; } -/** Build the CLI harness with one explicit tool list and a caller-owned prompt. */ -export function buildCuaHarness(opts: BuildCuaHarnessOptions): CuaCliHarness { +/** Build the CLI session with one explicit tool list and a caller-owned prompt. */ +export function buildCuaHarness(opts: BuildCuaHarnessOptions): CuaCliSession { const skills = opts.skills ?? []; const contextFiles = opts.contextFiles ?? []; const model: CuaModelRef | Model = opts.modelBaseUrl @@ -64,21 +133,28 @@ export function buildCuaHarness(opts: BuildCuaHarnessOptions): CuaCliHarness { ...defaultInteractionTools(opts.model), ...defaultApplicationTools(), ]; - return new CuaAgentHarness({ - session: opts.session, - model, - models: opts.models, + const handle = attach({ browser: opts.browser, client: opts.client, - tools, + models: opts.models, + toolResultImageReplayLimit: opts.toolResultImageReplayLimit, + responseThreading: opts.responseThreading, + retry: opts.retry, + }); + const compiled = handle.compile({ model, tools }); + const harness: CuaCliHarness = new AgentHarness({ + session: opts.session, + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), toolContext: { env: new NodeExecutionEnv({ cwd: opts.cwd }) }, resources: { skills }, thinkingLevel: opts.thinkingLevel, systemPrompt: ({ resources }) => composeSystemPrompt(resources.skills ?? [], contextFiles), - toolResultImageReplayLimit: opts.toolResultImageReplayLimit, - responseThreading: opts.responseThreading, - retry: opts.retry, }); + compiled.activate(harness); + return { harness, catalog: new CuaCliCatalog(handle, harness, model, tools) }; } /** Coding tools owned by the CLI application rather than inferred from a compiled catalog. */ diff --git a/packages/cli/src/tui/main.ts b/packages/cli/src/tui/main.ts index 2f7047f..568534f 100644 --- a/packages/cli/src/tui/main.ts +++ b/packages/cli/src/tui/main.ts @@ -20,7 +20,7 @@ import { import { initTheme } from "@earendil-works/pi-coding-agent"; import { homedir } from "node:os"; import { type CuaModelRef, listCuaModels, type Model } from "@onkernel/cua-ai"; -import { type CuaCliHarness, type CuaCliTool } from "../harness"; +import { type CuaCliCatalog, type CuaCliHarness, type CuaCliTool } from "../harness"; import type { CuaBrowserHandle } from "../harness-browser"; import { resolveCuaModelRef } from "../harness-models"; import { updateNamedSessionRuntime } from "../harness-named-sessions"; @@ -43,6 +43,8 @@ import { cuaVersion } from "./version"; export interface InteractiveOptions { cwd: string; harness: CuaCliHarness; + /** The live (model, tools) selection `/model` and `/tools` change. */ + catalog: CuaCliCatalog; browserHandle: CuaBrowserHandle; session: Session; skills?: Skill[]; @@ -379,9 +381,9 @@ export async function runInteractive(opts: InteractiveOptions): Promise // Native catalogs are incompatible across providers, and the selected // tools decide the transport, so the new model and its interaction // catalog have to compile as one pair rather than in sequence. - await opts.harness.setModelAndTools(resolved, installedTools); + await opts.catalog.setModelAndTools(resolved, installedTools); } else { - await opts.harness.setModel(resolved); + await opts.catalog.setModel(resolved); } const model = opts.harness.getModel(); footer.update({ @@ -453,7 +455,7 @@ export async function runInteractive(opts: InteractiveOptions): Promise catalogQueue.run(async () => { const next = toolsForSelection(items, enabledKeys); try { - await opts.harness.setTools(next); + await opts.catalog.setTools(next); toolSelectionCustomized = !sameToolList(next, baselineTools); messages.addNotice(`tools → ${next.length} enabled`); debug?.log("tools_applied", { enabled: next.length, baseline: baselineTools.length }); @@ -472,7 +474,7 @@ export async function runInteractive(opts: InteractiveOptions): Promise requestRender("tools_no_ref"); return; } - const live = opts.harness.getTools(); + const live = opts.catalog.getTools(); // Availability is pairwise, so the menu is rebuilt against each staged // selection rather than computed once when the picker opens. const menuFor = (selected: readonly CuaCliTool[]) => describeMenu(modelRef, opts.applicationTools, selected); @@ -770,7 +772,7 @@ function tryResolveModelRef(input: string | undefined): CuaModelRef | undefined * customization would have shrunk. */ function composeBaselineTools(opts: InteractiveOptions, ref: CuaModelRef | undefined): readonly CuaCliTool[] { - if (!opts.interactionToolsForModel || !ref) return opts.harness.getTools(); + if (!opts.interactionToolsForModel || !ref) return opts.catalog.getTools(); return [...opts.interactionToolsForModel(ref), ...opts.applicationTools]; } diff --git a/packages/cli/test/fixtures/harness.ts b/packages/cli/test/fixtures/harness.ts index ca5a5e9..27420d1 100644 --- a/packages/cli/test/fixtures/harness.ts +++ b/packages/cli/test/fixtures/harness.ts @@ -7,7 +7,7 @@ import { tmpdir } from "node:os"; import { mkdtempSync } from "node:fs"; import { join } from "node:path"; import { parseCuaModelRef } from "@onkernel/cua-ai"; -import { buildCuaHarness, type CuaCliTool, defaultInteractionTools } from "../../src/harness"; +import { buildCuaHarness, type CuaCliSession, type CuaCliTool, defaultInteractionTools } from "../../src/harness"; import { createFakeKernelEnvironment, type FakeKernelEnvironment } from "./fake-kernel"; import type { ScriptedProviderHandle, ScriptedTurn } from "./scripted-provider"; import { createScriptedCuaModels } from "./scripted-provider"; @@ -17,7 +17,8 @@ export interface TestHarnessFixture { kernel: FakeKernelEnvironment; session: Session; cwd: string; - harness: ReturnType; + harness: CuaCliSession["harness"]; + catalog: CuaCliSession["catalog"]; } export interface BuildTestHarnessOptions { @@ -39,7 +40,7 @@ export async function buildTestHarness(opts: BuildTestHarnessOptions): Promise { const contextFiles = fixture.contextFiles ?? []; const applicationTools = fixture.tools ? defaultApplicationTools(cwd) : []; const interactionToolsForModel = fixture.tools ? defaultInteractionTools : undefined; - const harness = buildCuaHarness({ + const { harness, catalog } = buildCuaHarness({ cwd, client: kernel.client, browser: kernel.browser, @@ -64,6 +64,7 @@ async function main(): Promise { const code = await runInteractive({ cwd, harness, + catalog, browserHandle: { client: kernel.client, browser: kernel.browser, diff --git a/packages/cli/test/harness-assembly.test.ts b/packages/cli/test/harness-assembly.test.ts index 0a7fd13..70837f9 100644 --- a/packages/cli/test/harness-assembly.test.ts +++ b/packages/cli/test/harness-assembly.test.ts @@ -41,14 +41,14 @@ describe("buildCuaHarness", () => { const cwd = mkdtempSync(join(tmpdir(), "cua-cli-harness-")); const kernel = createFakeKernelEnvironment(); const session = await new InMemorySessionRepo().create(); - const harness = buildCuaHarness({ + const { harness, catalog } = buildCuaHarness({ cwd, client: kernel.client, browser: kernel.browser, session, model: "openai:gpt-5.5", }); - const toolNames = harness.getTools().map((tool) => tool.name); + const toolNames = catalog.getTools().map((tool) => tool.name); expect(toolNames).toContain("browser_click"); expect(toolNames).toContain("browser_screenshot"); expect(toolNames).toContain("browser_act"); @@ -69,7 +69,7 @@ describe("buildCuaHarness", () => { content: "Use the demo workflow.", filePath: join(cwd, "demo.md"), }; - const harness = buildCuaHarness({ + const { harness } = buildCuaHarness({ cwd, client: kernel.client, browser: kernel.browser, @@ -96,7 +96,7 @@ describe("buildCuaHarness", () => { const cwd = mkdtempSync(join(tmpdir(), "cua-cli-harness-")); const kernel = createFakeKernelEnvironment(); const session = await new InMemorySessionRepo().create(); - const harness = buildCuaHarness({ + const { harness } = buildCuaHarness({ cwd, client: kernel.client, browser: kernel.browser, @@ -123,7 +123,7 @@ describe("buildCuaHarness", () => { const cwd = mkdtempSync(join(tmpdir(), "cua-cli-harness-")); const kernel = createFakeKernelEnvironment(); const session = await new InMemorySessionRepo().create(); - const harness = buildCuaHarness({ + const { harness } = buildCuaHarness({ cwd, client: kernel.client, browser: kernel.browser, @@ -147,7 +147,7 @@ describe("buildCuaHarness", () => { const cwd = mkdtempSync(join(tmpdir(), "cua-cli-harness-")); const kernel = createFakeKernelEnvironment(); const session = await new InMemorySessionRepo().create(); - const harness = buildCuaHarness({ + const { harness } = buildCuaHarness({ cwd, client: kernel.client, browser: kernel.browser, diff --git a/packages/cli/test/kimi-reasoning-payload.test.ts b/packages/cli/test/kimi-reasoning-payload.test.ts index 43fd623..d84fcef 100644 --- a/packages/cli/test/kimi-reasoning-payload.test.ts +++ b/packages/cli/test/kimi-reasoning-payload.test.ts @@ -29,7 +29,7 @@ async function capturePayloads(apiKeyEnv: string, model: CuaModelRef): Promise sseResponse())); const kernel = createFakeKernelEnvironment(); - const harness = buildCuaHarness({ + const { harness } = buildCuaHarness({ cwd: mkdtempSync(join(tmpdir(), "cua-kimi-payload-")), client: kernel.client, browser: kernel.browser, diff --git a/packages/cli/test/tool-revalidation.test.ts b/packages/cli/test/tool-revalidation.test.ts index f1fdee6..8b8834c 100644 --- a/packages/cli/test/tool-revalidation.test.ts +++ b/packages/cli/test/tool-revalidation.test.ts @@ -5,7 +5,7 @@ import { buildTestHarness } from "./fixtures/harness"; /** * The `/tools` picker applies a selection of the model's tool menu via - * `harness.setTools()`. These tests pin the behavior the picker relies on: + * `catalog.setTools()`. These tests pin the behavior the picker relies on: * compile-and-validate happens before any mutation, so a rejected selection * leaves the live catalog untouched. */ @@ -19,21 +19,21 @@ describe("/tools selection revalidation", () => { const dropped = items.find((item) => item.group === "native" && item.available)!; const next = baseline.filter((tool) => toolKey(tool) !== dropped.key); - await fixture.harness.setTools(next); - expect(fixture.harness.getTools().map(toolKey)).toEqual(next.map(toolKey)); - expect(fixture.harness.getTools().map(toolKey)).not.toContain(dropped.key); + await fixture.catalog.setTools(next); + expect(fixture.catalog.getTools().map(toolKey)).toEqual(next.map(toolKey)); + expect(fixture.catalog.getTools().map(toolKey)).not.toContain(dropped.key); }); it("rejects a duplicated tool name and leaves the catalog unchanged", async () => { const modelRef = "google:gemini-3.6-flash"; const baseline = [...defaultInteractionTools(modelRef), ...defaultApplicationTools()]; const fixture = await buildTestHarness({ turns: [], modelRef, tools: baseline }); - const before = fixture.harness.getTools().map(toolKey); + const before = fixture.catalog.getTools().map(toolKey); const [first] = baseline; - await expect(fixture.harness.setTools([...baseline, first!])).rejects.toThrow(/requested more than once/); + await expect(fixture.catalog.setTools([...baseline, first!])).rejects.toThrow(/requested more than once/); // Atomicity: the failed compile must not have mutated live state. - expect(fixture.harness.getTools().map(toolKey)).toEqual(before); + expect(fixture.catalog.getTools().map(toolKey)).toEqual(before); }); it("accepts dropping the whole Google native group", async () => { @@ -45,8 +45,8 @@ describe("/tools selection revalidation", () => { const nativeKeys = new Set(items.filter((item) => item.group === "native").flatMap((item) => item.tools.map(toolKey))); const next = baseline.filter((tool) => !nativeKeys.has(toolKey(tool))); - await fixture.harness.setTools(next); - expect(fixture.harness.getTools().map(toolKey)).toEqual(next.map(toolKey)); + await fixture.catalog.setTools(next); + expect(fixture.catalog.getTools().map(toolKey)).toEqual(next.map(toolKey)); }); it("accepts an empty selection (text-only agent)", async () => { @@ -54,8 +54,8 @@ describe("/tools selection revalidation", () => { const baseline = [...defaultInteractionTools(modelRef), ...defaultApplicationTools()]; const fixture = await buildTestHarness({ turns: [], modelRef, tools: baseline }); - await fixture.harness.setTools([]); - expect(fixture.harness.getTools()).toEqual([]); + await fixture.catalog.setTools([]); + expect(fixture.catalog.getTools()).toEqual([]); }); it("recomposes the baseline after a model switch across providers", async () => { @@ -71,10 +71,10 @@ describe("/tools selection revalidation", () => { // Mirrors switchModel(): the new model and its interaction catalog compile // as one pair, because the selected tools decide the transport. - await fixture.harness.setModelAndTools(to, [...defaultInteractionTools(to), ...application]); + await fixture.catalog.setModelAndTools(to, [...defaultInteractionTools(to), ...application]); const expected = [...defaultInteractionTools(to), ...application].map(toolKey); - expect(fixture.harness.getTools().map(toolKey)).toEqual(expected); + expect(fixture.catalog.getTools().map(toolKey)).toEqual(expected); expect(fixture.harness.getModel().provider).toBe("anthropic"); }); @@ -91,7 +91,7 @@ describe("/tools selection revalidation", () => { expect(playwright.available).toBe(true); const enabled = new Set([...selectedKeys(items, baseline), playwright.key]); - await fixture.harness.setTools(toolsForSelection(items, enabled)); - expect(fixture.harness.getTools().map((tool) => tool.name)).toContain("playwright_execute"); + await fixture.catalog.setTools(toolsForSelection(items, enabled)); + expect(fixture.catalog.getTools().map((tool) => tool.name)).toContain("playwright_execute"); }); }); diff --git a/packages/cli/test/tui.fixture.test.ts b/packages/cli/test/tui.fixture.test.ts index dbe23ad..94a8fd4 100644 --- a/packages/cli/test/tui.fixture.test.ts +++ b/packages/cli/test/tui.fixture.test.ts @@ -10,7 +10,7 @@ import { describeMenu } from "../src/tui/tool-selection"; /** * Drive the interactive TUI through ptywright with a scripted provider sitting - * below the real {@link CuaAgentHarness}. The runner script ({@link tuiRunnerPath}) + * below the real harness. The runner script ({@link tuiRunnerPath}) * registers the scripted provider, assembles the harness via the production * {@link buildCuaHarness}, and starts {@link runInteractive}. Each test case * spawns a fresh process with its own per-scenario fixture JSON so the