From 70254ab30cb938478d07a9acecef36cf276350d5 Mon Sep 17 00:00:00 2001 From: rgarcia <72655+rgarcia@users.noreply.github.com> Date: Fri, 14 Aug 2026 03:47:12 +0000 Subject: [PATCH 1/2] Retire CuaAgent and CuaAgentHarness in favor of the handle The CLI now builds a stock pi AgentHarness from attach() and keeps the live (model, tools) selection in its own CuaCliCatalog. With no class owning mutable agent state, setTools/setModel/setModelAndTools and the tool manager's prepare/commit machinery go away: a change compiles a new pair and applies it. compile() throws before anything reaches pi and apply() restores the previous pair if pi rejects the new one, so the atomicity the mutation methods provided is preserved without a second copy of the state. The handle now owns one Models collection serving whichever pair is active. pi fixes models at construction while the headers, payload transforms and tool plan it carries are per-catalog, so a per-compile collection would have left a swapped harness streaming the first catalog's plan for the rest of the session. --- README.md | 6 +- docs/agent-tool-configuration-spec.md | 7 +- docs/architecture.md | 34 +- docs/cua-cli-harness-migration.md | 6 +- packages/agent/CHANGELOG.md | 82 +-- packages/agent/README.md | 129 ++-- packages/agent/src/agent.ts | 495 -------------- packages/agent/src/attach.ts | 125 +++- packages/agent/src/index.ts | 2 - packages/agent/src/resources.ts | 2 +- packages/agent/src/tool-manager.ts | 234 +------ packages/agent/test/agent.test.ts | 634 ------------------ packages/agent/test/attach-session.test.ts | 313 +++++++++ packages/agent/test/attach.test.ts | 47 +- packages/agent/test/e2e.live.test.ts | 73 +- packages/agent/test/harness-context.test.ts | 37 +- .../agent/test/openai-deferred-tools.test.ts | 109 --- .../agent/test/published-declarations.test.ts | 48 +- packages/agent/test/tool-manager.test.ts | 67 +- packages/ai/README.md | 4 +- packages/ai/src/providers.ts | 2 +- packages/cli/CHANGELOG.md | 5 + packages/cli/README.md | 2 +- packages/cli/src/cli-harness.ts | 15 +- packages/cli/src/harness.ts | 110 ++- packages/cli/src/tui/main.ts | 14 +- packages/cli/test/fixtures/harness.ts | 8 +- .../cli/test/fixtures/tui-fixture-runner.ts | 3 +- packages/cli/test/harness-assembly.test.ts | 12 +- .../cli/test/kimi-reasoning-payload.test.ts | 2 +- packages/cli/test/tool-revalidation.test.ts | 30 +- packages/cli/test/tui.fixture.test.ts | 2 +- 32 files changed, 895 insertions(+), 1764 deletions(-) delete mode 100644 packages/agent/src/agent.ts delete mode 100644 packages/agent/test/agent.test.ts create mode 100644 packages/agent/test/attach-session.test.ts delete mode 100644 packages/agent/test/openai-deferred-tools.test.ts diff --git a/README.md b/README.md index 7970f58..d030e31 100644 --- a/README.md +++ b/README.md @@ -41,8 +41,8 @@ packages/ ``` **Building your own agent? Start here:** [`packages/agent`](packages/agent) -(`@onkernel/cua-agent`) — `CuaAgent`/`CuaAgentHarness` run the full -computer-use loop against a Kernel browser. It sits on +(`@onkernel/cua-agent`) — `attach()` binds a Kernel browser and compiles a +(model, tools) pair into plain pi objects you drive yourself. It sits on [`packages/ai`](packages/ai) (`@onkernel/cua-ai`), the model layer with the pi-ai model catalog, canonical tool schemas, and per-provider adapters on top of pi-ai; reach for cua-ai directly only when you bring your @@ -148,7 +148,7 @@ cua -p -o jsonl "open example.com and tell me the heading" `pi-agent-core`'s `Agent`/`AgentHarness`. It materializes the caller's exact catalog over one shared resource pool and executes canonical actions through Kernel's computer API or a raw-CDP browser executor. -3. **CLI** — `@onkernel/cua-cli` assembles a `CuaAgentHarness` from +3. **CLI** — `@onkernel/cua-cli` assembles a pi `AgentHarness` from command-line flags, env-var-based API keys, a `JsonlSessionRepo` for transcripts, and pi skills; renders the result either as plain text (`--print`), JSONL events (`-o jsonl`), or an interactive pi-tui diff --git a/docs/agent-tool-configuration-spec.md b/docs/agent-tool-configuration-spec.md index 4125edb..53d96bc 100644 --- a/docs/agent-tool-configuration-spec.md +++ b/docs/agent-tool-configuration-spec.md @@ -1,6 +1,11 @@ # Agent Tool Configuration -**Status:** Implemented +**Status:** Superseded by `attach()` (2026-08-14). The tool-array-as-single-source-of-truth +rule it establishes still holds; what changed is where the array lives. `CuaAgent` and +`CuaAgentHarness` are gone, and a caller now compiles a (model, tools) pair through +`attach()` and owns the selection itself, so the `setTools()`/`setModel()` mutation surface +and its in-tool `executionMode: "sequential"` guard described below no longer exist. +Retained as the record of why the tool array is explicit and required. **Scope:** `@onkernel/cua-agent` and the tool-building surface in `@onkernel/cua-ai` **Compatibility:** Not a goal; these packages are alpha and may make breaking API changes. diff --git a/docs/architecture.md b/docs/architecture.md index 66dcde9..8cbd5d6 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -82,15 +82,17 @@ before a model request. ## Dynamic catalogs -`CuaAgent` and `CuaAgentHarness` use composition around pi and expose: +`attach()` returns a handle; `compile()` turns a (model, tools) pair into plain +pi objects, and `apply()` swaps a running harness onto a new pair: ```ts -agent.getTools(); -agent.setTools(nextTools); -agent.setModel(nextModel); +const kb = attach({ browser, client }); +const compiled = kb.compile({ model, tools }); +await compiled.apply(harness); ``` -`setTools()` recompiles atomically before mutating pi state. Existing tool +Nothing mutates in place: a change compiles a new pair, and `compile()` throws +before anything reaches pi. Existing tool identity with a changed schema, executor, or coordinates counts as a real replacement. Additions made from inside a running tool are recorded in pi's Anthropic-compatible `addedToolNames` marker only when that provider/model can @@ -200,10 +202,14 @@ derives a transport that the rest of the selection must be compatible with. Callers rebuild the menu after each staged change rather than caching a per-tool verdict. -`CuaAgent` and `CuaAgentHarness` push the compiled `catalog.model` into pi on -every construction and on every `setTools()`/`setModel()`, so the derived -transport applies uniformly regardless of which mutation path selected the -tools. +`apply()` pushes the compiled `catalog.model` into pi alongside its tools, and +only when the derived transport actually moved, so a tools-only change records +no model change while a transport-moving change records exactly one. + +pi fixes a harness's `models` at construction, but the headers, payload +transforms, and incoming tool plan it applies are per-catalog. The handle +therefore owns one `Models` collection that serves whichever pair was last +activated; `activate()` is what redirects it, and `apply()` calls it. Generated payload processing has fixed order: model preparation, tool serialization, provider fields, then the caller's `onPayload` hook. @@ -221,7 +227,9 @@ serialization, provider fields, then the caller's `onPayload` hook. - Anthropic's native browser tool when the model supports it; - Google's native browser action set; 3. creates and retains its own application-level coding-tool list; -4. passes the complete list to `CuaAgentHarness`; +4. compiles the complete list through the handle and hands the result to a + stock pi `AgentHarness`, retaining the selection in `CuaCliCatalog` so + `/model` and `/tools` can recompile it; 5. builds a caller-owned prompt from loaded skills and context files; 6. uses one `Session` for transcript persistence and resume; 7. exposes `cua act ''` as a model-free path to the same `browser_act` @@ -257,14 +265,14 @@ switch — run through that one queue, because each suspends across several `setTools()`/`setModel()` calls. Without it an apply could land between a switch's `setModel()` and its final `setTools()` and fail its compile against the wrong provider. Selectors also refuse to open mid-turn: the agent's -execution-scope guard only covers mutation from inside a tool's `execute`, so -this TUI-side check is what protects a streaming request. +compiled pair is immutable, so this TUI-side check is what keeps a swap from +landing mid-request. ## Per-turn flow ```text user prompt - -> CuaAgentHarness / pi agent loop + -> pi agent loop -> active identity-keyed catalog -> generated headers and payload transforms -> caller onPayload diff --git a/docs/cua-cli-harness-migration.md b/docs/cua-cli-harness-migration.md index a45c055..f53029b 100644 --- a/docs/cua-cli-harness-migration.md +++ b/docs/cua-cli-harness-migration.md @@ -1,8 +1,10 @@ # CUA CLI Harness Migration -**Status:** Completed and superseded. +**Status:** Historical (2026-08-14). The CLI now composes a stock pi `AgentHarness` from +`attach()` rather than `CuaAgentHarness`, which no longer exists. Retained as the record of +the print/action/interactive consolidation this describes. -The CLI now uses the shared `CuaAgentHarness` composition path for print, +The CLI uses one shared composition path for print, action, and TUI flows. Its current architecture—including explicit tool-list selection, coding-tool composition, sessions, skills, and rendering—is documented in [`architecture.md`](architecture.md#cli-composition). diff --git a/packages/agent/CHANGELOG.md b/packages/agent/CHANGELOG.md index 188fc2d..34b8c22 100644 --- a/packages/agent/CHANGELOG.md +++ b/packages/agent/CHANGELOG.md @@ -2,52 +2,56 @@ ## Unreleased +Breaking: `CuaAgent` and `CuaAgentHarness` are removed. cua-agent hands back +plain pi objects; the caller constructs the agent. + - Add `attach({ browser, client })`, returning a handle that compiles - (model, tools) pairs into plain pi objects: the model carrying the transport - its tools derive, executables materialized against the handle's browser pool, - a `Models` collection adding provider retry, required headers, the catalog's - payload transforms and the tool-result image bound, and an `install(harness)` - for the behaviors that are pi event handlers rather than constructor options. - The handle owns what actually persists — the Kernel client and browser, the - translator, the raw-CDP executor, ref and frame state — so a spec materializes - once across repeat compiles. -- `CuaAgent` and `CuaAgentHarness` are unchanged and now share their internals - with `attach()` rather than owning private copies. They are slated to retire - in favor of the handle. -- Add `CuaAgentHarness.setModelAndTools()`. A model switch that also swaps - interaction tools has to compile as one pair now that the selected tools - decide the transport: staging the two in sequence produces an intermediate - catalog whose derived transport differs from both the old and the new one, and - records a model change for a transport nothing ever streamed with. -- `CuaAgentHarness` no longer refuses a model ref that is absent from its - supplied `Models` collection: it falls back to the registry, and an id the - registry lacks is synthesized. -- The model streamed for a Google model now depends on which tools `CuaAgent` / - `CuaAgentHarness` were constructed or mutated with: selecting Google's native - browser toolset still compiles to the CUA-owned Interactions API, but a Google - model selected with only CDP browser tools now streams through pi's builtin - Google transport instead of always carrying the CUA-owned api. This applies - uniformly across construction, `setTools()`, and `setModel()`, since all three - feed the same compiled `catalog.model` into pi. -- Fix `setTools()` recompiling from the previously *compiled* model instead of - the caller's model selection: dropping a native toolset that had derived a - tool-selection-dependent api (e.g. Google's Interactions API) left subsequent - tools-only recompiles stuck on that api even though the new selection no - longer required it. `CuaToolManager` now recompiles tools-only changes from - the model input the caller last selected. -- `responseThreading` (`CuaAgentOptions`/`CuaAgentHarnessOptions`) no longer - affects OpenAI models: OpenAI now streams through pi-ai's builtin Responses - transport and its automatic prompt caching regardless of this flag. The option - still governs Google's `previous_response_id`-style continuation. + (model, tools) pairs into plain pi objects: `model` carrying the transport its + tools derive, `tools` / `agentTools` materialized against the handle's browser + pool, `models` adding provider retry, required headers, the catalog's payload + transforms and the tool-result image bound, `activate(harness)` for the + behaviors that are pi event handlers rather than constructor options, and + `apply(harness)` to swap a running harness onto a new pair. The handle owns + what actually persists — the Kernel client and browser, the translator, the + raw-CDP executor, ref and frame state — so a spec materializes once across + repeat compiles. +- `getTools()`, `setTools()`, `setModel()`, and `setModelAndTools()` are gone + with the classes. A change compiles a new pair and applies it, so the current + selection belongs to the caller and there is no second copy of it to drift. + `compile()` throws before anything reaches pi, and `apply()` restores the + previous pair if pi rejects the new one, so the atomicity those methods + provided is preserved. `apply()` sets the model only when the derived + transport actually moved, so a tools-only change records no model change. +- `CuaToolManager` is now immutable: one compiled pair per instance, with + `prepareTools`/`prepareModel`/`prepareModelAndTools`/`commit`/`getTools` and + the async-local execution scope removed. Removing the execution scope also + removes cache-preserving deferred tool addition: a tool that added tools mid + execution used to have those names recorded on its result as + `addedToolNames`, letting pi extend an OpenAI request without invalidating the + prompt-cache prefix. Nothing produced them outside that mutation path. The + transport still consumes `addedToolNames` on a transcript that carries them. +- The tool-result image bound, payload transforms, and required headers now + follow whichever pair is active rather than the one a harness was built with. + pi fixes `models` at construction while those are per-catalog, so one + collection per handle is what makes a swap possible at all. +- A model ref absent from a supplied `Models` collection falls back to the + registry, and an id the registry lacks is synthesized. +- The model streamed for a Google model depends on which tools it was compiled + with: selecting Google's native browser toolset compiles to the CUA-owned + Interactions API, while a Google model selected with only CDP browser tools + streams through pi's builtin Google transport. +- `responseThreading` (`CuaAttachOptions`) no longer affects OpenAI models: + OpenAI streams through pi-ai's builtin Responses transport and its automatic + prompt caching regardless of this flag. The option still governs Google's + `previous_response_id`-style continuation. - Exempt OpenAI's native computer tool from the tool-result image replay limit. Its `computer_call_output` items must each carry a screenshot, and stateless replay no longer leaves them in provider-stored state. Breaking: Tzafon and Yutori support is removed. -- Constructing a `CuaAgent` or `CuaAgentHarness` with a Tzafon or Yutori model - ref now fails to resolve the model, and `cua.providers.tzafon` / - `cua.providers.yutori` no longer exist. +- Compiling a Tzafon or Yutori model ref now fails to resolve the model, and + `cua.providers.tzafon` / `cua.providers.yutori` no longer exist. - Remove `CuaExecutionResources.viewport`. It only fed the removed catalog viewport option; the same value is still on `resources.browser.viewport`. diff --git a/packages/agent/README.md b/packages/agent/README.md index a020bfc..8f94c08 100644 --- a/packages/agent/README.md +++ b/packages/agent/README.md @@ -12,22 +12,31 @@ npm install @onkernel/cua-agent @onkernel/cua-ai @onkernel/sdk Requires Node 22.19 or newer, `KERNEL_API_KEY`, and the selected model provider's API key. -## `CuaAgent` +## `attach()` + +`attach()` binds a Kernel browser to CUA's execution resources and returns a +handle. `compile()` turns a (model, tools) pair into plain pi objects; you +construct whatever pi agent you want with them. There is no CUA agent class. ```ts import Kernel from "@onkernel/sdk"; import { cua } from "@onkernel/cua-ai"; -import { CuaAgent } from "@onkernel/cua-agent"; +import { Agent, attach } from "@onkernel/cua-agent"; const client = new Kernel({ apiKey: process.env.KERNEL_API_KEY! }); const browser = await client.browsers.create({ stealth: true }); +const kb = attach({ client, browser }); -const agent = new CuaAgent({ - client, - browser, +const { model, agentTools, models } = kb.compile({ + model: "anthropic:claude-opus-5", tools: cua.toolsets.browser(), +}); + +const agent = new Agent({ + streamFn: (selected, context, options) => models.streamSimple(selected, context, options), initialState: { - model: "anthropic:claude-opus-5", + model, + tools: [...agentTools], systemPrompt: "Inspect and interact with the page using the requested tools.", }, }); @@ -35,36 +44,54 @@ const agent = new CuaAgent({ try { await agent.prompt("Open example.com and report the heading."); } finally { + await kb.dispose(); await client.browsers.deleteByID(browser.session_id); } ``` -## `CuaAgentHarness` +The compiled `model` carries the transport its tools derive: selecting a +provider-native browser or computer surface can change `model.api`, so the pair +has to reach pi together. -Use the harness for session-backed transcripts, skills, prompt templates, -compaction, steering, and follow-ups: +## With pi's `AgentHarness` + +Use pi's harness for session-backed transcripts, skills, prompt templates, +compaction, steering, and follow-ups. `activate()` registers the behaviors CUA +owns that are pi event handlers rather than constructor options, and points the +handle's `models` at this catalog: ```ts -import { - CuaAgentHarness, - InMemorySessionRepo, -} from "@onkernel/cua-agent"; +import { AgentHarness, attach, InMemorySessionRepo } from "@onkernel/cua-agent"; import { cua } from "@onkernel/cua-ai"; -const repo = new InMemorySessionRepo(); -const session = await repo.create(); -const harness = new CuaAgentHarness({ - client, - browser, +const session = await new InMemorySessionRepo().create(); +const kb = attach({ client, browser }); +const compiled = kb.compile({ model: "openai:gpt-5.6-sol", tools: cua.toolsets.browser() }); + +const harness = new AgentHarness({ session, - model: "openai:gpt-5.6-sol", - tools: cua.toolsets.browser(), + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), systemPrompt: "Use the supplied browser tools.", }); +compiled.activate(harness); await harness.prompt("Find the pricing page."); ``` +To change the model or the tool list on a running harness, compile the new pair +and apply it: + +```ts +await kb.compile({ model: "google:gemini-3.6-flash", tools: cua.providers.google.toolsets.browser() }).apply(harness); +``` + +`apply()` moves the model and tools together, sets the model only when the +derived transport actually moved, and restores the previous pair if pi rejects +the new one. + The package re-exports pi-agent-core session, skill, prompt-template, compaction, and execution-environment primitives used with the harness. @@ -77,27 +104,34 @@ to every tool call: ```ts import { - CuaAgentHarness, + AgentHarness, + attach, NodeExecutionEnv, createBashTool, createReadTool, type ExecutionToolContext, } from "@onkernel/cua-agent"; -const harness = new CuaAgentHarness({ - client, - browser, - session, +const compiled = kb.compile({ model: "openai:gpt-5.6-sol", tools: [createReadTool(), createBashTool(), ...cua.toolsets.browser()], +}); +const harness = new AgentHarness({ + session, + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), toolContext: { env: new NodeExecutionEnv({ cwd: process.cwd() }) }, systemPrompt: "Use the supplied tools.", }); +compiled.activate(harness); ``` -CUA specs and plain pi `AgentTool`s are accepted too — they simply ignore the -context. The low-level `CuaAgent` stays context-free: its tools are ordinary -pi `AgentTool`s (`CuaAgentTool`). +Compile for the same context the harness delivers, so a later swap stays +type-compatible. CUA specs and plain pi `AgentTool`s are accepted too — they +simply ignore the context. `compiled.agentTools` is the context-free view for +the low-level `Agent`. ## Choosing tools @@ -150,24 +184,23 @@ the active credential cannot access `browser_20260701`. ## Dynamic catalogs -Both classes expose the same catalog controls: +Changing the model or the tool list compiles a new pair; nothing mutates in +place: ```ts -agent.getTools(); // copy of the exact requested inputs -agent.setTools(next); // atomic compile + replace -agent.setModel(nextModel); +const next = kb.compile({ model: nextModel, tools: nextTools }); +await next.apply(harness); ``` -The requested catalog is recompiled on every `setTools()` or `setModel()`. Duplicate identities, caller-visible name collisions, provider-normalized name -collisions, and incompatible model/tool combinations fail before state changes. +collisions, and incompatible model/tool combinations fail in `compile()`, before +anything reaches pi. `apply()` then moves the model and tools together and +restores the previous pair if pi rejects the new one. -Tools may call `setTools()` or `setModel()` while they execute, but only if they -declare `executionMode: "sequential"`; mutating the catalog from a tool that can -run in parallel is rejected. Eligible ordinary function-tool additions are -recorded for pi's deferred-loading protocol; additions outside a tool execution -are eager. Provider-native tools are always eager. Replacing an existing -identity's schema, executor, or coordinates is a real replacement. +The handle holds the current selection only in the sense that `models` serves +whichever pair was last activated; the *selection* itself belongs to the caller, +which is what removes the class of bug where a compiled pair and the live one +disagree. One shared execution-resource pool survives all catalog/model changes, so browser refs, tabs, connections, and translator state are not reset. @@ -222,7 +255,7 @@ const lookup = { }, }; -agent.setTools([lookup, ...cua.toolsets.browser()]); +kb.compile({ model, tools: [lookup, ...cua.toolsets.browser()] }); ``` Caller tools receive identity `caller.` through cua-ai's canonical @@ -230,17 +263,15 @@ Caller tools receive identity `caller.` through cua-ai's canonical fingerprint rules. `CuaAgentTool` is defined and exported by this package: cua-ai compiles declaration-only catalogs and never sees executors, while cua-agent projects caller `AgentTool`s into fresh declarations, joins compiled -entries back by identity, materializes each CUA spec exactly once per shared -execution-resource pool, and owns implementation identity for replacement -detection (a reused `execute` function keeps its identity across wrappers; a -new `execute` or freshly created spec object is a conservative replacement). +entries back by identity, and materializes each CUA spec exactly once per shared +execution-resource pool, so repeat compiles hand pi a stable implementation. ## Events and state -`CuaAgent` delegates pi's prompt/continue/steer/follow-up/abort lifecycle and -subscriptions. `CuaAgentHarness` delegates harness events and session APIs. -`CuaAgent.state.tools` is the active materialized list; use `getTools()` for a -copy of the exact requested catalog. +The agent is pi's, so its lifecycle, events, and session APIs are pi's too. +What CUA adds on top is `activate()`: failed tool results are marked, a turn's +remaining calls are blocked after one fails, and an empty successful response +can be followed up. It returns a release. ## Development diff --git a/packages/agent/src/agent.ts b/packages/agent/src/agent.ts deleted file mode 100644 index 6e38b5f..0000000 --- a/packages/agent/src/agent.ts +++ /dev/null @@ -1,495 +0,0 @@ -import { - Agent, - AgentHarness, - type AgentEvent, - type AgentHarnessEvent, - type AgentHarnessEventResultMap, - type AgentHarnessOptions, - type AgentHarnessOwnEvent, - type AgentHarnessToolContextSource, - type AgentHarnessResources, - type AgentHarnessStreamOptions, - type AgentHarnessTool, - type AgentMessage, - type AgentOptions, - type AgentTool, - type NavigateTreeResult, - type PromptTemplate, - type QueueMode, - type Session, - type Skill, - type StreamFn, - type ThinkingLevel, -} from "@earendil-works/pi-agent-core"; -import { - type Api, - type Context, - cuaModels, - type CuaIncomingToolPlan, - type CuaModelRef, - getCuaModel, - parseCuaModelRef, - type CuaSimpleStreamOptions, - getCuaEnvApiKey, - type ImageContent, - type Model, - type Models, - type SimpleStreamOptions, -} from "@onkernel/cua-ai"; -import type Kernel from "@onkernel/sdk"; -import { - resolveProviderRetryPolicy, - type CuaRetryOptions, - withProviderRetry, - withProviderRetryModels, -} from "./provider-retry"; -import { CuaExecutionResources, type CuaExecutionDetails } from "./resources"; -import { CuaToolManager, type CuaAgentTool, type CuaHarnessTool } from "./tool-manager"; -import { - type CuaEmptyResponseRecoveryOptions, - type CuaModelInput, - defaultCuaStream, - hasExecutionError, - isEmptyAssistantResponse, - modelTransportChanged, - projectToolResultImages, - requiredImageToolNames, - resolveEmptyResponseRecovery, - resolveModelFromCollection, - resolveResponseThreading, - resolveToolResultImageReplayLimit, - type ToolResultImageReplayLimit, - turnFailureStopMessage, - withCatalogModels, -} from "./attach"; -import type { KernelBrowser } from "./translator/translator"; - -/** Mutable conversation state exposed by {@link CuaAgent}. */ -export interface CuaAgentState { - systemPrompt: string; - model: Model | CuaModelRef; - thinkingLevel: ThinkingLevel; - messages: AgentMessage[]; - readonly tools: readonly AgentTool[]; - readonly isStreaming: boolean; - readonly streamingMessage?: AgentMessage; - readonly pendingToolCalls: ReadonlySet; - readonly errorMessage?: string; -} - -type CuaAgentInitialState = Omit, "model" | "tools"> & { - model: CuaModelInput; -}; - -/** Construction options for {@link CuaAgent}, including its exact caller-owned tool catalog. */ -export type CuaAgentOptions = Omit & { - browser: KernelBrowser; - client: Kernel; - tools: readonly CuaAgentTool[]; - initialState: CuaAgentInitialState; - /** Defaults to streaming through the shared {@link cuaModels} collection. */ - streamFn?: AgentOptions["streamFn"]; - emptyResponseRecovery?: CuaEmptyResponseRecoveryOptions; - toolResultImageReplayLimit?: ToolResultImageReplayLimit; - /** Governs Google's `previous_response_id`-style continuation. Every other provider streams through pi's transports and their automatic prompt caching regardless of this flag. Defaults to `true`. */ - responseThreading?: boolean; - retry?: CuaRetryOptions; -}; - -type CuaAgentHarnessOptionsBase< - TContext extends object | undefined, - TSkill extends Skill, - TPromptTemplate extends PromptTemplate, -> = Omit< - AgentHarnessOptions>, - "activeToolNames" | "model" | "models" | "tools" | "toolContext" | "retry" -> & { - browser: KernelBrowser; - client: Kernel; - model: CuaModelInput; - models?: Models; - tools: readonly CuaHarnessTool[]; - onPayload?: SimpleStreamOptions["onPayload"]; - emptyResponseRecovery?: CuaEmptyResponseRecoveryOptions; - toolResultImageReplayLimit?: ToolResultImageReplayLimit; - /** Governs Google's `previous_response_id`-style continuation. Every other provider streams through pi's transports and their automatic prompt caching regardless of this flag. Defaults to `true`. */ - responseThreading?: boolean; - retry?: CuaRetryOptions; -}; - -/** - * Construction options for {@link CuaAgentHarness}, including its exact - * caller-owned tool catalog. Mirrors pi's `AgentHarnessOptions` generic order: - * the tool context first, then skill and prompt-template resource types. The - * supplied `toolContext` is forwarded to pi untouched, and every executable - * tool receives it on each call. - */ -export type CuaAgentHarnessOptions< - TContext extends object | undefined = undefined, - TSkill extends Skill = Skill, - TPromptTemplate extends PromptTemplate = PromptTemplate, -> = CuaAgentHarnessOptionsBase & ([TContext] extends [undefined] ? { - /** Context-free harnesses do not need a tool context. */ - toolContext?: undefined; -} : { - /** Static context or zero-argument context provider resolved for each turn snapshot. */ - toolContext: AgentHarnessToolContextSource; -}); - -/** Pi Agent behavior with an explicit, identity-keyed CUA tool catalog. */ -export class CuaAgent { - private readonly coreAgent: Agent; - private readonly tools: CuaToolManager; - private runtimeDirty = false; - private readonly stateView: CuaAgentState; - private emptyResponseRecoveryAttempts = 0; - - constructor(options: CuaAgentOptions) { - const { - browser, - client, - tools: requestedTools, - initialState, - onPayload, - streamFn, - prepareNextTurn, - prepareNextTurnWithContext, - transformContext, - afterToolCall, - beforeToolCall, - emptyResponseRecovery, - toolResultImageReplayLimit, - responseThreading, - retry, - ...agentOptions - } = options; - const recovery = resolveEmptyResponseRecovery(emptyResponseRecovery); - const imageReplayLimit = resolveToolResultImageReplayLimit(toolResultImageReplayLimit); - const useResponseThreading = resolveResponseThreading(responseThreading); - const resources = new CuaExecutionResources({ browser, client }); - const manager = new CuaToolManager(resources, initialState.model, requestedTools); - const retryingStream = withProviderRetry(streamFn ?? defaultCuaStream, resolveProviderRetryPolicy(retry)); - const streamWithCatalog: StreamFn = (model, context, streamOptions) => { - const catalog = manager.catalog; - const generatedOnPayload = async (payload: unknown, selectedModel: Model) => { - const generated = await catalog.payload.apply(payload, selectedModel); - return onPayload ? (await onPayload(generated, selectedModel)) ?? generated : generated; - }; - const cuaOptions: CuaSimpleStreamOptions = { - ...streamOptions, - headers: catalog.headers.merge(streamOptions?.headers), - onPayload: generatedOnPayload, - disableResponseThreading: !useResponseThreading, - cuaIncomingToolPlan: catalog.incoming, - }; - return retryingStream(model, context, cuaOptions); - }; - const failedTurns = new WeakSet(); - let core!: Agent; - core = new Agent({ - ...agentOptions, - getApiKey: agentOptions.getApiKey ?? getCuaEnvApiKey, - streamFn: streamWithCatalog, - beforeToolCall: async (context, signal) => { - const stopMessage = turnFailureStopMessage(manager); - if (stopMessage && failedTurns.has(context.assistantMessage)) return { block: true, reason: stopMessage }; - return beforeToolCall?.(context, signal); - }, - afterToolCall: async (context, signal) => { - const result = await afterToolCall?.(context, signal); - const forcedError = hasExecutionError(result?.details ?? context.result.details); - if ((result?.isError ?? context.isError) || forcedError) failedTurns.add(context.assistantMessage); - return forcedError ? { ...result, isError: true } : result; - }, - transformContext: async (messages, signal) => projectToolResultImages( - transformContext ? await transformContext(messages, signal) : messages, - imageReplayLimit, - requiredImageToolNames(manager.catalog.incoming), - ), - prepareNextTurnWithContext: async (context, signal) => { - const update = prepareNextTurnWithContext - ? await prepareNextTurnWithContext(context, signal) - : await prepareNextTurn?.(signal); - if (update?.model) this.setModel(update.model as CuaModelInput); - if (!update && !this.runtimeDirty) return undefined; - this.runtimeDirty = false; - return { - ...update, - model: core.state.model, - context: { - ...(update?.context ?? context.context), - tools: core.state.tools.slice(), - }, - }; - }, - initialState: { - ...initialState, - model: manager.catalog.model, - tools: manager.agentTools(), - systemPrompt: initialState.systemPrompt ?? "", - }, - }); - this.coreAgent = core; - this.tools = manager; - this.stateView = this.createStateView(); - - if (recovery && recovery.maxAttempts > 0) { - this.subscribe((event, signal) => { - if (event.type === "agent_start") { - this.emptyResponseRecoveryAttempts = 0; - return; - } - if (event.type !== "turn_end" || !isEmptyAssistantResponse(event.message)) return; - if (signal.aborted || this.emptyResponseRecoveryAttempts >= recovery.maxAttempts || this.hasQueuedMessages()) return; - this.followUp({ role: "user", content: [{ type: "text", text: recovery.followUp }], timestamp: Date.now() }); - this.emptyResponseRecoveryAttempts += 1; - }); - } - } - - get state(): CuaAgentState { - return this.stateView; - } - - getTools(): readonly CuaAgentTool[] { - return this.tools.getTools(); - } - - setTools(tools: readonly CuaAgentTool[]): void { - const prepared = this.tools.prepareTools(tools); - this.coreAgent.state.model = prepared.catalog.model; - this.coreAgent.state.tools = this.tools.agentTools(prepared); - this.tools.commit(prepared); - this.runtimeDirty = true; - } - - getModel(): Model { - return this.tools.catalog.model; - } - - setModel(model: CuaModelInput): void { - const prepared = this.tools.prepareModel(model); - this.coreAgent.state.model = prepared.catalog.model; - this.coreAgent.state.tools = this.tools.agentTools(prepared); - this.tools.commit(prepared); - this.runtimeDirty = true; - } - - prompt(...args: Parameters): Promise { - return this.coreAgent.prompt(...args); - } - - continue(): Promise { return this.coreAgent.continue(); } - steer(message: AgentMessage): void { this.coreAgent.steer(message); } - followUp(message: AgentMessage): void { this.coreAgent.followUp(message); } - clearSteeringQueue(): void { this.coreAgent.clearSteeringQueue(); } - clearFollowUpQueue(): void { this.coreAgent.clearFollowUpQueue(); } - clearAllQueues(): void { this.coreAgent.clearAllQueues(); } - hasQueuedMessages(): boolean { return this.coreAgent.hasQueuedMessages(); } - abort(): void { this.coreAgent.abort(); } - waitForIdle(): Promise { return this.coreAgent.waitForIdle(); } - reset(): void { this.coreAgent.reset(); } - subscribe(listener: (event: AgentEvent, signal: AbortSignal) => Promise | void): () => void { return this.coreAgent.subscribe(listener); } - get signal(): AbortSignal | undefined { return this.coreAgent.signal; } - get steeringMode(): QueueMode { return this.coreAgent.steeringMode; } - set steeringMode(mode: QueueMode) { this.coreAgent.steeringMode = mode; } - get followUpMode(): QueueMode { return this.coreAgent.followUpMode; } - set followUpMode(mode: QueueMode) { this.coreAgent.followUpMode = mode; } - - async dispose(): Promise { - this.abort(); - await this.waitForIdle(); - await this.tools.resources.dispose(); - } - - private createStateView(): CuaAgentState { - const owner = this; - return Object.defineProperties({}, { - systemPrompt: { enumerable: true, get: () => owner.coreAgent.state.systemPrompt, set: (value: string) => { owner.coreAgent.state.systemPrompt = value; } }, - model: { enumerable: true, get: () => owner.getModel(), set: (value: CuaModelInput) => owner.setModel(value) }, - thinkingLevel: { enumerable: true, get: () => owner.coreAgent.state.thinkingLevel, set: (value: ThinkingLevel) => { owner.coreAgent.state.thinkingLevel = value; } }, - messages: { enumerable: true, get: () => owner.coreAgent.state.messages, set: (value: AgentMessage[]) => { owner.coreAgent.state.messages = value; } }, - tools: { enumerable: true, get: () => owner.coreAgent.state.tools.slice() }, - isStreaming: { enumerable: true, get: () => owner.coreAgent.state.isStreaming }, - streamingMessage: { enumerable: true, get: () => owner.coreAgent.state.streamingMessage }, - pendingToolCalls: { enumerable: true, get: () => owner.coreAgent.state.pendingToolCalls }, - errorMessage: { enumerable: true, get: () => owner.coreAgent.state.errorMessage }, - }) as CuaAgentState; - } -} - -/** Pi AgentHarness behavior through composition, without inherited active-tool APIs. */ -export class CuaAgentHarness< - TContext extends object | undefined = undefined, - TSkill extends Skill = Skill, - TPromptTemplate extends PromptTemplate = PromptTemplate, -> { - readonly models: Models; - private readonly coreHarness: AgentHarness>; - private readonly tools: CuaToolManager>; - private emptyResponseRecoveryAttempts = 0; - private hasPendingQueue = false; - private toolTurnFailed = false; - - constructor(options: CuaAgentHarnessOptions) { - const { - browser, - client, - model, - models, - tools: requestedTools, - onPayload, - emptyResponseRecovery, - toolResultImageReplayLimit, - responseThreading, - retry, - ...harnessOptions - } = options; - const recovery = resolveEmptyResponseRecovery(emptyResponseRecovery); - const imageReplayLimit = resolveToolResultImageReplayLimit(toolResultImageReplayLimit); - const useResponseThreading = resolveResponseThreading(responseThreading); - const resources = new CuaExecutionResources({ browser, client }); - const retrying = withProviderRetryModels(models ?? cuaModels(), resolveProviderRetryPolicy(retry)); - const manager = new CuaToolManager>(resources, model, requestedTools, (ref) => resolveModelFromCollection(ref, retrying)); - const catalogModels = withCatalogModels(retrying, manager, imageReplayLimit, useResponseThreading); - const materialized = manager.harnessTools(); - // A generic TContext leaves pi's conditional toolContext unverifiable - // here; the spread forwards the caller's toolContext verbatim. - const core = new AgentHarness>({ - ...harnessOptions, - model: manager.catalog.model, - models: catalogModels, - tools: materialized, - activeToolNames: materialized.map((tool) => tool.name), - } as AgentHarnessOptions>); - this.coreHarness = core; - this.tools = manager; - this.models = core.models; - - core.on("tool_result", (event) => hasExecutionError(event.details) ? { isError: true } : undefined); - core.on("tool_call", () => this.toolTurnFailed && turnFailureStopMessage(manager) - ? { block: true, reason: turnFailureStopMessage(manager) } - : undefined); - core.subscribe((event) => { - if (event.type === "message_end" && event.message.role === "assistant") this.toolTurnFailed = false; - else if (event.type === "tool_execution_end" && event.isError) this.toolTurnFailed = true; - else if (event.type === "queue_update") this.hasPendingQueue = event.steer.length > 0 || event.followUp.length > 0; - }); - core.on("before_agent_start", () => { - this.toolTurnFailed = false; - return undefined; - }); - if (onPayload) { - core.on("before_provider_payload", async ({ model: selectedModel, payload }) => ({ - payload: (await onPayload(payload, selectedModel)) ?? payload, - })); - } - if (recovery && recovery.maxAttempts > 0) { - core.on("before_agent_start", () => { - this.emptyResponseRecoveryAttempts = 0; - this.hasPendingQueue = false; - return undefined; - }); - core.subscribe(async (event, signal) => { - if (event.type !== "turn_end" || !isEmptyAssistantResponse(event.message)) return; - if (signal?.aborted || this.emptyResponseRecoveryAttempts >= recovery.maxAttempts || this.hasPendingQueue) return; - await core.followUp(recovery.followUp); - this.emptyResponseRecoveryAttempts += 1; - }); - } - } - - getTools(): readonly CuaHarnessTool[] { return this.tools.getTools(); } - - async setTools(tools: readonly CuaHarnessTool[]): Promise { - const previousModel = this.tools.catalog.model; - const previousTools = this.tools.harnessTools(); - const prepared = this.tools.prepareTools(tools); - const materialized = this.tools.harnessTools(prepared); - const transportChanged = modelTransportChanged(previousModel, prepared.catalog.model); - try { - if (transportChanged) await this.coreHarness.setModel(prepared.catalog.model); - await this.coreHarness.setTools(materialized, materialized.map((tool) => tool.name)); - } catch (error) { - if (transportChanged) await this.coreHarness.setModel(previousModel); - await this.coreHarness.setTools(previousTools, previousTools.map((tool) => tool.name)); - throw error; - } - this.tools.commit(prepared); - } - - getModel(): Model { return this.tools.catalog.model; } - - async setModel(model: CuaModelInput): Promise { - const previousModel = this.tools.catalog.model; - const previousTools = this.tools.harnessTools(); - const prepared = this.tools.prepareModel(model); - const materialized = this.tools.harnessTools(prepared); - try { - await this.coreHarness.setModel(prepared.catalog.model); - await this.coreHarness.setTools(materialized, materialized.map((tool) => tool.name)); - } catch (error) { - await this.coreHarness.setModel(previousModel); - await this.coreHarness.setTools(previousTools, previousTools.map((tool) => tool.name)); - throw error; - } - this.tools.commit(prepared); - } - - /** - * Select a model and its tool list in one compile. A model switch that also - * swaps interaction tools would otherwise stage through an intermediate - * catalog whose derived transport differs from both the old and the new one, - * recording a `model_change` for a transport nothing ever streamed with. - */ - async setModelAndTools(model: CuaModelInput, tools: readonly CuaHarnessTool[]): Promise { - const previousModel = this.tools.catalog.model; - const previousTools = this.tools.harnessTools(); - const prepared = this.tools.prepareModelAndTools(model, tools); - const materialized = this.tools.harnessTools(prepared); - try { - await this.coreHarness.setModel(prepared.catalog.model); - await this.coreHarness.setTools(materialized, materialized.map((tool) => tool.name)); - } catch (error) { - await this.coreHarness.setModel(previousModel); - await this.coreHarness.setTools(previousTools, previousTools.map((tool) => tool.name)); - throw error; - } - this.tools.commit(prepared); - } - - prompt(text: string, options?: { images?: ImageContent[] }) { return this.coreHarness.prompt(text, options); } - skill(name: string, additionalInstructions?: string) { return this.coreHarness.skill(name, additionalInstructions); } - promptFromTemplate(name: string, args?: string[]) { return this.coreHarness.promptFromTemplate(name, args); } - steer(text: string, options?: { images?: ImageContent[] }) { return this.coreHarness.steer(text, options); } - followUp(text: string, options?: { images?: ImageContent[] }) { return this.coreHarness.followUp(text, options); } - nextTurn(text: string, options?: { images?: ImageContent[] }) { return this.coreHarness.nextTurn(text, options); } - appendMessage(message: AgentMessage) { return this.coreHarness.appendMessage(message); } - compact(customInstructions?: string) { return this.coreHarness.compact(customInstructions); } - navigateTree(targetId: string, options?: Parameters[1]): Promise { return this.coreHarness.navigateTree(targetId, options); } - getThinkingLevel(): ThinkingLevel { return this.coreHarness.getThinkingLevel(); } - setThinkingLevel(level: ThinkingLevel): Promise { return this.coreHarness.setThinkingLevel(level); } - getSteeringMode(): QueueMode { return this.coreHarness.getSteeringMode(); } - setSteeringMode(mode: QueueMode): Promise { return this.coreHarness.setSteeringMode(mode); } - getFollowUpMode(): QueueMode { return this.coreHarness.getFollowUpMode(); } - setFollowUpMode(mode: QueueMode): Promise { return this.coreHarness.setFollowUpMode(mode); } - getResources(): AgentHarnessResources { return this.coreHarness.getResources(); } - setResources(resources: AgentHarnessResources): Promise { return this.coreHarness.setResources(resources); } - getStreamOptions(): AgentHarnessStreamOptions { return this.coreHarness.getStreamOptions(); } - setStreamOptions(options: AgentHarnessStreamOptions): Promise { return this.coreHarness.setStreamOptions(options); } - abort() { return this.coreHarness.abort(); } - waitForIdle(): Promise { return this.coreHarness.waitForIdle(); } - subscribe(listener: (event: AgentHarnessEvent, signal?: AbortSignal) => Promise | void) { return this.coreHarness.subscribe(listener); } - - on( - type: TType, - handler: (event: Extract) => Promise | AgentHarnessEventResultMap[TType], - ): () => void { - return this.coreHarness.on(type, handler); - } - - async dispose(): Promise { - await this.abort(); - await this.tools.resources.dispose(); - } -} diff --git a/packages/agent/src/attach.ts b/packages/agent/src/attach.ts index faefc04..e632115 100644 --- a/packages/agent/src/attach.ts +++ b/packages/agent/src/attach.ts @@ -63,17 +63,30 @@ export interface CuaCompiled { /** Same tools viewed as pi `AgentTool`s, for the low-level `Agent`. */ readonly agentTools: readonly AgentTool[]; /** - * A `Models` collection that adds what CUA owns per request: provider retry, - * required headers, the catalog's payload transforms, and the tool-result - * image bound. + * The handle's `Models` collection, adding what CUA owns per request: + * provider retry, required headers, the catalog's payload transforms, and + * the tool-result image bound. Shared by every compile from this handle, + * because pi fixes `models` at construction while those transforms are + * per-catalog; it serves whichever pair is currently active. */ readonly models: Models; /** - * Register the behaviors that are pi event handlers rather than constructor - * options: marking failed tool results, blocking a turn's remaining calls - * after one fails, and empty-response recovery. Returns an unsubscribe. + * Make this the handle's live pair on a harness: register the behaviors that + * are pi event handlers rather than constructor options (marking failed tool + * results, blocking a turn's remaining calls after one fails, empty-response + * recovery), and point `models` at this catalog. Returns a release that undoes + * both. Activating another pair releases this one. */ - install(harness: AgentHarness): () => void; + activate(harness: AgentHarness): () => void; + /** + * Swap a running harness onto this pair, then activate it. Model and tools + * move together because the transport is derived from both: setting one + * without the other leaves pi streaming a combination that never compiled. + * The model is only set when the derived transport actually moved, so a + * tools-only change records no model change. A failure restores the harness's + * previous pair before rethrowing, leaving the old pair active. + */ + apply(harness: AgentHarness>): Promise; } /** @@ -93,6 +106,8 @@ export interface CuaBrowserHandle { }): CuaCompiled; /** The shared execution pool, for callers that need it directly. */ readonly resources: CuaExecutionResources; + /** Same collection every {@link CuaCompiled.models} returns; see the note there. */ + readonly models: Models; dispose(): Promise; } @@ -113,8 +128,28 @@ export function attach(options: CuaAttachOptions): CuaBrowserHandle { const recovery = resolveEmptyResponseRecovery(options.emptyResponseRecovery); const retrying = withProviderRetryModels(options.models ?? cuaModels(), resolveProviderRetryPolicy(options.retry)); + // pi fixes `models` at construction, but the headers, payload transforms and + // incoming tool plan it applies are per-catalog. One collection per handle, + // reading whichever pair is live, is what lets a caller swap the pair on a + // running harness at all. + let live: CuaToolManager | undefined; + let lastCompiled: CuaToolManager | undefined; + let release: (() => void) | undefined; + const models = withCatalogModels( + retrying, + () => { + const manager = live ?? lastCompiled; + if (!manager) throw new Error("cua: compile a (model, tools) pair before streaming"); + return manager; + }, + imageReplayLimit, + useResponseThreading, + options.onPayload, + ); + return { resources, + models, dispose: () => resources.dispose(), compile(request: { model: CuaModelInput; @@ -126,18 +161,56 @@ export function attach(options: CuaAttachOptions): CuaBrowserHandle { request.tools, (ref) => resolveModelFromCollection(ref, retrying), ); - const models = withCatalogModels(retrying, manager, imageReplayLimit, useResponseThreading, options.onPayload); + lastCompiled = manager; + const model = manager.catalog.model; + const tools = manager.harnessTools() as readonly AgentHarnessTool[]; + const activate = (harness: AgentHarness): (() => void) => { + release?.(); + const uninstall = installCuaBehaviors(harness, manager, recovery); + live = manager; + // Identity-checked so calling a stale release cannot clear a newer + // activation: only the pair still live releases anything. + const releaseThis = (): void => { + uninstall(); + if (live === manager) live = undefined; + if (release === releaseThis) release = undefined; + }; + release = releaseThis; + return releaseThis; + }; return { - model: manager.catalog.model, - tools: manager.harnessTools() as readonly AgentHarnessTool[], + model, + tools, agentTools: manager.agentTools(), models, - install: (harness) => installCuaBehaviors(harness, manager, recovery), + activate, + async apply(harness) { + await applyCompiled(harness, model, tools); + activate(harness); + }, }; }, }; } +async function applyCompiled( + harness: AgentHarness>, + model: Model, + tools: readonly AgentHarnessTool[], +): Promise { + const previousModel = harness.getModel(); + const previousTools = harness.getTools(); + const transportChanged = modelTransportChanged(previousModel, model); + try { + if (transportChanged) await harness.setModel(model); + await harness.setTools([...tools], tools.map((tool) => tool.name)); + } catch (error) { + if (transportChanged) await harness.setModel(previousModel); + await harness.setTools(previousTools, previousTools.map((tool) => tool.name)); + throw error; + } +} + /** * Wire the pi event handlers CUA owns. Kept separate from `compile()` because * they are handlers on a constructed harness, not constructor options. @@ -179,25 +252,25 @@ export function installCuaBehaviors( }; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export const defaultCuaStream: StreamFn = (model, context, options) => cuaModels().streamSimple(model, context, options); -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function resolveModelFromCollection(ref: CuaModelRef, models: Models): Model { const { provider, model: id } = parseCuaModelRef(ref); return models.getModel(provider, id) ?? getCuaModel(ref); } /** Whether a tools-only recompile actually changed the model pi streams with, so `setTools()` only pushes `setModel()` (and its session/event side effects) when the derived transport moved. */ -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function modelTransportChanged(previous: Model, next: Model): boolean { return previous.provider !== next.provider || previous.id !== next.id || previous.api !== next.api; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function withCatalogModels( models: Models, - manager: CuaToolManager, + liveManager: () => CuaToolManager, imageReplayLimit: ToolResultImageReplayLimit, responseThreading: boolean, handleOnPayload?: SimpleStreamOptions["onPayload"], @@ -205,10 +278,10 @@ export function withCatalogModels( const contextFor = (context: Context) => projectModelContext( context, imageReplayLimit, - requiredImageToolNames(manager.catalog.incoming), + requiredImageToolNames(liveManager().catalog.incoming), ); const optionsFor = (options: T): T => { - const catalog = manager.catalog; + const catalog = liveManager().catalog; const callerOnPayload = options?.onPayload ?? handleOnPayload; return { ...options, @@ -239,7 +312,7 @@ export function withCatalogModels( }; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function resolveToolResultImageReplayLimit(limit: ToolResultImageReplayLimit | undefined): ToolResultImageReplayLimit { if (limit === undefined) return DEFAULT_TOOL_RESULT_IMAGE_REPLAY_LIMIT; if (limit !== false && (!Number.isFinite(limit) || !Number.isInteger(limit) || limit < 0)) { @@ -249,12 +322,12 @@ export function resolveToolResultImageReplayLimit(limit: ToolResultImageReplayLi } /** Native computer tool names whose screenshot history the provider protocol requires in full, regardless of the image replay limit. */ -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function requiredImageToolNames(incoming: CuaIncomingToolPlan): ReadonlySet { return new Set(incoming.openaiComputerName ? [incoming.openaiComputerName] : []); } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function projectToolResultImages( messages: TMessage[], limit: ToolResultImageReplayLimit, @@ -299,7 +372,7 @@ function projectModelContext( return messages === context.messages ? context : { ...context, messages }; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function resolveEmptyResponseRecovery(options: CuaEmptyResponseRecoveryOptions | undefined): CuaEmptyResponseRecoveryOptions | undefined { if (!options) return undefined; if (options.followUp.trim().length === 0) throw new Error("emptyResponseRecovery.followUp must not be blank"); @@ -307,23 +380,23 @@ export function resolveEmptyResponseRecovery(options: CuaEmptyResponseRecoveryOp return { followUp: options.followUp, maxAttempts: options.maxAttempts }; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function resolveResponseThreading(value: boolean | undefined): boolean { if (value !== undefined && typeof value !== "boolean") throw new TypeError("responseThreading must be a boolean"); return value ?? true; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function isEmptyAssistantResponse(message: AgentMessage): boolean { return message.role === "assistant" && message.stopReason === "stop" && message.content.length === 0; } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function hasExecutionError(details: unknown): boolean { return Boolean(details && typeof details === "object" && (details as CuaExecutionDetails).isError === true); } -/** @internal shared with the agent classes until they retire. */ +/** @internal */ export function turnFailureStopMessage(manager: CuaToolManager): string | undefined { for (const entry of manager.catalog.entries) { const execution = manager.specFor(entry.identity)?.execution; diff --git a/packages/agent/src/index.ts b/packages/agent/src/index.ts index d890993..aa78a66 100644 --- a/packages/agent/src/index.ts +++ b/packages/agent/src/index.ts @@ -31,7 +31,6 @@ export type { BrowserWaitForResult, BrowserWaitReason, } from "./translator/types"; -export { CuaAgent, CuaAgentHarness } from "./agent"; export { attach } from "./attach"; export type { CuaAttachOptions, @@ -41,5 +40,4 @@ export type { CuaModelInput, ToolResultImageReplayLimit, } from "./attach"; -export type { CuaAgentHarnessOptions, CuaAgentOptions, CuaAgentState } from "./agent"; export type { CuaRetryOptions } from "./provider-retry"; diff --git a/packages/agent/src/resources.ts b/packages/agent/src/resources.ts index 390a2bb..a6cc07f 100644 --- a/packages/agent/src/resources.ts +++ b/packages/agent/src/resources.ts @@ -17,7 +17,7 @@ export interface CuaExecutionDetails { readResults?: Array>; skippedActions?: number; failedActionIndex?: number; - /** Internal marker consumed by CuaAgent/CuaAgentHarness to set ToolResultMessage.isError. */ + /** Internal marker consumed by the behaviors `activate()` installs, to set ToolResultMessage.isError. */ isError?: boolean; result?: unknown; stdout?: string; diff --git a/packages/agent/src/tool-manager.ts b/packages/agent/src/tool-manager.ts index ec81e03..12d9fc1 100644 --- a/packages/agent/src/tool-manager.ts +++ b/packages/agent/src/tool-manager.ts @@ -1,12 +1,10 @@ -import { AsyncLocalStorage } from "node:async_hooks"; -import type { AgentHarnessTool, AgentTool, AgentToolResult } from "@earendil-works/pi-agent-core"; +import type { AgentHarnessTool, AgentTool } from "@earendil-works/pi-agent-core"; import type { Api, Model } from "@earendil-works/pi-ai"; import { callerToolIdentity, compileCuaToolCatalog, getCuaModel, isCuaToolSpec, - modelSupportsDeferredTools, type CuaCatalogToolInput, type CuaModelRef, type CuaToolCatalog, @@ -15,244 +13,84 @@ import { import { CuaExecutionResources } from "./resources"; /** - * Caller-owned tool for {@link CuaAgent}: a declarative CUA spec materialized - * by this package, or an already executable pi `AgentTool`. Defined here - * because cua-agent is the only package that holds both halves; cua-ai - * compiles declaration-only catalogs. + * Caller-owned tool: a declarative CUA spec materialized by this package, or an + * already executable pi `AgentTool`. Defined here because cua-agent is the only + * package that holds both halves; cua-ai compiles declaration-only catalogs. */ export type CuaAgentTool = CuaToolSpec | AgentTool; /** - * Caller-owned tool for {@link CuaAgentHarness}: a declarative CUA spec - * materialized by this package, or an executable pi `AgentHarnessTool` that - * receives the harness's tool context on every call. A plain `AgentTool` is - * assignable (it simply ignores the context), but the two APIs are kept - * distinct: `CuaAgent` takes `CuaAgentTool`, `CuaAgentHarness` takes this. + * Caller-owned tool for a harness: a declarative CUA spec, or an executable pi + * `AgentHarnessTool` that receives the harness's tool context on every call. A + * plain `AgentTool` is assignable, since it simply ignores the context. */ export type CuaHarnessTool = CuaToolSpec | AgentHarnessTool; /** - * One atomically committable tools state. `requested` is the sole caller-owned - * source of truth; `catalog` is its pure declarative projection; `tools` and - * `harnessTools` are the wrapped executables joined back by identity after - * compilation, viewed as pi `AgentTool`s (for `CuaAgent`) or as - * context-delivering `AgentHarnessTool`s (for `CuaAgentHarness`). + * One compiled (model, tools) pair: the caller's list joined back to its + * declarative catalog by compiled identity, with each spec materialized against + * the shared browser resources. + * + * Immutable by construction. Changing the model or the tool list compiles a new + * one, which is what lets a caller hand pi a fresh pair instead of mutating a + * live catalog underneath it. */ -export interface PreparedCuaTools = CuaAgentTool> { - readonly requested: readonly TRequested[]; - /** - * The model input this state was compiled from, before catalog compilation - * derived a tool-selection-dependent transport onto it. A tools-only - * recompile reuses this (not `catalog.model`, which may carry a transport - * the new tool selection no longer requires) so the derivation re-runs from - * a clean model every time. - */ - readonly modelSelection: CuaModelRef | Model; - readonly catalog: CuaToolCatalog; - readonly tools: readonly AgentTool[]; - readonly harnessTools: readonly AgentHarnessTool[]; - /** Identity → CUA spec, for execution metadata (e.g. stop-on-failure policy). */ - readonly specs: ReadonlyMap; - /** Declaration fingerprint composed with implementation identity, in entry order. */ - readonly fingerprints: readonly string[]; -} - -interface ToolExecutionScope { - readonly toolName: string; - readonly executionMode: AgentTool["executionMode"]; - readonly baseline: PreparedCuaTools; -} - -/** - * Implementation identity, owned by cua-agent: keyed on a caller tool's - * `execute` function (a new wrapper reusing the same function retains - * identity) and on the spec object itself (a freshly created spec is - * conservatively a replacement, the same object stays stable). - */ -const implementationIds = new WeakMap(); -let nextImplementationId = 1; - -function implementationId(key: object): number { - let id = implementationIds.get(key); - if (id === undefined) { - id = nextImplementationId++; - implementationIds.set(key, id); - } - return id; -} - -/** Owns the caller's requested list and its compiled catalog while browser resources live independently. */ export class CuaToolManager = CuaAgentTool> { - private readonly execution = new AsyncLocalStorage(); - private current: PreparedCuaTools; + readonly catalog: CuaToolCatalog; + private readonly executables: readonly (AgentTool | AgentHarnessTool)[]; + private readonly specs: ReadonlyMap; constructor( readonly resources: CuaExecutionResources, model: CuaModelRef | Model, requestedTools: readonly TRequested[], - private readonly resolveModel: (model: CuaModelRef) => Model = getCuaModel, + resolveModel: (model: CuaModelRef) => Model = getCuaModel, ) { - this.current = this.prepare(model, requestedTools); - } - - get catalog(): CuaToolCatalog { - return this.current.catalog; - } - - getTools(): TRequested[] { - return [...this.current.requested]; - } - - /** Wrapped pi `AgentTool` view of a prepared (or the committed) state, in entry order. */ - agentTools(prepared: PreparedCuaTools = this.current): AgentTool[] { - return [...prepared.tools]; - } - - /** Wrapped pi `AgentHarnessTool` view of a prepared (or the committed) state, in entry order. */ - harnessTools(prepared: PreparedCuaTools = this.current): AgentHarnessTool[] { - return [...prepared.harnessTools]; - } - - /** Execution metadata for one committed catalog identity. */ - specFor(identity: string): CuaToolSpec | undefined { - return this.current.specs.get(identity); - } - - prepareTools(tools: readonly TRequested[]): PreparedCuaTools { - this.assertMutationScope("setTools"); - return this.prepare(this.current.modelSelection, tools); - } - - prepareModel(model: CuaModelRef | Model): PreparedCuaTools { - this.assertMutationScope("setModel"); - return this.prepare(model, this.current.requested); - } - - prepareModelAndTools(model: CuaModelRef | Model, tools: readonly TRequested[]): PreparedCuaTools { - this.assertMutationScope("setModelAndTools"); - return this.prepare(model, tools); - } - - commit(prepared: PreparedCuaTools): void { - this.current = prepared; - } - - /** - * Compile and materialize without touching committed state. Ordinary - * AgentTools are projected into fresh declaration-only objects before - * compilation, then joined back strictly by compiled identity — never by - * position. Any failure leaves the committed state untouched. - */ - private prepare(model: CuaModelRef | Model, tools: readonly TRequested[]): PreparedCuaTools { - const requested = Object.freeze([...tools]); const inputs: CuaCatalogToolInput[] = []; const executables = new Map(); - const implementations = new Map(); const specs = new Map(); - for (const tool of requested) { + for (const tool of requestedTools) { if (isCuaToolSpec(tool)) { inputs.push(tool); executables.set(tool.identity, tool); - implementations.set(tool.identity, tool); specs.set(tool.identity, tool); } else { - const identity = callerToolIdentity(tool.name); // Fresh declaration-only projection: execute, label, prepareArguments, // and executionMode never cross into cua-ai. inputs.push({ name: tool.name, description: tool.description, parameters: tool.parameters }); - executables.set(identity, tool); - implementations.set(identity, tool.execute); + executables.set(callerToolIdentity(tool.name), tool); } } - const catalog = compileCuaToolCatalog({ - model: typeof model === "string" ? this.resolveModel(model) : model, + this.catalog = compileCuaToolCatalog({ + model: typeof model === "string" ? resolveModel(model) : model, requestedTools: inputs, }); - const fingerprints: string[] = []; - const joined: Array> = catalog.entries.map((entry) => { + // Joined strictly by compiled identity, never by position. + this.executables = this.catalog.entries.map((entry) => { const executable = executables.get(entry.identity); - const implementation = implementations.get(entry.identity); - if (!executable || !implementation) { - throw new Error(`compiled catalog entry "${entry.identity}" has no matching requested tool`); - } + if (!executable) throw new Error(`compiled catalog entry "${entry.identity}" has no matching requested tool`); executables.delete(entry.identity); - fingerprints.push(`${entry.fingerprint}#impl-${implementationId(implementation)}`); - return isCuaToolSpec(executable) ? this.resources.materialize(executable) : (executable as AgentHarnessTool); + return isCuaToolSpec(executable) ? resources.materialize(executable) : (executable as AgentHarnessTool); }); if (executables.size > 0) { throw new Error(`requested tool(s) ${[...executables.keys()].join(", ")} missing from the compiled catalog`); } - - return Object.freeze({ - requested, - modelSelection: model, - catalog, - tools: Object.freeze(joined.map((tool) => this.wrapAgentExecutable(tool as AgentTool))), - harnessTools: Object.freeze(joined.map((tool) => this.wrapHarnessExecutable(tool))), - specs, - fingerprints: Object.freeze(fingerprints), - }); - } - - /** Low-level `AgentTool` view for {@link CuaAgent}; caller tools there never declare a harness context. */ - private wrapAgentExecutable(tool: AgentTool): AgentTool { - return { - ...tool, - execute: (toolCallId, input, signal, onUpdate) => - this.executeWithScope(tool, () => tool.execute(toolCallId, input, signal, onUpdate)), - }; + this.specs = specs; } - /** Context-delivering `AgentHarnessTool` view for {@link CuaAgentHarness}. */ - private wrapHarnessExecutable(tool: AgentHarnessTool): AgentHarnessTool { - return { - ...tool, - execute: (toolCallId, params, signal, onUpdate, context) => - this.executeWithScope(tool, () => tool.execute(toolCallId, params, signal, onUpdate, context)), - }; + /** Pi `AgentTool` view, in catalog entry order. */ + agentTools(): AgentTool[] { + return [...(this.executables as readonly AgentTool[])]; } - private async executeWithScope( - tool: { readonly name: string; readonly executionMode?: AgentTool["executionMode"] }, - call: () => Promise>, - ): Promise> { - const scope: ToolExecutionScope = { - toolName: tool.name, - executionMode: tool.executionMode, - baseline: this.current, - }; - return this.execution.run(scope, async () => { - const result = await call(); - return mergeAddedToolNames(result, cachePreservingAdditions(scope.baseline, this.current) ?? []); - }); + /** Context-delivering pi `AgentHarnessTool` view, in catalog entry order. */ + harnessTools(): AgentHarnessTool[] { + return [...(this.executables as readonly AgentHarnessTool[])]; } - private assertMutationScope(api: "setTools" | "setModel" | "setModelAndTools"): void { - const scope = this.execution.getStore(); - if (scope && scope.executionMode !== "sequential") { - throw new Error(`tool "${scope.toolName}" must declare executionMode: "sequential" before calling ${api}() during execution`); - } - } -} - -function cachePreservingAdditions(previous: PreparedCuaTools, next: PreparedCuaTools): string[] | undefined { - const previousModel = previous.catalog.model; - const nextModel = next.catalog.model; - if (previousModel.provider !== nextModel.provider || previousModel.id !== nextModel.id || previousModel.api !== nextModel.api) return undefined; - if (next.fingerprints.length <= previous.fingerprints.length) return undefined; - for (let index = 0; index < previous.fingerprints.length; index += 1) { - if (previous.fingerprints[index] !== next.fingerprints[index]) return undefined; + /** Execution metadata for one catalog identity. */ + specFor(identity: string): CuaToolSpec | undefined { + return this.specs.get(identity); } - const added = next.catalog.entries.slice(previous.fingerprints.length); - if (!modelSupportsDeferredTools(nextModel) || added.some((entry) => entry.dynamicLoading !== "eligible")) return undefined; - return added.map((entry) => entry.name); -} - -function mergeAddedToolNames(result: AgentToolResult, names: readonly string[]): AgentToolResult { - if (names.length === 0) return result; - return { - ...result, - addedToolNames: [...new Set([...(result.addedToolNames ?? []), ...names])], - }; } diff --git a/packages/agent/test/agent.test.ts b/packages/agent/test/agent.test.ts deleted file mode 100644 index 742ecf5..0000000 --- a/packages/agent/test/agent.test.ts +++ /dev/null @@ -1,634 +0,0 @@ -import { describe, expect, it, vi } from "vitest"; -import { - createAssistantMessageEventStream, - createCuaModels, - getCuaModel, - cua, - GOOGLE_CUA_INTERACTIONS_API, - isCuaToolSpec, - type AssistantMessage, - type Context, - type Model, -} from "@onkernel/cua-ai"; -import type Kernel from "@onkernel/sdk"; -import { - Agent, - AgentHarness, - CuaAgent, - CuaAgentHarness, - InMemorySessionRepo, - type AgentMessage, - type AgentTool, - type KernelBrowser, - type StreamFn, -} from "../src/index"; - -const browser = { session_id: "browser_123", viewport: { width: 1440, height: 900 } } as KernelBrowser; -const client = {} as Kernel; - -function assistant(model: Model, content: AssistantMessage["content"] = [], stopReason: AssistantMessage["stopReason"] = "stop"): AssistantMessage { - return { - role: "assistant", - content, - api: model.api, - provider: model.provider, - model: model.id, - usage: { - input: 0, - output: 0, - cacheRead: 0, - cacheWrite: 0, - totalTokens: 0, - cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 }, - }, - stopReason, - timestamp: Date.now(), - }; -} - -function scriptedStream( - turns: Array<(model: Model) => AssistantMessage>, - contexts: Context[] = [], -): StreamFn { - let call = 0; - return (model, context) => { - contexts.push({ ...context, messages: structuredClone(context.messages), tools: context.tools?.slice() }); - const stream = createAssistantMessageEventStream(); - const message = turns[call++]?.(model) ?? assistant(model); - stream.push({ type: "start", partial: message }); - stream.push({ type: "done", reason: message.stopReason as "stop" | "length" | "toolUse", message }); - stream.end(message); - return stream; - }; -} - -function callerTool(name: string, execute?: AgentTool["execute"], executionMode?: AgentTool["executionMode"]): AgentTool { - return { - name, - label: name, - description: `${name} tool`, - parameters: { type: "object", properties: {}, additionalProperties: false } as never, - ...(executionMode ? { executionMode } : {}), - execute: execute ?? (async () => ({ content: [{ type: "text", text: "ok" }], details: {} })), - }; -} - -async function harnessServices() { - const repo = new InMemorySessionRepo(); - return { - session: await repo.create(), - }; -} - -function modelsFromStream(streamFn: StreamFn, provider = "openai") { - const models = createCuaModels(); - models.setProvider({ - id: provider, - name: "scripted", - auth: { apiKey: { name: "test", resolve: async () => ({ auth: { apiKey: "test" } }) } }, - getModels: () => [], - stream: streamFn, - streamSimple: streamFn, - } as never); - return models; -} - -describe("CuaAgent explicit tools", () => { - it("uses composition and accepts tools: [] without a prompt or implicit tool", () => { - const agent = new CuaAgent({ browser, client, tools: [], initialState: { model: "openai:gpt-5.5" } }); - expect(agent).not.toBeInstanceOf(Agent); - expect(agent.getTools()).toEqual([]); - expect("inspectTools" in agent).toBe(false); - expect(agent.state.tools).toEqual([]); - expect(agent.state.systemPrompt).toBe(""); - expect("setMode" in agent).toBe(false); - expect("getMode" in agent).toBe(false); - }); - - it("retains protocol-required OpenAI native computer screenshot results outside the image replay limit", async () => { - const contexts: Context[] = []; - const model = getCuaModel("openai:gpt-5.5"); - const nativeComputer = cua.providers.openai.tools.computer(); - const ordinaryTool = callerTool("ordinary"); - const image = { type: "image" as const, data: "c2NyZWVuc2hvdA==", mimeType: "image/png" }; - const messages: AgentMessage[] = [ - assistant(model, [{ type: "toolCall", id: "native-shot", name: "computer", arguments: { action: { type: "screenshot" } } }], "toolUse"), - { role: "toolResult", toolCallId: "native-shot", toolName: "computer", content: [image], isError: false, timestamp: 1 }, - assistant(model, [{ type: "toolCall", id: "ordinary-shot", name: "ordinary", arguments: {} }], "toolUse"), - { role: "toolResult", toolCallId: "ordinary-shot", toolName: "ordinary", content: [image], isError: false, timestamp: 2 }, - ]; - const agent = new CuaAgent({ - browser, - client, - tools: [nativeComputer, ordinaryTool], - streamFn: scriptedStream([(selectedModel) => assistant(selectedModel)], contexts), - toolResultImageReplayLimit: 0, - initialState: { model, messages }, - }); - - await agent.prompt("continue"); - - const results = contexts[0]!.messages.filter((message) => message.role === "toolResult"); - expect(results[0]!.content).toEqual([image]); - expect(results[1]!.content).toEqual([{ type: "text", text: "[stale tool-result images omitted]" }]); - }); - - it("installs exact native-only, native-plus-CUA, Playwright-only, and browser-act-only catalogs", () => { - const custom = callerTool("customer_lookup"); - const cases = [ - { - model: "anthropic:claude-opus-5" as const, - tools: [cua.providers.anthropic.tools.browser(), custom], - names: ["browser", "customer_lookup"], - }, - { - model: "openai:gpt-5.5" as const, - tools: [cua.providers.openai.tools.computer(), cua.tools.browser.snapshot(), cua.tools.browser.act()], - names: ["computer", "browser_snapshot", "browser_act"], - }, - { model: "openai:gpt-5.5" as const, tools: [cua.tools.playwright()], names: ["playwright_execute"] }, - { model: "openai:gpt-5.5" as const, tools: [cua.tools.browser.act()], names: ["browser_act"] }, - ]; - for (const entry of cases) { - const agent = new CuaAgent({ browser, client, tools: entry.tools, initialState: { model: entry.model } }); - expect(agent.state.tools.map((tool) => tool.name)).toEqual(entry.names); - expect(agent.getTools()).toEqual(entry.tools); - } - }); - - it("returns a copy of the exact requested specs", () => { - const requested = [cua.tools.browser.snapshot(), callerTool("customer_lookup")]; - const agent = new CuaAgent({ browser, client, tools: requested, initialState: { model: "anthropic:claude-opus-5" } }); - expect(agent.getTools()).toEqual(requested); - expect(agent.getTools()).not.toBe(requested); - expect(agent.state.tools.map((tool) => tool.name)).toEqual(["browser_snapshot", "customer_lookup"]); - }); - - it("keeps the caller system prompt stable across setTools", () => { - const agent = new CuaAgent({ - browser, - client, - tools: [], - initialState: { model: "openai:gpt-5.5", systemPrompt: "caller-owned" }, - }); - agent.setTools([cua.tools.playwright()]); - expect(agent.state.systemPrompt).toBe("caller-owned"); - expect(agent.getTools().map((tool) => tool.name)).toEqual(["playwright_execute"]); - }); - - it("marks a sequential in-tool prefix addition for deferred loading", async () => { - const contexts: Context[] = []; - let agent!: CuaAgent; - const added = callerTool("added"); - const loader = callerTool("loader", async () => { - agent.setTools([...agent.getTools(), added]); - return { content: [{ type: "text", text: "loaded" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [loader], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load-1", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model, [{ type: "text", text: "done" }]), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - - await agent.prompt("load it"); - - expect(agent.getTools()).toEqual([loader, added]); - expect(contexts[1]?.tools?.map((tool) => tool.name)).toEqual(["loader", "added"]); - expect(contexts[1]?.messages.find((message) => message.role === "toolResult")).toMatchObject({ addedToolNames: ["added"] }); - }); - - it("computes deferred additions from the final in-tool catalog", async () => { - const contexts: Context[] = []; - let agent!: CuaAgent; - const added = callerTool("temporary"); - const loader = callerTool("loader", async () => { - agent.setTools([...agent.getTools(), added]); - agent.setTools([loader]); - return { content: [{ type: "text", text: "unchanged" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [loader], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("load"); - expect(contexts[1]?.messages.find((message) => message.role === "toolResult")).not.toHaveProperty("addedToolNames"); - }); - - it("uses eager fallback for removals and replacements", async () => { - const run = async (replacement: "remove" | "replace") => { - const contexts: Context[] = []; - let agent!: CuaAgent; - const loader = callerTool("loader", async () => { - agent.setTools(replacement === "remove" ? [] : [callerTool("loader")]); - return { content: [{ type: "text", text: replacement }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [loader], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "change", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("change"); - return contexts[1]?.messages.find((message) => message.role === "toolResult"); - }; - expect(await run("remove")).not.toHaveProperty("addedToolNames"); - expect(await run("replace")).not.toHaveProperty("addedToolNames"); - }); - - it("keeps deferred additions when a replacement wrapper reuses the same execute function", async () => { - const contexts: Context[] = []; - let agent!: CuaAgent; - const sharedExecute: AgentTool["execute"] = async () => ({ content: [{ type: "text", text: "ok" }], details: {} }); - const original = callerTool("original", sharedExecute); - const added = callerTool("added"); - const loader = callerTool("loader", async () => { - const rewrapped = { ...original }; - expect(rewrapped).not.toBe(original); - agent.setTools([rewrapped, loader, added]); - return { content: [{ type: "text", text: "loaded" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [original, loader], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("load"); - expect(contexts[1]?.messages.find((message) => message.role === "toolResult")).toMatchObject({ addedToolNames: ["added"] }); - }); - - it("installs and invokes a replacement executor with an identical name and schema", async () => { - const contexts: Context[] = []; - const calls: string[] = []; - let agent!: CuaAgent; - const workerV1 = callerTool("worker", async () => { - calls.push("v1"); - return { content: [{ type: "text", text: "v1" }], details: {} }; - }); - const loader = callerTool("loader", async () => { - const workerV2 = callerTool("worker", async () => { - calls.push("v2"); - return { content: [{ type: "text", text: "v2" }], details: {} }; - }); - agent.setTools([loader, workerV2]); - return { content: [{ type: "text", text: "replaced" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [loader, workerV1], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model, [{ type: "toolCall", id: "work", name: "worker", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("replace then work"); - expect(calls).toEqual(["v2"]); - expect(contexts[1]?.messages.find((message) => message.role === "toolResult")).not.toHaveProperty("addedToolNames"); - }); - - it("treats a freshly created spec object as a replacement but the same object as stable", async () => { - const run = async (reuseSpec: boolean) => { - const contexts: Context[] = []; - let agent!: CuaAgent; - const spec = cua.tools.browser.snapshot(); - const added = callerTool("added"); - const loader = callerTool("loader", async () => { - agent.setTools([reuseSpec ? spec : cua.tools.browser.snapshot(), loader, added]); - return { content: [{ type: "text", text: "loaded" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [spec, loader], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("load"); - return contexts[1]?.messages.find((message) => message.role === "toolResult"); - }; - expect(await run(true)).toMatchObject({ addedToolNames: ["added"] }); - expect(await run(false)).not.toHaveProperty("addedToolNames"); - }); - - it("leaves model, requested tools, and installed executables untouched when setTools fails", () => { - const spec = cua.tools.browser.snapshot(); - const keep = callerTool("keep"); - const agent = new CuaAgent({ browser, client, tools: [spec, keep], initialState: { model: "openai:gpt-5.5" } }); - const installed = agent.state.tools; - expect(() => agent.setTools([cua.providers.anthropic.tools.browser()])).toThrow(/requires a anthropic model/); - expect(() => agent.setTools([callerTool("keep"), callerTool("keep")])).toThrow(/caller\.keep/); - expect(agent.getModel().id).toBe("gpt-5.5"); - expect(agent.getTools()).toEqual([spec, keep]); - expect(agent.state.tools).toEqual(installed); - agent.state.tools.forEach((tool, index) => expect(tool).toBe(installed[index])); - }); - - it("leaves model, requested tools, and installed executables untouched when setModel fails", () => { - const spec = cua.providers.anthropic.tools.browser(); - const agent = new CuaAgent({ browser, client, tools: [spec], initialState: { model: "anthropic:claude-opus-5" } }); - const installed = agent.state.tools; - expect(() => agent.setModel("openai:gpt-5.5")).toThrow(/requires a anthropic model/); - expect(agent.getModel().provider).toBe("anthropic"); - expect(agent.getTools()).toEqual([spec]); - agent.state.tools.forEach((tool, index) => expect(tool).toBe(installed[index])); - }); - - it("rejects in-tool mutation from a non-sequential caller tool", async () => { - let agent!: CuaAgent; - const loader = callerTool("loader", async () => { - agent.setTools([cua.tools.playwright()]); - return { content: [{ type: "text", text: "unexpected" }], details: {} }; - }); - agent = new CuaAgent({ - browser, - client, - tools: [loader], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model), - ]), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("load"); - const result = agent.state.messages.find((message) => message.role === "toolResult"); - expect(result).toMatchObject({ isError: true }); - expect(result?.content).toEqual([expect.objectContaining({ text: expect.stringContaining('executionMode: "sequential"') })]); - }); - - it("rejects in-tool model switching from a non-sequential caller tool", async () => { - let agent!: CuaAgent; - const switcher = callerTool("switcher", async () => { - agent.setModel("anthropic:claude-opus-5"); - return { content: [{ type: "text", text: "unexpected" }], details: {} }; - }); - agent = new CuaAgent({ - browser, - client, - tools: [switcher], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "switch", name: "switcher", arguments: {} }], "toolUse"), - (model) => assistant(model), - ]), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("switch"); - const result = agent.state.messages.find((message) => message.role === "toolResult"); - expect(result).toMatchObject({ isError: true }); - expect(result?.content).toEqual([expect.objectContaining({ text: expect.stringContaining('before calling setModel()') })]); - expect(agent.getModel().provider).toBe("openai"); - }); - - it("allows in-tool model switching from a sequential caller tool", async () => { - const contexts: Context[] = []; - let agent!: CuaAgent; - const switcher = callerTool("switcher", async () => { - agent.setModel("anthropic:claude-opus-5"); - return { content: [{ type: "text", text: "switched" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser, - client, - tools: [switcher], - streamFn: scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "switch", name: "switcher", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts), - initialState: { model: "openai:gpt-5.5" }, - }); - await agent.prompt("switch"); - const result = agent.state.messages.find((message) => message.role === "toolResult"); - expect(result).not.toMatchObject({ isError: true }); - expect(agent.getModel().provider).toBe("anthropic"); - expect(agent.getTools()).toEqual([switcher]); - expect(contexts[1]?.messages.find((message) => message.role === "toolResult")).not.toHaveProperty("addedToolNames"); - }); - - it("streams with the transport setTools derives, not just the catalog it compiles", async () => { - const streamedApis: string[] = []; - const agent = new CuaAgent({ - browser, - client, - tools: [cua.tools.browser.snapshot()], - streamFn: (model, context, options) => { - streamedApis.push(model.api); - return scriptedStream([(selectedModel) => assistant(selectedModel)])(model, context, options); - }, - initialState: { model: "google:gemini-3.6-flash" }, - }); - expect(agent.getModel().api).toBe("google-generative-ai"); - - agent.setTools(cua.providers.google.toolsets.browser()); - expect(agent.getModel().api).toBe(GOOGLE_CUA_INTERACTIONS_API); - - await agent.prompt("go"); - expect(streamedApis).toEqual([GOOGLE_CUA_INTERACTIONS_API]); - }); -}); - -describe("CuaAgentHarness explicit tools", () => { - it("resolves refs from supplied models for construction and setModel", async () => { - const models = createCuaModels(); - const openai = models.getProvider("openai")!; - const first = { ...getCuaModel("openai:gpt-5.5"), baseUrl: "https://first.example" }; - const second = { ...getCuaModel("openai:gpt-5.6-sol"), baseUrl: "https://second.example" }; - models.setProvider({ ...openai, getModels: () => [first, second] }); - const services = await harnessServices(); - const harness = new CuaAgentHarness({ ...services, browser, client, models, model: "openai:gpt-5.5", tools: [] }); - expect(harness.getModel()).toBe(first); - await harness.setModel("openai:gpt-5.6-sol"); - expect(harness.getModel()).toBe(second); - - // A ref the supplied collection does not carry falls back to the registry - // rather than being refused: the provider decides what exists. - const fallback = new CuaAgentHarness({ ...services, browser, client, models, model: "openai:gpt-5.4", tools: [] }); - expect(fallback.getModel().id).toBe("gpt-5.4"); - }); - - it("uses composition, hides active-tool APIs, and supports an empty catalog", async () => { - const harness = new CuaAgentHarness({ ...(await harnessServices()), browser, client, model: "openai:gpt-5.5", tools: [] }); - expect(harness).not.toBeInstanceOf(AgentHarness); - expect(harness.getTools()).toEqual([]); - expect("inspectTools" in harness).toBe(false); - expect("getActiveTools" in harness).toBe(false); - expect("setActiveTools" in harness).toBe(false); - expect("setMode" in harness).toBe(false); - }); - - it("preserves requested tools on compatible model changes and rejects incompatible native tools", async () => { - const custom = callerTool("custom"); - const harness = new CuaAgentHarness({ ...(await harnessServices()), browser, client, model: "openai:gpt-5.5", tools: [custom] }); - await harness.setModel("anthropic:claude-opus-5"); - expect(harness.getTools()).toEqual([custom]); - expect(harness.getModel().provider).toBe("anthropic"); - - await harness.setTools([cua.providers.anthropic.tools.browser()]); - await expect(harness.setModel("openai:gpt-5.5")).rejects.toThrow(/requires a anthropic model/); - expect(harness.getModel().provider).toBe("anthropic"); - const installed = harness.getTools()[0]; - expect(installed && isCuaToolSpec(installed) ? installed.identity : undefined).toBe("provider.anthropic.native.browser.20260701"); - }); - - it("keeps the harness catalog and executors unchanged when setTools fails", async () => { - const keep = callerTool("keep"); - const harness = new CuaAgentHarness({ ...(await harnessServices()), browser, client, model: "openai:gpt-5.5", tools: [keep] }); - await expect(harness.setTools([cua.providers.anthropic.tools.browser()])).rejects.toThrow(/requires a anthropic model/); - expect(harness.getModel().id).toBe("gpt-5.5"); - expect(harness.getTools()).toEqual([keep]); - }); - - it("persists exact tool-name changes and anchors additive in-tool loading", async () => { - const contexts: Context[] = []; - const services = await harnessServices(); - let harness!: CuaAgentHarness; - const added = callerTool("added"); - const loader = callerTool("loader", async () => { - await harness.setTools([...harness.getTools(), added]); - return { content: [{ type: "text", text: "loaded" }], details: {} }; - }, "sequential"); - harness = new CuaAgentHarness({ - ...services, - browser, - client, - model: "openai:gpt-5.5", - models: modelsFromStream(scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "load", name: "loader", arguments: {} }], "toolUse"), - (model) => assistant(model), - ], contexts)), - tools: [loader], - systemPrompt: "stable", - }); - - await harness.prompt("load"); - - expect(contexts[0]?.systemPrompt).toBe("stable"); - expect(contexts[1]?.systemPrompt).toBe("stable"); - expect(contexts[1]?.messages.find((message) => message.role === "toolResult")).toMatchObject({ addedToolNames: ["added"] }); - const changes = (await services.session.getBranch()).filter((entry) => entry.type === "active_tools_change"); - expect(changes.at(-1)).toMatchObject({ activeToolNames: ["loader", "added"] }); - }); - - it("does not emit an artificial tool result for idle additions", async () => { - const contexts: Context[] = []; - const harness = new CuaAgentHarness({ - ...(await harnessServices()), - browser, - client, - model: "openai:gpt-5.5", - models: modelsFromStream(scriptedStream([(model) => assistant(model)], contexts)), - tools: [], - }); - await harness.setTools([callerTool("added")]); - await harness.prompt("next"); - expect(contexts[0]?.tools?.map((tool) => tool.name)).toEqual(["added"]); - expect(contexts[0]?.messages.some((message) => message.role === "toolResult")).toBe(false); - }); - - it("clears failed-turn state before the next prompt", async () => { - let successfulCalls = 0; - const failing = callerTool("failing", async () => { - throw new Error("expected failure"); - }); - const succeeding = callerTool("succeeding", async () => { - successfulCalls += 1; - return { content: [{ type: "text", text: "ok" }], details: {} }; - }); - const harness = new CuaAgentHarness({ - ...(await harnessServices()), - browser, - client, - model: "anthropic:claude-opus-5", - models: modelsFromStream(scriptedStream([ - (model) => assistant(model, [{ type: "toolCall", id: "fail", name: "failing", arguments: {} }], "toolUse"), - (model) => assistant(model, [{ type: "toolCall", id: "succeed", name: "succeeding", arguments: {} }], "toolUse"), - (model) => assistant(model, [{ type: "text", text: "done" }]), - ]), "anthropic"), - tools: [cua.providers.anthropic.tools.browser(), failing, succeeding], - }); - harness.on("tool_result", (event) => event.toolName === "failing" ? { terminate: true } : undefined); - - await harness.prompt("fail once"); - await harness.prompt("try again"); - - expect(successfulCalls).toBe(1); - }); - - it("streams with the transport setTools derives, not just the catalog it compiles", async () => { - const streamedApis: string[] = []; - const script = scriptedStream([(selectedModel) => assistant(selectedModel)]); - const harness = new CuaAgentHarness({ - ...(await harnessServices()), - browser, - client, - model: "google:gemini-3.6-flash", - models: modelsFromStream((model, context, options) => { - streamedApis.push(model.api); - return script(model, context, options); - }, "google"), - tools: [cua.tools.browser.snapshot()], - }); - expect(harness.getModel().api).toBe("google-generative-ai"); - - await harness.setTools(cua.providers.google.toolsets.browser()); - expect(harness.getModel().api).toBe(GOOGLE_CUA_INTERACTIONS_API); - - await harness.prompt("go"); - expect(streamedApis).toEqual([GOOGLE_CUA_INTERACTIONS_API]); - }); - - it("does not record a model change for a setTools() call that leaves the derived transport unchanged", async () => { - const services = await harnessServices(); - const harness = new CuaAgentHarness({ - ...services, - browser, - client, - model: "openai:gpt-5.5", - tools: [callerTool("first")], - }); - await harness.setTools([callerTool("second")]); - const modelChanges = (await services.session.getBranch()).filter((entry) => entry.type === "model_change"); - expect(modelChanges).toEqual([]); - }); - - it("records one model change for a switch that changes both the model and its derived transport", async () => { - const services = await harnessServices(); - const harness = new CuaAgentHarness({ - ...services, - browser, - client, - model: "google:gemini-3.6-flash", - tools: cua.providers.google.toolsets.browser(), - }); - expect(harness.getModel().api).toBe(GOOGLE_CUA_INTERACTIONS_API); - - await harness.setModelAndTools("openai:gpt-5.5", [cua.tools.browser.snapshot()]); - expect(harness.getModel().api).toBe("openai-responses"); - - const modelChanges = (await services.session.getBranch()).filter((entry) => entry.type === "model_change"); - expect(modelChanges).toHaveLength(1); - }); -}); diff --git a/packages/agent/test/attach-session.test.ts b/packages/agent/test/attach-session.test.ts new file mode 100644 index 0000000..2f8da1f --- /dev/null +++ b/packages/agent/test/attach-session.test.ts @@ -0,0 +1,313 @@ +import { describe, expect, it } from "vitest"; +import { + createAssistantMessageEventStream, + createCuaModels, + cua, + getCuaModel, + GOOGLE_CUA_INTERACTIONS_API, + type AssistantMessage, + type Context, + type Model, + type Models, +} from "@onkernel/cua-ai"; +import type Kernel from "@onkernel/sdk"; +import { + Agent, + AgentHarness, + attach, + InMemorySessionRepo, + type AgentMessage, + type AgentTool, + type CuaHarnessTool, + type CuaModelInput, + type KernelBrowser, + type Session, + type StreamFn, +} from "../src/index"; + +const browser = { session_id: "browser_123", viewport: { width: 1440, height: 900 } } as KernelBrowser; +const client = {} as Kernel; + +function assistant(model: Model, content: AssistantMessage["content"] = [], stopReason: AssistantMessage["stopReason"] = "stop"): AssistantMessage { + return { + role: "assistant", + content, + api: model.api, + provider: model.provider, + model: model.id, + usage: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, totalTokens: 0, cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 } }, + stopReason, + timestamp: Date.now(), + }; +} + +function scriptedStream(turns: Array<(model: Model) => AssistantMessage>, contexts: Context[] = []): StreamFn { + let call = 0; + return (model, context) => { + contexts.push({ ...context, messages: structuredClone(context.messages), tools: context.tools?.slice() }); + const stream = createAssistantMessageEventStream(); + const message = turns[call++]?.(model) ?? assistant(model); + stream.push({ type: "start", partial: message }); + stream.push({ type: "done", reason: message.stopReason as "stop" | "length" | "toolUse", message }); + stream.end(message); + return stream; + }; +} + +function callerTool(name: string, execute?: AgentTool["execute"]): AgentTool { + return { + name, + label: name, + description: `${name} tool`, + parameters: { type: "object", properties: {}, additionalProperties: false } as never, + execute: execute ?? (async () => ({ content: [{ type: "text", text: "ok" }], details: {} })), + }; +} + +function modelsFromStream(streamFn: StreamFn, provider = "openai"): Models { + const models = createCuaModels(); + models.setProvider({ + id: provider, + name: "scripted", + auth: { apiKey: { name: "test", resolve: async () => ({ auth: { apiKey: "test" } }) } }, + getModels: () => [], + stream: streamFn, + streamSimple: streamFn, + } as never); + return models; +} + +/** + * What a consumer does with a handle: compile a pair, hand it to a stock pi + * harness, and recompile-then-apply to change it. The CLI's `CuaCliCatalog` is + * this same shape. + */ +async function openSession(options: { + model: CuaModelInput; + tools: readonly CuaHarnessTool[]; + models?: Models; +}): Promise<{ + harness: AgentHarness; + session: Session; + select: (model: CuaModelInput, tools: readonly CuaHarnessTool[]) => Promise; +}> { + const session = await new InMemorySessionRepo().create(); + const handle = attach({ browser, client, models: options.models }); + const compiled = handle.compile({ model: options.model, tools: options.tools }); + const harness = new AgentHarness({ + session, + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), + }); + compiled.activate(harness); + return { + harness, + session, + select: async (model, tools) => { + const next = handle.compile({ model, tools }); + await next.apply(harness); + }, + }; +} + +describe("compiling a pair", () => { + it("compiles exact native-only, native-plus-CUA, Playwright-only, and browser-act-only catalogs", () => { + const custom = callerTool("customer_lookup"); + const cases = [ + { + model: "anthropic:claude-opus-5" as const, + tools: [cua.providers.anthropic.tools.browser(), custom], + names: ["browser", "customer_lookup"], + }, + { + model: "openai:gpt-5.5" as const, + tools: [cua.providers.openai.tools.computer(), cua.tools.browser.snapshot(), cua.tools.browser.act()], + names: ["computer", "browser_snapshot", "browser_act"], + }, + { model: "openai:gpt-5.5" as const, tools: [cua.tools.playwright()], names: ["playwright_execute"] }, + { model: "openai:gpt-5.5" as const, tools: [cua.tools.browser.act()], names: ["browser_act"] }, + ]; + const handle = attach({ browser, client }); + for (const entry of cases) { + expect(handle.compile({ model: entry.model, tools: entry.tools }).tools.map((tool) => tool.name)).toEqual(entry.names); + } + }); + + it("resolves a ref through the supplied collection, falling back to the registry", () => { + const models = createCuaModels(); + const openai = models.getProvider("openai")!; + const first = { ...getCuaModel("openai:gpt-5.5"), baseUrl: "https://first.example" }; + const second = { ...getCuaModel("openai:gpt-5.6-sol"), baseUrl: "https://second.example" }; + models.setProvider({ ...openai, getModels: () => [first, second] }); + const handle = attach({ browser, client, models }); + + expect(handle.compile({ model: "openai:gpt-5.5", tools: [] }).model).toBe(first); + expect(handle.compile({ model: "openai:gpt-5.6-sol", tools: [] }).model).toBe(second); + // A ref the supplied collection does not carry falls back to the registry + // rather than being refused: the provider decides what exists. + expect(handle.compile({ model: "openai:gpt-5.4", tools: [] }).model.id).toBe("gpt-5.4"); + }); + + it("refuses a native tool the model cannot take, before anything is applied", async () => { + const keep = callerTool("keep"); + const { harness, select } = await openSession({ model: "openai:gpt-5.5", tools: [keep] }); + + await expect(select("openai:gpt-5.5", [cua.providers.anthropic.tools.browser()])).rejects.toThrow(/requires a anthropic model/); + expect(harness.getModel().id).toBe("gpt-5.5"); + expect(harness.getTools().map((tool) => tool.name)).toEqual(["keep"]); + }); + + it("retains protocol-required OpenAI native computer screenshot results outside the image replay limit", async () => { + const contexts: Context[] = []; + const model = getCuaModel("openai:gpt-5.5"); + const image = { type: "image" as const, data: "c2NyZWVuc2hvdA==", mimeType: "image/png" }; + const messages: AgentMessage[] = [ + assistant(model, [{ type: "toolCall", id: "native-shot", name: "computer", arguments: { action: { type: "screenshot" } } }], "toolUse"), + { role: "toolResult", toolCallId: "native-shot", toolName: "computer", content: [image], isError: false, timestamp: 1 }, + assistant(model, [{ type: "toolCall", id: "ordinary-shot", name: "ordinary", arguments: {} }], "toolUse"), + { role: "toolResult", toolCallId: "ordinary-shot", toolName: "ordinary", content: [image], isError: false, timestamp: 2 }, + ]; + const handle = attach({ + browser, + client, + toolResultImageReplayLimit: 0, + models: modelsFromStream(scriptedStream([(selected) => assistant(selected)], contexts)), + }); + const compiled = handle.compile({ model, tools: [cua.providers.openai.tools.computer(), callerTool("ordinary")] }); + const agent = new Agent({ + streamFn: (selected, context, options) => compiled.models.streamSimple(selected, context, options), + initialState: { model: compiled.model, tools: [...compiled.agentTools], messages }, + }); + + await agent.prompt("continue"); + + const results = contexts[0]!.messages.filter((message) => message.role === "toolResult"); + expect(results[0]!.content).toEqual([image]); + expect(results[1]!.content).toEqual([{ type: "text", text: "[stale tool-result images omitted]" }]); + }); +}); + +describe("applying a pair to a live harness", () => { + it("streams the transport the selection derives", async () => { + const streamedApis: string[] = []; + const script = scriptedStream([(selected) => assistant(selected)]); + const { harness, select } = await openSession({ + model: "google:gemini-3.6-flash", + tools: [cua.tools.browser.snapshot()], + models: modelsFromStream((model, context, options) => { + streamedApis.push(model.api); + return script(model, context, options); + }, "google"), + }); + expect(harness.getModel().api).toBe("google-generative-ai"); + + await select("google:gemini-3.6-flash", cua.providers.google.toolsets.browser()); + expect(harness.getModel().api).toBe(GOOGLE_CUA_INTERACTIONS_API); + + await harness.prompt("go"); + expect(streamedApis).toEqual([GOOGLE_CUA_INTERACTIONS_API]); + }); + + it("records no model change when the derived transport is unchanged", async () => { + const { session, select } = await openSession({ model: "openai:gpt-5.5", tools: [callerTool("first")] }); + + await select("openai:gpt-5.5", [callerTool("second")]); + + expect((await session.getBranch()).filter((entry) => entry.type === "model_change")).toEqual([]); + }); + + it("records one model change for a switch that moves both the model and its transport", async () => { + const { harness, session, select } = await openSession({ + model: "google:gemini-3.6-flash", + tools: cua.providers.google.toolsets.browser(), + }); + expect(harness.getModel().api).toBe(GOOGLE_CUA_INTERACTIONS_API); + + await select("openai:gpt-5.5", [cua.tools.browser.snapshot()]); + expect(harness.getModel().api).toBe("openai-responses"); + + expect((await session.getBranch()).filter((entry) => entry.type === "model_change")).toHaveLength(1); + }); + + it("ignores a release from a pair that is no longer live", async () => { + const seen: Model[] = []; + const script = scriptedStream([(selected) => assistant(selected)]); + const models = modelsFromStream((model, context, options) => { + seen.push(model); + return script(model, context, options); + }, "google"); + const session = await new InMemorySessionRepo().create(); + const handle = attach({ browser, client, models }); + const first = handle.compile({ model: "google:gemini-3.6-flash", tools: [cua.tools.browser.snapshot()] }); + const harness = new AgentHarness({ + session, + model: first.model, + models: first.models, + tools: [...first.tools], + activeToolNames: first.tools.map((tool) => tool.name), + }); + const releaseFirst = first.activate(harness); + + // Count what each activation registers on the harness and what it takes + // back, so a leaked pair of handlers is visible. + let live = 0; + const on = harness.on.bind(harness); + harness.on = ((type: never, handler: never) => { + live += 1; + const off = on(type, handler); + return () => { + live -= 1; + off(); + }; + }) as typeof harness.on; + const subscribe = harness.subscribe.bind(harness); + harness.subscribe = ((listener: never) => { + live += 1; + const off = subscribe(listener); + return () => { + live -= 1; + off(); + }; + }) as typeof harness.subscribe; + + await handle.compile({ model: "google:gemini-3.6-flash", tools: cua.providers.google.toolsets.browser() }).apply(harness); + const afterSwap = live; + // The caller still holds the first pair's release. Calling it must neither + // strand `models` with no live catalog nor drop the handle's grip on the + // pair that *is* live — otherwise the next activation cannot release it. + releaseFirst(); + await handle.compile({ model: "google:gemini-3.6-flash", tools: [cua.tools.browser.snapshot()] }).apply(harness); + expect(live).toBe(afterSwap); + + await harness.prompt("go"); + expect(seen.map((model) => model.api)).toEqual(["google-generative-ai"]); + }); + + it("clears failed-turn state before the next prompt", async () => { + let successfulCalls = 0; + const failing = callerTool("failing", async () => { + throw new Error("expected failure"); + }); + const succeeding = callerTool("succeeding", async () => { + successfulCalls += 1; + return { content: [{ type: "text", text: "ok" }], details: {} }; + }); + const { harness } = await openSession({ + model: "anthropic:claude-opus-5", + tools: [cua.providers.anthropic.tools.browser(), failing, succeeding], + models: modelsFromStream(scriptedStream([ + (model) => assistant(model, [{ type: "toolCall", id: "fail", name: "failing", arguments: {} }], "toolUse"), + (model) => assistant(model, [{ type: "toolCall", id: "succeed", name: "succeeding", arguments: {} }], "toolUse"), + (model) => assistant(model, [{ type: "text", text: "done" }]), + ]), "anthropic"), + }); + harness.on("tool_result", (event) => (event.toolName === "failing" ? { terminate: true } : undefined)); + + await harness.prompt("fail once"); + await harness.prompt("try again"); + + expect(successfulCalls).toBe(1); + }); +}); diff --git a/packages/agent/test/attach.test.ts b/packages/agent/test/attach.test.ts index fbffca7..bb5a71b 100644 --- a/packages/agent/test/attach.test.ts +++ b/packages/agent/test/attach.test.ts @@ -4,8 +4,10 @@ import { createCuaModels, cua, GOOGLE_CUA_INTERACTIONS_API, + OPENAI_CUA_COMPUTER_API, type AssistantMessage, type Context, + type CuaSimpleStreamOptions, type Model, } from "@onkernel/cua-ai"; import type Kernel from "@onkernel/sdk"; @@ -28,9 +30,9 @@ function assistant(model: Model): AssistantMessage { }; } -function recordingStream(seen: { model: Model; context: Context }[]): StreamFn { - return (model, context) => { - seen.push({ model, context: { ...context, tools: context.tools?.slice() } }); +function recordingStream(seen: { model: Model; context: Context; options?: CuaSimpleStreamOptions }[]): StreamFn { + return (model, context, options) => { + seen.push({ model, context: { ...context, tools: context.tools?.slice() }, options: options as CuaSimpleStreamOptions }); const stream = createAssistantMessageEventStream(); const message = assistant(model); stream.push({ type: "start", partial: message }); @@ -77,7 +79,7 @@ describe("attach", () => { }); it("drives a plain pi Agent with no CUA agent class", async () => { - const seen: { model: Model; context: Context }[] = []; + const seen: { model: Model; context: Context; options?: CuaSimpleStreamOptions }[] = []; const handle = attach({ browser, client }); const compiled = handle.compile({ model: "openai:gpt-5.5", tools: [cua.tools.browser.snapshot()] }); @@ -93,7 +95,7 @@ describe("attach", () => { }); it("drives a plain pi AgentHarness, with CUA's behaviors installed", async () => { - const seen: { model: Model; context: Context }[] = []; + const seen: { model: Model; context: Context; options?: CuaSimpleStreamOptions }[] = []; const handle = attach({ browser, client, models: modelsFromStream(recordingStream(seen)) }); const compiled = handle.compile({ model: "openai:gpt-5.5", tools: [cua.tools.browser.snapshot()] }); const session = await new InMemorySessionRepo().create(); @@ -104,14 +106,41 @@ describe("attach", () => { models: compiled.models, tools: [...compiled.tools], activeToolNames: compiled.tools.map((tool) => tool.name), - } as never); - const uninstall = compiled.install(harness); + }); + const release = compiled.activate(harness); await harness.prompt("go"); expect(seen).toHaveLength(1); expect(seen[0]!.context.tools?.map((tool) => tool.name)).toEqual(["browser_snapshot"]); - expect(typeof uninstall).toBe("function"); - uninstall(); + expect(typeof release).toBe("function"); + release(); + }); + + it("streams the live catalog's tool plan after a swap, not the one the harness was built with", async () => { + const seen: { model: Model; context: Context; options?: CuaSimpleStreamOptions }[] = []; + const handle = attach({ browser, client, models: modelsFromStream(recordingStream(seen)) }); + const first = handle.compile({ model: "openai:gpt-5.5", tools: [cua.tools.browser.snapshot()] }); + const session = await new InMemorySessionRepo().create(); + + const harness = new AgentHarness({ + session, + model: first.model, + models: first.models, + tools: [...first.tools], + activeToolNames: first.tools.map((tool) => tool.name), + }); + first.activate(harness); + + const second = handle.compile({ model: "openai:gpt-5.5", tools: [cua.providers.openai.tools.computer()] }); + await second.apply(harness); + await harness.prompt("go"); + + // pi fixes `models` at construction while the headers, payload transforms + // and tool plan it carries are per-catalog, so a per-compile collection + // would keep sending the first catalog's plan for the rest of the session. + expect(seen).toHaveLength(1); + expect(seen[0]!.options?.cuaIncomingToolPlan?.openaiComputerName).toBe("computer"); + expect(seen[0]!.model.api).toBe(OPENAI_CUA_COMPUTER_API); }); it("spends an empty-response retry only when the follow-up is queued", async () => { diff --git a/packages/agent/test/e2e.live.test.ts b/packages/agent/test/e2e.live.test.ts index 80b1638..2b9355e 100644 --- a/packages/agent/test/e2e.live.test.ts +++ b/packages/agent/test/e2e.live.test.ts @@ -1,8 +1,9 @@ import Kernel from "@onkernel/sdk"; import { describe, expect, it } from "vitest"; import { - CuaAgent, - CuaAgentHarness, + Agent, + AgentHarness, + attach, cua, InMemorySessionRepo, type AgentEvent, @@ -259,18 +260,17 @@ describe("Cua live e2e", () => { const test = shouldRunCase(c) ? it : it.skip; test( - `${c.name}: CuaAgent executes browser steps`, + `${c.name}: a plain pi Agent executes browser steps`, async () => { await withBrowser(async (client, browser) => { const stats = createRunStats(); - const agent = new CuaAgent({ - browser, - client, - getApiKey: () => apiKeyForCase(c), + const compiled = attach({ browser, client }).compile({ model: c.modelRef, tools: toolsForCase(c) }); + const agent = new Agent({ + streamFn: (model, context, options) => compiled.models.streamSimple(model, context, options), afterToolCall: async () => ({ terminate: true }), - tools: toolsForCase(c), initialState: { - model: c.modelRef, + model: compiled.model, + tools: [...compiled.agentTools], systemPrompt: systemPromptForCase(c), }, }); @@ -286,18 +286,20 @@ describe("Cua live e2e", () => { ); test( - `${c.name}: CuaAgentHarness executes browser steps`, + `${c.name}: a plain pi AgentHarness executes browser steps`, async () => { await withBrowser(async (client, browser) => { const stats = createRunStats(); - const harness = new CuaAgentHarness({ + const compiled = attach({ browser, client }).compile({ model: c.modelRef, tools: toolsForCase(c) }); + const harness = new AgentHarness({ ...(await createHarnessServices(`${c.name}-harness`)), - browser, - client, - model: c.modelRef, - tools: toolsForCase(c), + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), systemPrompt: systemPromptForCase(c), }); + compiled.activate(harness); harness.on("tool_result", () => ({ terminate: true })); harness.subscribe((event) => { @@ -316,21 +318,17 @@ describe("Cua live e2e", () => { const test = shouldRunSwitchCase(c) ? it : it.skip; test( - `${c.name}: CuaAgent switches models after a turn`, + `${c.name}: a plain pi Agent switches models after a turn`, async () => { await withBrowser(async (client, browser) => { let stats = createRunStats(); - const agent = new CuaAgent({ - browser, - client, - getApiKey: (provider) => { - if (provider === c.from.modelRef.split(":")[0]) return apiKeyForCase(c.from); - if (provider === c.to.modelRef.split(":")[0]) return apiKeyForCase(c.to); - return undefined; - }, - tools: modelSwitchTools(), + const handle = attach({ browser, client }); + const from = handle.compile({ model: c.from.modelRef, tools: modelSwitchTools() }); + const agent = new Agent({ + streamFn: (model, context, options) => handle.models.streamSimple(model, context, options), initialState: { - model: c.from.modelRef, + model: from.model, + tools: [...from.agentTools], systemPrompt: "Use only the explicitly selected screenshot tool.", }, }); @@ -341,8 +339,12 @@ describe("Cua live e2e", () => { await agent.prompt(modelSwitchPrompt); assertStats(stats, c.from, "agent"); + // A switch recompiles: the new model carries the transport its + // tools derive, and its executables replace the old pair. stats = createRunStats(); - agent.state.model = c.to.modelRef; + const to = handle.compile({ model: c.to.modelRef, tools: modelSwitchTools() }); + agent.state.model = to.model; + agent.state.tools = [...to.agentTools]; await agent.prompt(modelSwitchPrompt); assertStats(stats, c.to, "agent"); }); @@ -351,18 +353,21 @@ describe("Cua live e2e", () => { ); test( - `${c.name}: CuaAgentHarness switches models after a turn`, + `${c.name}: a plain pi AgentHarness switches models after a turn`, async () => { await withBrowser(async (client, browser) => { let stats = createRunStats(); - const harness = new CuaAgentHarness({ + const handle = attach({ browser, client }); + const from = handle.compile({ model: c.from.modelRef, tools: modelSwitchTools() }); + const harness = new AgentHarness({ ...(await createHarnessServices(`${c.name}-harness-switch`)), - browser, - client, - model: c.from.modelRef, - tools: modelSwitchTools(), + model: from.model, + models: from.models, + tools: [...from.tools], + activeToolNames: from.tools.map((tool) => tool.name), systemPrompt: "Use only the explicitly selected screenshot tool.", }); + from.activate(harness); harness.subscribe((event) => { recordRunEvent(stats, event); }); @@ -371,7 +376,7 @@ describe("Cua live e2e", () => { assertStats(stats, c.from, "harness"); stats = createRunStats(); - await harness.setModel(c.to.modelRef); + await handle.compile({ model: c.to.modelRef, tools: modelSwitchTools() }).apply(harness); await harness.prompt(modelSwitchPrompt); assertStats(stats, c.to, "harness"); }); diff --git a/packages/agent/test/harness-context.test.ts b/packages/agent/test/harness-context.test.ts index fa6ae63..89e3f06 100644 --- a/packages/agent/test/harness-context.test.ts +++ b/packages/agent/test/harness-context.test.ts @@ -14,7 +14,8 @@ import { createEditTool, createReadTool, createWriteTool, - CuaAgentHarness, + AgentHarness, + attach, InMemorySessionRepo, NodeExecutionEnv, type AgentHarnessTool, @@ -71,11 +72,27 @@ function modelsFromStream(streamFn: StreamFn, provider = "openai") { return models; } -async function harnessSession() { - return new InMemorySessionRepo().create(); +/** Build a stock pi harness from a handle, exactly as a consumer does. */ +async function openHarness(options: { + models: ReturnType; + tools: readonly AgentHarnessTool[]; + toolContext: TContext; +}): Promise> { + const handle = attach({ browser, client, models: options.models }); + const compiled = handle.compile({ model: "openai:gpt-5.5", tools: options.tools }); + const harness = new AgentHarness({ + session: await new InMemorySessionRepo().create(), + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), + toolContext: options.toolContext, + }); + compiled.activate(harness); + return harness; } -describe("CuaAgentHarness tool context", () => { +describe("harness tool context", () => { it("delivers the exact supplied context object to custom harness tools", async () => { interface CustomContext { env: ExecutionToolContext["env"]; @@ -94,11 +111,7 @@ describe("CuaAgentHarness tool context", () => { return { content: [{ type: "text", text: "ok" }], details: {} }; }, }; - const harness = new CuaAgentHarness({ - browser, - client, - session: await harnessSession(), - model: "openai:gpt-5.5", + const harness = await openHarness({ models: modelsFromStream(scriptedStream([ (model) => assistant(model, [{ type: "toolCall", id: "call-1", name: "custom_context", arguments: {} }], "toolUse"), (model) => assistant(model, [{ type: "text", text: "done" }]), @@ -115,11 +128,7 @@ describe("CuaAgentHarness tool context", () => { it("runs pi's native read/write/edit/bash tools against the context's execution env", async () => { const cwd = mkdtempSync(join(tmpdir(), "cua-harness-tools-")); - const harness = new CuaAgentHarness({ - browser, - client, - session: await harnessSession(), - model: "openai:gpt-5.5", + const harness = await openHarness({ models: modelsFromStream(scriptedStream([ (model) => assistant(model, [{ type: "toolCall", id: "write-1", name: "write", arguments: { path: "notes.txt", content: "hello cua\n" } }], "toolUse"), (model) => assistant(model, [{ type: "toolCall", id: "edit-1", name: "edit", arguments: { path: "notes.txt", edits: [{ oldText: "hello", newText: "goodbye" }] } }], "toolUse"), diff --git a/packages/agent/test/openai-deferred-tools.test.ts b/packages/agent/test/openai-deferred-tools.test.ts deleted file mode 100644 index 962725d..0000000 --- a/packages/agent/test/openai-deferred-tools.test.ts +++ /dev/null @@ -1,109 +0,0 @@ -import type { AgentTool } from "@earendil-works/pi-agent-core"; -import type { ToolCall } from "@onkernel/cua-ai"; -import type Kernel from "@onkernel/sdk"; -import { describe, expect, it, vi } from "vitest"; -import { CuaAgent, type KernelBrowser } from "../src/index"; - -const { responsesCreate } = vi.hoisted(() => ({ responsesCreate: vi.fn() })); - -vi.mock("openai", () => ({ - default: class { - responses = { - create: (payload: unknown) => ({ - withResponse: async () => ({ - data: responseEvents(responsesCreate(payload)), - response: { status: 200, headers: new Headers() }, - }), - }), - }; - }, -})); - -async function* responseEvents(response: Record) { - yield { type: "response.created", response: { id: response.id } }; - for (const [outputIndex, item] of ((response.output as unknown[]) ?? []).entries()) { - yield { type: "response.output_item.added", output_index: outputIndex, item }; - yield { type: "response.output_item.done", output_index: outputIndex, item }; - } - yield { type: "response.completed", response }; -} - -function functionCall(id: string, callId: string, name: string, namespace?: string) { - return { - type: "function_call", - id, - call_id: callId, - name, - arguments: "{}", - status: "completed", - ...(namespace ? { namespace } : {}), - }; -} - -function response(id: string, output: unknown[]) { - return { - id, - status: "completed", - output, - usage: { input_tokens: 1, output_tokens: 1, total_tokens: 2 }, - }; -} - -function callerTool(name: string, execute: AgentTool["execute"], executionMode?: AgentTool["executionMode"]): AgentTool { - return { - name, - label: name, - description: `${name} tool`, - parameters: { type: "object", properties: {}, additionalProperties: false } as never, - execute, - ...(executionMode ? { executionMode } : {}), - }; -} - -describe("OpenAI deferred tool namespace continuation", () => { - it("executes a deferred-added call and replays its provider namespace", async () => { - responsesCreate - .mockReturnValueOnce(response("resp_loader", [functionCall("fc_loader", "call_loader", "loader")])) - .mockReturnValueOnce(response("resp_added", [functionCall("fc_added", "call_added", "added", "deferred_tools")])) - .mockReturnValueOnce(response("resp_done", [{ - type: "message", - id: "msg_done", - role: "assistant", - status: "completed", - content: [{ type: "output_text", text: "done", annotations: [] }], - }])); - - const addedExecute = vi.fn(async () => ({ content: [{ type: "text" as const, text: "added result" }], details: {} })); - const added = callerTool("added", addedExecute); - let agent!: CuaAgent; - const loader = callerTool("loader", async () => { - agent.setTools([...agent.getTools(), added]); - return { content: [{ type: "text", text: "loaded" }], details: {} }; - }, "sequential"); - agent = new CuaAgent({ - browser: { session_id: "browser_123" } as KernelBrowser, - client: {} as Kernel, - tools: [loader], - initialState: { model: "openai:gpt-5.5" }, - getApiKey: () => "test", - }); - - await agent.prompt("load and run the added tool"); - - expect(addedExecute).toHaveBeenCalledTimes(1); - const deferredCall = agent.state.messages - .find((message) => message.role === "assistant" && message.content.some((part) => part.type === "toolCall" && part.name === "added")) - ?.content.find((part): part is ToolCall => part.type === "toolCall") as (ToolCall & { namespace?: string }) | undefined; - expect(deferredCall?.namespace).toBe("deferred_tools"); - - const secondPayload = responsesCreate.mock.calls[1]?.[0] as { input: Array> }; - expect(secondPayload.input).toContainEqual(expect.objectContaining({ type: "tool_search_output" })); - const replayPayload = responsesCreate.mock.calls[2]?.[0] as { input: Array> }; - expect(replayPayload.input).toContainEqual(expect.objectContaining({ - type: "function_call", - call_id: "call_added", - name: "added", - namespace: "deferred_tools", - })); - }); -}); diff --git a/packages/agent/test/published-declarations.test.ts b/packages/agent/test/published-declarations.test.ts index 2442aa4..f174c95 100644 --- a/packages/agent/test/published-declarations.test.ts +++ b/packages/agent/test/published-declarations.test.ts @@ -16,8 +16,9 @@ const repoRoot = resolve(agentRoot, "..", ".."); const CONSUMER = ` import { Type } from "typebox"; import { - CuaAgent, - CuaAgentHarness, + Agent, + AgentHarness, + attach, InMemorySessionRepo, NodeExecutionEnv, createBashTool, @@ -63,32 +64,31 @@ const agentTools: readonly CuaAgentTool[] = [...cua.toolsets.browser()]; async function build() { const session = await new InMemorySessionRepo().create(); - const harness = new CuaAgentHarness({ - browser, - client, + const handle = attach({ browser, client }); + + const compiled = handle.compile({ model: "openai:gpt-5.6-sol", tools: harnessTools }); + const harness = new AgentHarness({ session, - model: "openai:gpt-5.6-sol", - tools: harnessTools, + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), toolContext, }); - const contextFree = new CuaAgentHarness({ - browser, - client, - session, - model: getCuaModel("openai:gpt-5.6-sol"), - tools: [], - }); - new CuaAgentHarness({ - browser, - client, - session, - model: "openai:gpt-5.6-sol", - tools: [], - // @ts-expect-error env was removed; the tool context now carries the execution env - env: new NodeExecutionEnv({ cwd: process.cwd() }), + const release = compiled.activate(harness); + // A swap compiles for the same tool context the harness delivers. + await handle.compile({ model: getCuaModel("openai:gpt-5.6-sol"), tools: [] }).apply(harness); + + const lowLevel = handle.compile({ model: "openai:gpt-5.6-sol", tools: agentTools }); + const agent = new Agent({ + streamFn: (model, context, options) => lowLevel.models.streamSimple(model, context, options), + initialState: { model: lowLevel.model, tools: [...lowLevel.agentTools] }, }); - const agent = new CuaAgent({ browser, client, tools: agentTools, initialState: { model: "openai:gpt-5.6-sol" } }); - return { harness, contextFree, agent }; + + // @ts-expect-error the browser and client belong to attach(), not to a compile + handle.compile({ browser, model: "openai:gpt-5.6-sol", tools: [] }); + + return { harness, agent, release }; } void build; diff --git a/packages/agent/test/tool-manager.test.ts b/packages/agent/test/tool-manager.test.ts index c9d93d9..15cc28d 100644 --- a/packages/agent/test/tool-manager.test.ts +++ b/packages/agent/test/tool-manager.test.ts @@ -48,13 +48,6 @@ describe("CuaToolManager declaration projection", () => { expect("agentTools" in manager.catalog).toBe(false); }); - it("keeps the caller list as the sole owner of requested objects", () => { - const spec = cua.tools.browser.snapshot(); - const tool = callerTool("lookup"); - const manager = new CuaToolManager(setup(), "openai:gpt-5.5", [spec, tool]); - expect(manager.getTools()[0]).toBe(spec); - expect(manager.getTools()[1]).toBe(tool); - }); }); describe("CuaToolManager identity join", () => { @@ -93,63 +86,29 @@ describe("CuaToolManager identity join", () => { }); describe("CuaToolManager transport derivation", () => { - it("derives the compiled model's api from selected tools on construction and on both mutation paths", () => { - const manager = new CuaToolManager(setup(), "google:gemini-3.6-flash", [cua.tools.browser.snapshot()]); - expect(manager.catalog.model.api).toBe("google-generative-ai"); - - manager.commit(manager.prepareTools(cua.providers.google.toolsets.browser())); - expect(manager.catalog.model.api).toBe(GOOGLE_CUA_INTERACTIONS_API); - - manager.commit(manager.prepareModel("google:gemini-3.6-flash")); - expect(manager.catalog.model.api).toBe(GOOGLE_CUA_INTERACTIONS_API); + it("derives the compiled model's api from the tools selected with it", () => { + const resources = setup(); + const cdp = new CuaToolManager(resources, "google:gemini-3.6-flash", [cua.tools.browser.snapshot()]); + const native = new CuaToolManager(resources, "google:gemini-3.6-flash", cua.providers.google.toolsets.browser()); - manager.commit(manager.prepareTools([cua.tools.browser.snapshot()])); - expect(manager.catalog.model.api).toBe("google-generative-ai"); + expect(cdp.catalog.model.api).toBe("google-generative-ai"); + expect(native.catalog.model.api).toBe(GOOGLE_CUA_INTERACTIONS_API); }); }); -describe("CuaToolManager implementation identity", () => { - it("materializes each spec exactly once across model and tool recompilation", () => { +describe("CuaToolManager materialization", () => { + it("materializes each spec exactly once, however many pairs it is compiled into", () => { const resources = setup(); const spy = vi.spyOn(resources, "materialize"); const spec = cua.tools.browser.snapshot(); - const manager = new CuaToolManager(resources, "openai:gpt-5.5", [spec]); - manager.commit(manager.prepareModel("openai:gpt-5.6-sol")); - manager.commit(manager.prepareTools([...manager.getTools(), callerTool("added")])); + new CuaToolManager(resources, "openai:gpt-5.5", [spec]); + new CuaToolManager(resources, "openai:gpt-5.6-sol", [spec]); + new CuaToolManager(resources, "openai:gpt-5.6-sol", [spec, callerTool("added")]); - // Every materialization for the same spec object returned one identical executable. + // The executable is cached per pool and per spec object, so pi sees one + // stable implementation across every recompile. expect(spy.mock.calls.length).toBeGreaterThan(1); expect(new Set(spy.mock.results.map((result) => result.value)).size).toBe(1); }); - - it("keeps the same spec object stable across model recompilation", () => { - const manager = new CuaToolManager(setup(), "openai:gpt-5.5", [cua.tools.browser.snapshot()]); - const first = manager.prepareModel("openai:gpt-5.6-sol"); - const second = manager.prepareModel("openai:gpt-5.6-sol"); - expect(second.fingerprints).toEqual(first.fingerprints); - }); - - it("treats a freshly created spec object as a conservative replacement", () => { - const manager = new CuaToolManager(setup(), "openai:gpt-5.5", [cua.tools.browser.snapshot()]); - const stable = manager.prepareModel("openai:gpt-5.6-sol"); - const replaced = manager.prepareTools([cua.tools.browser.snapshot()]); - expect(replaced.catalog.entries[0]?.fingerprint).toBe(stable.catalog.entries[0]?.fingerprint); - expect(replaced.fingerprints[0]).not.toBe(stable.fingerprints[0]); - }); - - it("retains implementation identity for a new wrapper reusing the same execute function", () => { - const sharedExecute: AgentTool["execute"] = async () => ({ content: [{ type: "text", text: "ok" }], details: {} }); - const original = callerTool("worker", sharedExecute); - const manager = new CuaToolManager(setup(), "openai:gpt-5.5", [original]); - const baseline = manager.prepareModel("openai:gpt-5.6-sol"); - - const rewrapped = { ...original }; - expect(rewrapped).not.toBe(original); - expect(manager.prepareTools([rewrapped]).fingerprints).toEqual(baseline.fingerprints); - - const freshExecute = manager.prepareTools([callerTool("worker")]); - expect(freshExecute.catalog.entries[0]?.fingerprint).toBe(baseline.catalog.entries[0]?.fingerprint); - expect(freshExecute.fingerprints[0]).not.toBe(baseline.fingerprints[0]); - }); }); diff --git a/packages/ai/README.md b/packages/ai/README.md index 5d6ec5c..75c43d3 100644 --- a/packages/ai/README.md +++ b/packages/ai/README.md @@ -231,8 +231,8 @@ unioned and deduplicated; exact-value conflicts throw. Ordinary function tools are marked eligible only where pi 0.83.0 supports deferred loading. Provider-native tools are eager-only. The catalog itself does -not guess when tools were added; `CuaAgent`/`CuaAgentHarness` record in-tool -additions through pi's active-tool change entries. +not guess when tools were added; a caller that adds tools mid-turn records the +addition through pi's active-tool change entries. ## Provider behavior diff --git a/packages/ai/src/providers.ts b/packages/ai/src/providers.ts index 3b19e65..3e3fadb 100644 --- a/packages/ai/src/providers.ts +++ b/packages/ai/src/providers.ts @@ -60,7 +60,7 @@ let defaultCuaModels: MutableModels | undefined; /** * Shared default {@link createCuaModels} collection, created on first use. * - * `CuaAgent` and `CuaAgentHarness` stream through this instance unless given + * cua-agent streams through this instance unless given * another one. Auth resolves from the documented CUA env-var convention (see * `cuaApiKeyEnvVarsForProvider`); pass an explicit `options.apiKey` per * request to override. diff --git a/packages/cli/CHANGELOG.md b/packages/cli/CHANGELOG.md index a87266a..15d20c7 100644 --- a/packages/cli/CHANGELOG.md +++ b/packages/cli/CHANGELOG.md @@ -2,6 +2,11 @@ ## Unreleased +- The CLI now builds a stock pi `AgentHarness` from `attach()` instead of + `CuaAgentHarness`. `buildCuaHarness()` returns `{ harness, catalog }`: the + harness is pi's, and `CuaCliCatalog` holds the live (model, tools) selection + that `/model` and `/tools` recompile. Behavior is unchanged, including the + guarantee that a rejected selection leaves the session exactly as it was. - `/tools` now offers the model's whole tool menu instead of filtering the list the CLI composed. Tools the CLI did not choose — `playwright_execute`, the computer toolset, a provider-native surface — can be enabled, and tools the diff --git a/packages/cli/README.md b/packages/cli/README.md index c5ae1b1..2e3c9d2 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -1,7 +1,7 @@ # `@onkernel/cua-cli` The CLI / TUI binary for the [`cua`](../../README.md) monorepo. Wires -[`@onkernel/cua-agent`](../agent)'s `CuaAgentHarness` to +[`@onkernel/cua-agent`](../agent)'s `attach()` handle and a pi `AgentHarness` to [`pi-tui`](https://www.npmjs.com/package/@earendil-works/pi-tui) for an interactive front-end and to [`pi-coding-agent`](https://www.npmjs.com/package/@earendil-works/pi-coding-agent)'s diff --git a/packages/cli/src/cli-harness.ts b/packages/cli/src/cli-harness.ts index aad9178..f62cc81 100644 --- a/packages/cli/src/cli-harness.ts +++ b/packages/cli/src/cli-harness.ts @@ -23,7 +23,13 @@ import { type ModelActionType, } from "./action/prompts"; import { runAction, emitCompact } from "./action/harness-runner"; -import { buildCuaHarness, defaultApplicationTools, defaultInteractionTools } from "./harness"; +import { + buildCuaHarness, + type CuaCliCatalog, + type CuaCliHarness, + defaultApplicationTools, + defaultInteractionTools, +} from "./harness"; import { provisionBrowser } from "./harness-browser"; import { DEFAULT_CUA_MODEL_REF, listSupportedModels, resolveCuaModelRef } from "./harness-models"; import { @@ -472,7 +478,8 @@ interface HarnessRuntime { skills: Skill[]; contextFiles: ContextFile[]; applicationTools: ReturnType; - harness: ReturnType; + harness: CuaCliHarness; + catalog: CuaCliCatalog; provider: string; modelRef: CuaModelRef; } @@ -567,7 +574,7 @@ async function finishHarnessRuntime( const thinkingLevel = mapThinkingLevel(flags.thinking); const baseUrlOverride = providerBaseUrlOverride(provider); const applicationTools = defaultApplicationTools(); - const harness = buildCuaHarness({ + const { harness, catalog } = buildCuaHarness({ cwd, client: provisioned.handle.client, browser: provisioned.handle.browser, @@ -588,6 +595,7 @@ async function finishHarnessRuntime( contextFiles, applicationTools, harness, + catalog, provider, modelRef: auth.modelRef, }; @@ -672,6 +680,7 @@ export async function runInteractiveCommand( return await runInteractive({ cwd: process.cwd(), harness: runtime.harness, + catalog: runtime.catalog, browserHandle: runtime.handle, session: runtime.session, skills: runtime.skills, diff --git a/packages/cli/src/harness.ts b/packages/cli/src/harness.ts index a21a5de..78e4a3c 100644 --- a/packages/cli/src/harness.ts +++ b/packages/cli/src/harness.ts @@ -1,7 +1,10 @@ import { - CuaAgentHarness, - type CuaAgentHarnessOptions, + AgentHarness, + attach, + type CuaAttachOptions, + type CuaBrowserHandle, type CuaHarnessTool, + type CuaModelInput, formatSkillsForSystemPrompt, type KernelBrowser, type Session, @@ -21,20 +24,86 @@ import { } from "@onkernel/cua-ai"; import type Kernel from "@onkernel/sdk"; import { + type AgentHarnessTool, createBashTool, createEditTool, createReadTool, createWriteTool, type ExecutionToolContext, + type PromptTemplate, } from "@earendil-works/pi-agent-core"; import { NodeExecutionEnv } from "@earendil-works/pi-agent-core/node"; import type { ContextFile } from "./harness-skills"; -/** CLI harness: a CUA harness whose tool context carries the coding tools' execution environment. */ -export type CuaCliHarness = CuaAgentHarness; /** One tool in the CLI harness's caller-owned list. */ export type CuaCliTool = CuaHarnessTool; +/** + * The CLI's harness: stock pi, with no CUA class wrapping it. Its tool type is + * pi's own — {@link CuaCliTool} is the caller-owned input the catalog compiles + * from, and a CUA spec is not executable until the handle materializes it. + */ +export type CuaCliHarness = AgentHarness>; + +/** + * The live (model, tools) selection, and the compile-then-swap that changes it. + * + * `attach()` hands back an immutable compiled pair, so the current selection + * lives with whoever can change it — here, `/model` and `/tools`. Both steps + * fail safe: `compile()` throws before anything mutates, and `apply()` restores + * the previous pair if pi rejects the new one, so a rejected selection leaves + * the session exactly as it was. + */ +export class CuaCliCatalog { + private selection: CuaModelInput; + private requested: readonly CuaCliTool[]; + + constructor( + private readonly handle: CuaBrowserHandle, + private readonly harness: CuaCliHarness, + selection: CuaModelInput, + requested: readonly CuaCliTool[], + ) { + this.selection = selection; + this.requested = [...requested]; + } + + /** The caller-owned tool list, as selected — CUA specs, not materialized pi tools. */ + getTools(): readonly CuaCliTool[] { + return [...this.requested]; + } + + setTools(tools: readonly CuaCliTool[]): Promise { + return this.swap(this.selection, tools); + } + + setModel(model: CuaModelInput): Promise { + return this.swap(model, this.requested); + } + + /** + * Select a model and its tool list in one compile. Staging the two in + * sequence would compile an intermediate catalog whose derived transport + * differs from both the old and the new one. + */ + setModelAndTools(model: CuaModelInput, tools: readonly CuaCliTool[]): Promise { + return this.swap(model, tools); + } + + private async swap(model: CuaModelInput, tools: readonly CuaCliTool[]): Promise { + const compiled = this.handle.compile({ model, tools }); + await compiled.apply(this.harness); + this.selection = model; + this.requested = [...tools]; + } +} + +/** A CLI session: stock pi driving the agent, a CUA handle owning the browser. */ +export interface CuaCliSession { + readonly harness: CuaCliHarness; + readonly catalog: CuaCliCatalog; +} + export interface BuildCuaHarnessOptions { cwd: string; client: Kernel; @@ -47,14 +116,14 @@ export interface BuildCuaHarnessOptions { /** Override the CLI's explicit interaction + coding tool list. */ tools?: CuaCliTool[]; models?: Models; - toolResultImageReplayLimit?: CuaAgentHarnessOptions["toolResultImageReplayLimit"]; - responseThreading?: CuaAgentHarnessOptions["responseThreading"]; - retry?: CuaAgentHarnessOptions["retry"]; + toolResultImageReplayLimit?: CuaAttachOptions["toolResultImageReplayLimit"]; + responseThreading?: CuaAttachOptions["responseThreading"]; + retry?: CuaAttachOptions["retry"]; modelBaseUrl?: string; } -/** Build the CLI harness with one explicit tool list and a caller-owned prompt. */ -export function buildCuaHarness(opts: BuildCuaHarnessOptions): CuaCliHarness { +/** Build the CLI session with one explicit tool list and a caller-owned prompt. */ +export function buildCuaHarness(opts: BuildCuaHarnessOptions): CuaCliSession { const skills = opts.skills ?? []; const contextFiles = opts.contextFiles ?? []; const model: CuaModelRef | Model = opts.modelBaseUrl @@ -64,21 +133,28 @@ export function buildCuaHarness(opts: BuildCuaHarnessOptions): CuaCliHarness { ...defaultInteractionTools(opts.model), ...defaultApplicationTools(), ]; - return new CuaAgentHarness({ - session: opts.session, - model, - models: opts.models, + const handle = attach({ browser: opts.browser, client: opts.client, - tools, + models: opts.models, + toolResultImageReplayLimit: opts.toolResultImageReplayLimit, + responseThreading: opts.responseThreading, + retry: opts.retry, + }); + const compiled = handle.compile({ model, tools }); + const harness: CuaCliHarness = new AgentHarness({ + session: opts.session, + model: compiled.model, + models: compiled.models, + tools: [...compiled.tools], + activeToolNames: compiled.tools.map((tool) => tool.name), toolContext: { env: new NodeExecutionEnv({ cwd: opts.cwd }) }, resources: { skills }, thinkingLevel: opts.thinkingLevel, systemPrompt: ({ resources }) => composeSystemPrompt(resources.skills ?? [], contextFiles), - toolResultImageReplayLimit: opts.toolResultImageReplayLimit, - responseThreading: opts.responseThreading, - retry: opts.retry, }); + compiled.activate(harness); + return { harness, catalog: new CuaCliCatalog(handle, harness, model, tools) }; } /** Coding tools owned by the CLI application rather than inferred from a compiled catalog. */ diff --git a/packages/cli/src/tui/main.ts b/packages/cli/src/tui/main.ts index 2f7047f..568534f 100644 --- a/packages/cli/src/tui/main.ts +++ b/packages/cli/src/tui/main.ts @@ -20,7 +20,7 @@ import { import { initTheme } from "@earendil-works/pi-coding-agent"; import { homedir } from "node:os"; import { type CuaModelRef, listCuaModels, type Model } from "@onkernel/cua-ai"; -import { type CuaCliHarness, type CuaCliTool } from "../harness"; +import { type CuaCliCatalog, type CuaCliHarness, type CuaCliTool } from "../harness"; import type { CuaBrowserHandle } from "../harness-browser"; import { resolveCuaModelRef } from "../harness-models"; import { updateNamedSessionRuntime } from "../harness-named-sessions"; @@ -43,6 +43,8 @@ import { cuaVersion } from "./version"; export interface InteractiveOptions { cwd: string; harness: CuaCliHarness; + /** The live (model, tools) selection `/model` and `/tools` change. */ + catalog: CuaCliCatalog; browserHandle: CuaBrowserHandle; session: Session; skills?: Skill[]; @@ -379,9 +381,9 @@ export async function runInteractive(opts: InteractiveOptions): Promise // Native catalogs are incompatible across providers, and the selected // tools decide the transport, so the new model and its interaction // catalog have to compile as one pair rather than in sequence. - await opts.harness.setModelAndTools(resolved, installedTools); + await opts.catalog.setModelAndTools(resolved, installedTools); } else { - await opts.harness.setModel(resolved); + await opts.catalog.setModel(resolved); } const model = opts.harness.getModel(); footer.update({ @@ -453,7 +455,7 @@ export async function runInteractive(opts: InteractiveOptions): Promise catalogQueue.run(async () => { const next = toolsForSelection(items, enabledKeys); try { - await opts.harness.setTools(next); + await opts.catalog.setTools(next); toolSelectionCustomized = !sameToolList(next, baselineTools); messages.addNotice(`tools → ${next.length} enabled`); debug?.log("tools_applied", { enabled: next.length, baseline: baselineTools.length }); @@ -472,7 +474,7 @@ export async function runInteractive(opts: InteractiveOptions): Promise requestRender("tools_no_ref"); return; } - const live = opts.harness.getTools(); + const live = opts.catalog.getTools(); // Availability is pairwise, so the menu is rebuilt against each staged // selection rather than computed once when the picker opens. const menuFor = (selected: readonly CuaCliTool[]) => describeMenu(modelRef, opts.applicationTools, selected); @@ -770,7 +772,7 @@ function tryResolveModelRef(input: string | undefined): CuaModelRef | undefined * customization would have shrunk. */ function composeBaselineTools(opts: InteractiveOptions, ref: CuaModelRef | undefined): readonly CuaCliTool[] { - if (!opts.interactionToolsForModel || !ref) return opts.harness.getTools(); + if (!opts.interactionToolsForModel || !ref) return opts.catalog.getTools(); return [...opts.interactionToolsForModel(ref), ...opts.applicationTools]; } diff --git a/packages/cli/test/fixtures/harness.ts b/packages/cli/test/fixtures/harness.ts index ca5a5e9..27420d1 100644 --- a/packages/cli/test/fixtures/harness.ts +++ b/packages/cli/test/fixtures/harness.ts @@ -7,7 +7,7 @@ import { tmpdir } from "node:os"; import { mkdtempSync } from "node:fs"; import { join } from "node:path"; import { parseCuaModelRef } from "@onkernel/cua-ai"; -import { buildCuaHarness, type CuaCliTool, defaultInteractionTools } from "../../src/harness"; +import { buildCuaHarness, type CuaCliSession, type CuaCliTool, defaultInteractionTools } from "../../src/harness"; import { createFakeKernelEnvironment, type FakeKernelEnvironment } from "./fake-kernel"; import type { ScriptedProviderHandle, ScriptedTurn } from "./scripted-provider"; import { createScriptedCuaModels } from "./scripted-provider"; @@ -17,7 +17,8 @@ export interface TestHarnessFixture { kernel: FakeKernelEnvironment; session: Session; cwd: string; - harness: ReturnType; + harness: CuaCliSession["harness"]; + catalog: CuaCliSession["catalog"]; } export interface BuildTestHarnessOptions { @@ -39,7 +40,7 @@ export async function buildTestHarness(opts: BuildTestHarnessOptions): Promise { const contextFiles = fixture.contextFiles ?? []; const applicationTools = fixture.tools ? defaultApplicationTools(cwd) : []; const interactionToolsForModel = fixture.tools ? defaultInteractionTools : undefined; - const harness = buildCuaHarness({ + const { harness, catalog } = buildCuaHarness({ cwd, client: kernel.client, browser: kernel.browser, @@ -64,6 +64,7 @@ async function main(): Promise { const code = await runInteractive({ cwd, harness, + catalog, browserHandle: { client: kernel.client, browser: kernel.browser, diff --git a/packages/cli/test/harness-assembly.test.ts b/packages/cli/test/harness-assembly.test.ts index 0a7fd13..70837f9 100644 --- a/packages/cli/test/harness-assembly.test.ts +++ b/packages/cli/test/harness-assembly.test.ts @@ -41,14 +41,14 @@ describe("buildCuaHarness", () => { const cwd = mkdtempSync(join(tmpdir(), "cua-cli-harness-")); const kernel = createFakeKernelEnvironment(); const session = await new InMemorySessionRepo().create(); - const harness = buildCuaHarness({ + const { harness, catalog } = buildCuaHarness({ cwd, client: kernel.client, browser: kernel.browser, session, model: "openai:gpt-5.5", }); - const toolNames = harness.getTools().map((tool) => tool.name); + const toolNames = catalog.getTools().map((tool) => tool.name); expect(toolNames).toContain("browser_click"); expect(toolNames).toContain("browser_screenshot"); expect(toolNames).toContain("browser_act"); @@ -69,7 +69,7 @@ describe("buildCuaHarness", () => { content: "Use the demo workflow.", filePath: join(cwd, "demo.md"), }; - const harness = buildCuaHarness({ + const { harness } = buildCuaHarness({ cwd, client: kernel.client, browser: kernel.browser, @@ -96,7 +96,7 @@ describe("buildCuaHarness", () => { const cwd = mkdtempSync(join(tmpdir(), "cua-cli-harness-")); const kernel = createFakeKernelEnvironment(); const session = await new InMemorySessionRepo().create(); - const harness = buildCuaHarness({ + const { harness } = buildCuaHarness({ cwd, client: kernel.client, browser: kernel.browser, @@ -123,7 +123,7 @@ describe("buildCuaHarness", () => { const cwd = mkdtempSync(join(tmpdir(), "cua-cli-harness-")); const kernel = createFakeKernelEnvironment(); const session = await new InMemorySessionRepo().create(); - const harness = buildCuaHarness({ + const { harness } = buildCuaHarness({ cwd, client: kernel.client, browser: kernel.browser, @@ -147,7 +147,7 @@ describe("buildCuaHarness", () => { const cwd = mkdtempSync(join(tmpdir(), "cua-cli-harness-")); const kernel = createFakeKernelEnvironment(); const session = await new InMemorySessionRepo().create(); - const harness = buildCuaHarness({ + const { harness } = buildCuaHarness({ cwd, client: kernel.client, browser: kernel.browser, diff --git a/packages/cli/test/kimi-reasoning-payload.test.ts b/packages/cli/test/kimi-reasoning-payload.test.ts index 43fd623..d84fcef 100644 --- a/packages/cli/test/kimi-reasoning-payload.test.ts +++ b/packages/cli/test/kimi-reasoning-payload.test.ts @@ -29,7 +29,7 @@ async function capturePayloads(apiKeyEnv: string, model: CuaModelRef): Promise sseResponse())); const kernel = createFakeKernelEnvironment(); - const harness = buildCuaHarness({ + const { harness } = buildCuaHarness({ cwd: mkdtempSync(join(tmpdir(), "cua-kimi-payload-")), client: kernel.client, browser: kernel.browser, diff --git a/packages/cli/test/tool-revalidation.test.ts b/packages/cli/test/tool-revalidation.test.ts index f1fdee6..8b8834c 100644 --- a/packages/cli/test/tool-revalidation.test.ts +++ b/packages/cli/test/tool-revalidation.test.ts @@ -5,7 +5,7 @@ import { buildTestHarness } from "./fixtures/harness"; /** * The `/tools` picker applies a selection of the model's tool menu via - * `harness.setTools()`. These tests pin the behavior the picker relies on: + * `catalog.setTools()`. These tests pin the behavior the picker relies on: * compile-and-validate happens before any mutation, so a rejected selection * leaves the live catalog untouched. */ @@ -19,21 +19,21 @@ describe("/tools selection revalidation", () => { const dropped = items.find((item) => item.group === "native" && item.available)!; const next = baseline.filter((tool) => toolKey(tool) !== dropped.key); - await fixture.harness.setTools(next); - expect(fixture.harness.getTools().map(toolKey)).toEqual(next.map(toolKey)); - expect(fixture.harness.getTools().map(toolKey)).not.toContain(dropped.key); + await fixture.catalog.setTools(next); + expect(fixture.catalog.getTools().map(toolKey)).toEqual(next.map(toolKey)); + expect(fixture.catalog.getTools().map(toolKey)).not.toContain(dropped.key); }); it("rejects a duplicated tool name and leaves the catalog unchanged", async () => { const modelRef = "google:gemini-3.6-flash"; const baseline = [...defaultInteractionTools(modelRef), ...defaultApplicationTools()]; const fixture = await buildTestHarness({ turns: [], modelRef, tools: baseline }); - const before = fixture.harness.getTools().map(toolKey); + const before = fixture.catalog.getTools().map(toolKey); const [first] = baseline; - await expect(fixture.harness.setTools([...baseline, first!])).rejects.toThrow(/requested more than once/); + await expect(fixture.catalog.setTools([...baseline, first!])).rejects.toThrow(/requested more than once/); // Atomicity: the failed compile must not have mutated live state. - expect(fixture.harness.getTools().map(toolKey)).toEqual(before); + expect(fixture.catalog.getTools().map(toolKey)).toEqual(before); }); it("accepts dropping the whole Google native group", async () => { @@ -45,8 +45,8 @@ describe("/tools selection revalidation", () => { const nativeKeys = new Set(items.filter((item) => item.group === "native").flatMap((item) => item.tools.map(toolKey))); const next = baseline.filter((tool) => !nativeKeys.has(toolKey(tool))); - await fixture.harness.setTools(next); - expect(fixture.harness.getTools().map(toolKey)).toEqual(next.map(toolKey)); + await fixture.catalog.setTools(next); + expect(fixture.catalog.getTools().map(toolKey)).toEqual(next.map(toolKey)); }); it("accepts an empty selection (text-only agent)", async () => { @@ -54,8 +54,8 @@ describe("/tools selection revalidation", () => { const baseline = [...defaultInteractionTools(modelRef), ...defaultApplicationTools()]; const fixture = await buildTestHarness({ turns: [], modelRef, tools: baseline }); - await fixture.harness.setTools([]); - expect(fixture.harness.getTools()).toEqual([]); + await fixture.catalog.setTools([]); + expect(fixture.catalog.getTools()).toEqual([]); }); it("recomposes the baseline after a model switch across providers", async () => { @@ -71,10 +71,10 @@ describe("/tools selection revalidation", () => { // Mirrors switchModel(): the new model and its interaction catalog compile // as one pair, because the selected tools decide the transport. - await fixture.harness.setModelAndTools(to, [...defaultInteractionTools(to), ...application]); + await fixture.catalog.setModelAndTools(to, [...defaultInteractionTools(to), ...application]); const expected = [...defaultInteractionTools(to), ...application].map(toolKey); - expect(fixture.harness.getTools().map(toolKey)).toEqual(expected); + expect(fixture.catalog.getTools().map(toolKey)).toEqual(expected); expect(fixture.harness.getModel().provider).toBe("anthropic"); }); @@ -91,7 +91,7 @@ describe("/tools selection revalidation", () => { expect(playwright.available).toBe(true); const enabled = new Set([...selectedKeys(items, baseline), playwright.key]); - await fixture.harness.setTools(toolsForSelection(items, enabled)); - expect(fixture.harness.getTools().map((tool) => tool.name)).toContain("playwright_execute"); + await fixture.catalog.setTools(toolsForSelection(items, enabled)); + expect(fixture.catalog.getTools().map((tool) => tool.name)).toContain("playwright_execute"); }); }); diff --git a/packages/cli/test/tui.fixture.test.ts b/packages/cli/test/tui.fixture.test.ts index dbe23ad..94a8fd4 100644 --- a/packages/cli/test/tui.fixture.test.ts +++ b/packages/cli/test/tui.fixture.test.ts @@ -10,7 +10,7 @@ import { describeMenu } from "../src/tui/tool-selection"; /** * Drive the interactive TUI through ptywright with a scripted provider sitting - * below the real {@link CuaAgentHarness}. The runner script ({@link tuiRunnerPath}) + * below the real harness. The runner script ({@link tuiRunnerPath}) * registers the scripted provider, assembles the harness via the production * {@link buildCuaHarness}, and starts {@link runInteractive}. Each test case * spawns a fresh process with its own per-scenario fixture JSON so the From 9f03fa32b77a37fab4f41388469d872f3e7705e9 Mon Sep 17 00:00:00 2001 From: rgarcia <72655+rgarcia@users.noreply.github.com> Date: Fri, 14 Aug 2026 10:58:41 +0000 Subject: [PATCH 2/2] Point the release skill's smoke check at attach() The published-artifact check imported CuaAgent, which this branch deletes. --- .agents/skills/release/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.agents/skills/release/SKILL.md b/.agents/skills/release/SKILL.md index 8d2f9c0..7bd20e5 100644 --- a/.agents/skills/release/SKILL.md +++ b/.agents/skills/release/SKILL.md @@ -258,7 +258,7 @@ node --input-type=module -e "import('@onkernel/cua-ai').then((m) => { if (typeof ``` For `@onkernel/cua-agent`, install `@onkernel/cua-agent@` the same -way and check `typeof m.CuaAgent === "function"`. For the CLI, install it in a +way and check `typeof m.attach === "function"`. For the CLI, install it in a fresh directory and verify `./node_modules/.bin/cua --help` prints `Usage:`. If a workflow fails after a tag is pushed, do not reuse the same package