Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/release/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,7 @@ node --input-type=module -e "import('@onkernel/cua-ai').then((m) => { if (typeof
```

For `@onkernel/cua-agent`, install `@onkernel/cua-agent@<version>` the same
way and check `typeof m.CuaAgent === "function"`. For the CLI, install it in a
way and check `typeof m.attach === "function"`. For the CLI, install it in a
fresh directory and verify `./node_modules/.bin/cua --help` prints `Usage:`.

If a workflow fails after a tag is pushed, do not reuse the same package
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ packages/
```

**Building your own agent? Start here:** [`packages/agent`](packages/agent)
(`@onkernel/cua-agent`) — `CuaAgent`/`CuaAgentHarness` run the full
computer-use loop against a Kernel browser. It sits on
(`@onkernel/cua-agent`) — `attach()` binds a Kernel browser and compiles a
(model, tools) pair into plain pi objects you drive yourself. It sits on
[`packages/ai`](packages/ai) (`@onkernel/cua-ai`), the model layer with the
pi-ai model catalog, canonical tool schemas, and per-provider
adapters on top of pi-ai; reach for cua-ai directly only when you bring your
Expand Down Expand Up @@ -148,7 +148,7 @@ cua -p -o jsonl "open example.com and tell me the heading"
`pi-agent-core`'s `Agent`/`AgentHarness`. It materializes the caller's exact
catalog over one shared resource pool and executes canonical actions through
Kernel's computer API or a raw-CDP browser executor.
3. **CLI** — `@onkernel/cua-cli` assembles a `CuaAgentHarness` from
3. **CLI** — `@onkernel/cua-cli` assembles a pi `AgentHarness` from
command-line flags, env-var-based API keys, a `JsonlSessionRepo` for
transcripts, and pi skills; renders the result either as plain text
(`--print`), JSONL events (`-o jsonl`), or an interactive pi-tui
Expand Down
7 changes: 6 additions & 1 deletion docs/agent-tool-configuration-spec.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
# Agent Tool Configuration

**Status:** Implemented
**Status:** Superseded by `attach()` (2026-08-14). The tool-array-as-single-source-of-truth
rule it establishes still holds; what changed is where the array lives. `CuaAgent` and
`CuaAgentHarness` are gone, and a caller now compiles a (model, tools) pair through
`attach()` and owns the selection itself, so the `setTools()`/`setModel()` mutation surface
and its in-tool `executionMode: "sequential"` guard described below no longer exist.
Retained as the record of why the tool array is explicit and required.

**Scope:** `@onkernel/cua-agent` and the tool-building surface in `@onkernel/cua-ai`
**Compatibility:** Not a goal; these packages are alpha and may make breaking API changes.
Expand Down
34 changes: 21 additions & 13 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,15 +82,17 @@ before a model request.

## Dynamic catalogs

`CuaAgent` and `CuaAgentHarness` use composition around pi and expose:
`attach()` returns a handle; `compile()` turns a (model, tools) pair into plain
pi objects, and `apply()` swaps a running harness onto a new pair:

```ts
agent.getTools();
agent.setTools(nextTools);
agent.setModel(nextModel);
const kb = attach({ browser, client });
const compiled = kb.compile({ model, tools });
await compiled.apply(harness);
```

`setTools()` recompiles atomically before mutating pi state. Existing tool
Nothing mutates in place: a change compiles a new pair, and `compile()` throws
before anything reaches pi. Existing tool
identity with a changed schema, executor, or coordinates counts as a real
replacement. Additions made from inside a running tool are recorded in pi's
Anthropic-compatible `addedToolNames` marker only when that provider/model can
Expand Down Expand Up @@ -200,10 +202,14 @@ derives a transport that the rest of the selection must be compatible with.
Callers rebuild the menu after each staged change rather than caching a per-tool
verdict.

`CuaAgent` and `CuaAgentHarness` push the compiled `catalog.model` into pi on
every construction and on every `setTools()`/`setModel()`, so the derived
transport applies uniformly regardless of which mutation path selected the
tools.
`apply()` pushes the compiled `catalog.model` into pi alongside its tools, and
only when the derived transport actually moved, so a tools-only change records
no model change while a transport-moving change records exactly one.

pi fixes a harness's `models` at construction, but the headers, payload
transforms, and incoming tool plan it applies are per-catalog. The handle
therefore owns one `Models` collection that serves whichever pair was last
activated; `activate()` is what redirects it, and `apply()` calls it.

Generated payload processing has fixed order: model preparation, tool
serialization, provider fields, then the caller's `onPayload` hook.
Expand All @@ -221,7 +227,9 @@ serialization, provider fields, then the caller's `onPayload` hook.
- Anthropic's native browser tool when the model supports it;
- Google's native browser action set;
3. creates and retains its own application-level coding-tool list;
4. passes the complete list to `CuaAgentHarness`;
4. compiles the complete list through the handle and hands the result to a
stock pi `AgentHarness`, retaining the selection in `CuaCliCatalog` so
`/model` and `/tools` can recompile it;
5. builds a caller-owned prompt from loaded skills and context files;
6. uses one `Session` for transcript persistence and resume;
7. exposes `cua act '<json>'` as a model-free path to the same `browser_act`
Expand Down Expand Up @@ -257,14 +265,14 @@ switch — run through that one queue, because each suspends across several
`setTools()`/`setModel()` calls. Without it an apply could land between a
switch's `setModel()` and its final `setTools()` and fail its compile against
the wrong provider. Selectors also refuse to open mid-turn: the agent's
execution-scope guard only covers mutation from inside a tool's `execute`, so
this TUI-side check is what protects a streaming request.
compiled pair is immutable, so this TUI-side check is what keeps a swap from
landing mid-request.

## Per-turn flow

```text
user prompt
-> CuaAgentHarness / pi agent loop
-> pi agent loop
-> active identity-keyed catalog
-> generated headers and payload transforms
-> caller onPayload
Expand Down
6 changes: 4 additions & 2 deletions docs/cua-cli-harness-migration.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# CUA CLI Harness Migration

**Status:** Completed and superseded.
**Status:** Historical (2026-08-14). The CLI now composes a stock pi `AgentHarness` from
`attach()` rather than `CuaAgentHarness`, which no longer exists. Retained as the record of
the print/action/interactive consolidation this describes.

The CLI now uses the shared `CuaAgentHarness` composition path for print,
The CLI uses one shared composition path for print,
action, and TUI flows. Its current architecture—including explicit tool-list
selection, coding-tool composition, sessions, skills, and rendering—is
documented in [`architecture.md`](architecture.md#cli-composition).
Expand Down
82 changes: 43 additions & 39 deletions packages/agent/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,52 +2,56 @@

## Unreleased

Breaking: `CuaAgent` and `CuaAgentHarness` are removed. cua-agent hands back
plain pi objects; the caller constructs the agent.

- Add `attach({ browser, client })`, returning a handle that compiles
(model, tools) pairs into plain pi objects: the model carrying the transport
its tools derive, executables materialized against the handle's browser pool,
a `Models` collection adding provider retry, required headers, the catalog's
payload transforms and the tool-result image bound, and an `install(harness)`
for the behaviors that are pi event handlers rather than constructor options.
The handle owns what actually persists — the Kernel client and browser, the
translator, the raw-CDP executor, ref and frame state — so a spec materializes
once across repeat compiles.
- `CuaAgent` and `CuaAgentHarness` are unchanged and now share their internals
with `attach()` rather than owning private copies. They are slated to retire
in favor of the handle.
- Add `CuaAgentHarness.setModelAndTools()`. A model switch that also swaps
interaction tools has to compile as one pair now that the selected tools
decide the transport: staging the two in sequence produces an intermediate
catalog whose derived transport differs from both the old and the new one, and
records a model change for a transport nothing ever streamed with.
- `CuaAgentHarness` no longer refuses a model ref that is absent from its
supplied `Models` collection: it falls back to the registry, and an id the
registry lacks is synthesized.
- The model streamed for a Google model now depends on which tools `CuaAgent` /
`CuaAgentHarness` were constructed or mutated with: selecting Google's native
browser toolset still compiles to the CUA-owned Interactions API, but a Google
model selected with only CDP browser tools now streams through pi's builtin
Google transport instead of always carrying the CUA-owned api. This applies
uniformly across construction, `setTools()`, and `setModel()`, since all three
feed the same compiled `catalog.model` into pi.
- Fix `setTools()` recompiling from the previously *compiled* model instead of
the caller's model selection: dropping a native toolset that had derived a
tool-selection-dependent api (e.g. Google's Interactions API) left subsequent
tools-only recompiles stuck on that api even though the new selection no
longer required it. `CuaToolManager` now recompiles tools-only changes from
the model input the caller last selected.
- `responseThreading` (`CuaAgentOptions`/`CuaAgentHarnessOptions`) no longer
affects OpenAI models: OpenAI now streams through pi-ai's builtin Responses
transport and its automatic prompt caching regardless of this flag. The option
still governs Google's `previous_response_id`-style continuation.
(model, tools) pairs into plain pi objects: `model` carrying the transport its
tools derive, `tools` / `agentTools` materialized against the handle's browser
pool, `models` adding provider retry, required headers, the catalog's payload
transforms and the tool-result image bound, `activate(harness)` for the
behaviors that are pi event handlers rather than constructor options, and
`apply(harness)` to swap a running harness onto a new pair. The handle owns
what actually persists — the Kernel client and browser, the translator, the
raw-CDP executor, ref and frame state — so a spec materializes once across
repeat compiles.
- `getTools()`, `setTools()`, `setModel()`, and `setModelAndTools()` are gone
with the classes. A change compiles a new pair and applies it, so the current
selection belongs to the caller and there is no second copy of it to drift.
`compile()` throws before anything reaches pi, and `apply()` restores the
previous pair if pi rejects the new one, so the atomicity those methods
provided is preserved. `apply()` sets the model only when the derived
transport actually moved, so a tools-only change records no model change.
- `CuaToolManager` is now immutable: one compiled pair per instance, with
`prepareTools`/`prepareModel`/`prepareModelAndTools`/`commit`/`getTools` and
the async-local execution scope removed. Removing the execution scope also
removes cache-preserving deferred tool addition: a tool that added tools mid
execution used to have those names recorded on its result as
`addedToolNames`, letting pi extend an OpenAI request without invalidating the
prompt-cache prefix. Nothing produced them outside that mutation path. The
transport still consumes `addedToolNames` on a transcript that carries them.
- The tool-result image bound, payload transforms, and required headers now
follow whichever pair is active rather than the one a harness was built with.
pi fixes `models` at construction while those are per-catalog, so one
collection per handle is what makes a swap possible at all.
- A model ref absent from a supplied `Models` collection falls back to the
registry, and an id the registry lacks is synthesized.
- The model streamed for a Google model depends on which tools it was compiled
with: selecting Google's native browser toolset compiles to the CUA-owned
Interactions API, while a Google model selected with only CDP browser tools
streams through pi's builtin Google transport.
- `responseThreading` (`CuaAttachOptions`) no longer affects OpenAI models:
OpenAI streams through pi-ai's builtin Responses transport and its automatic
prompt caching regardless of this flag. The option still governs Google's
`previous_response_id`-style continuation.
- Exempt OpenAI's native computer tool from the tool-result image replay limit.
Its `computer_call_output` items must each carry a screenshot, and stateless
replay no longer leaves them in provider-stored state.

Breaking: Tzafon and Yutori support is removed.

- Constructing a `CuaAgent` or `CuaAgentHarness` with a Tzafon or Yutori model
ref now fails to resolve the model, and `cua.providers.tzafon` /
`cua.providers.yutori` no longer exist.
- Compiling a Tzafon or Yutori model ref now fails to resolve the model, and
`cua.providers.tzafon` / `cua.providers.yutori` no longer exist.
- Remove `CuaExecutionResources.viewport`. It only fed the removed catalog
viewport option; the same value is still on `resources.browser.viewport`.

Expand Down
Loading
Loading