Skip to content

Make the tool menu the selection surface - #78

Merged
rgarcia merged 3 commits into
mainfrom
hypeship/tool-menu-selection
Aug 14, 2026
Merged

Make the tool menu the selection surface#78
rgarcia merged 3 commits into
mainfrom
hypeship/tool-menu-selection

Conversation

@rgarcia

@rgarcia rgarcia commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Phase 3. /tools built its rows from the list the CLI had already composed, so it could only subtract:

export function describeTools(tools: readonly CuaCliTool[]): ToolSelectionItem[] {
  return tools.map(...)   // exactly what the app passed the harness
}

Everything else CUA can offer was invisible — playwright_execute, the computer toolset, a provider-native surface the default didn't pick, browser_act on a model whose default excluded it. The tool menu is the differentiator and the CLI hid most of it.

The menu decides availability by compiling

cuaToolMenu(model, selected) returns every offerable tool, each marked available or not. It does not restate the compiler's rules — it compiles the candidate catalog and reports the result:

try { compileCuaToolCatalog({ model, requestedTools: candidate }); available = true }
catch (error) { available = false; reason = error.message }

So the menu cannot drift from what harness.setTools() accepts: a row is available exactly when selecting it compiles, and the reason shown is the compiler's own message. compileCuaToolCatalog is documented as pure and declaration-only — it builds no executable tools and retains none of its inputs — so probing per entry is cheap and side-effect-free.

A test asserts that equivalence directly, across a model with a native browser surface, one with a native computer surface, one with neither, one carrying a quirk, and one synthesized from an id pi-ai's registry doesn't carry.

Availability is pairwise, so rows are re-evaluated as you stage

Two providers' native surfaces can't coexist, and a native surface derives a transport the rest of the selection must suit. A per-tool boolean computed once can't express that. The picker rebuilds its rows after every toggle, unavailable rows can't be staged, and bulk actions skip them.

open: 43 rows, 38 available, 5 selected
added playwright: 25 tools (baseline 24)
dropped google native: 5 tools, native row available=true
browser_act still blocked by the quirk: true

cua tools

Same menu, no TUI, with --json:

model: google:gemini-3.6-flash

TOOL                   GROUP     STATE        NOTE
browser_wait_for       browser   unavailable  provider google does not accept the schema used by "browser_wait_for"
browser_act            browser   unavailable  provider google does not accept the schema used by "browser_act"
computer               native    unavailable  provider.openai.native.computer.v1 requires a openai model; selected google:gemini-3.6-flash
google native browser  native    default      Google's predefined browser action set (20 actions), selected as one unit.

Testing

  • npm run typecheck clean; 94 / 296 / 142 tests pass.
  • The /tools tests were rewritten rather than deleted, and gained one that pins the new capability: enabling playwright_execute on a model whose baseline never included it.
  • Picker behavior preserved: baseline order, ctrl+r restoring model defaults, staged-until-confirm, the mutation queue against a /model switch, and refusal to open mid-turn.

One caveat worth flagging: a vitest file-level collect failure reports as "Test Files 1 failed" without any × line, so a grep watching only for failed assertions can miss it entirely. I hit that mid-change and only caught it because the total test count dropped.


Note

Medium Risk
Expands which tools a session can enable and changes transport/catalog combinations users can stage, though harness.setTools() still compiles before applying; mistakes affect agent behavior rather than security boundaries.

Overview
cuaToolMenu(model, selected) in @onkernel/cua-ai lists every offerable CUA tool for a model and marks each row available only when adding it to the current selection would make compileCuaToolCatalog succeed, surfacing the compiler’s error when it would not. Availability is relative to the staged selection (native surfaces, transport pinning), so callers rebuild the menu after each change.

The CLI /tools picker no longer mirrors only the composed baseline: it shows the full menu plus application tools, lets users add tools the app never composed (e.g. playwright_execute, computer tools), shows unavailable rows with reasons, and restages rows after every toggle/bulk action. Applies use menu order via toolsForSelection; ctrl+r still restores model defaults. cua tools prints the same menu to stdout (--json optional).

describeMenu preserves live spec instances for already-installed tools (e.g. Anthropic native browser options) so a no-op apply does not drop configuration.

Reviewed by Cursor Bugbot for commit d0371a1. Bugbot is set up for automated code reviews on this repo. Configure here.

`/tools` built its rows from the list the CLI had already composed, so it could
only subtract. Everything else CUA can offer — `playwright_execute`, the
computer toolset, a provider-native surface the default did not pick,
`browser_act` on a model whose default excluded it — was invisible. The menu is
the product's differentiator and the CLI hid most of it.

`cuaToolMenu(model, selected)` returns every offerable tool, each marked
available or not. It decides availability by compiling the candidate catalog
rather than restating the compiler's rules, so the menu cannot drift from what
`compileCuaToolCatalog` accepts: a row is available exactly when selecting it
compiles, and the reason shown when it is not is the compiler's own message.
A test asserts that equivalence across models with a native browser surface, a
native computer surface, neither, a quirk, and an id pi-ai's registry does not
carry.

Availability is relative to the current selection, because several rules are
pairwise: two providers' native surfaces cannot coexist, and a native surface
derives a transport the rest of the selection must suit. The picker therefore
rebuilds its rows after every toggle rather than fixing them when it opens,
unavailable rows cannot be staged, and bulk actions skip them.

`cua tools` prints the same menu for a model without the TUI, with `--json`.
Comment thread packages/ai/src/menu.ts
Comment thread packages/cli/src/tui/tools-picker.ts
The ptywright scenario pinned `22/22 enabled`, from when the picker's rows were
exactly the application-composed baseline. The footer now counts selectable
rows, of which the baseline is the part enabled on open, so the scenario derives
both numbers the same way the picker does and asserts ctrl+a grows the selection
past the baseline rather than merely restoring it.
@rgarcia

rgarcia commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Ctrl+A wait matches prior footer
    • I changed the Ctrl+A synchronization wait to ${selectable}/${selectable} enabled, which uniquely matches the post-select-all footer and eliminates the stale pre-keypress match race.

Create PR

Or push these changes by commenting:

@cursor push ad56e4b65d
Preview (ad56e4b65d)
diff --git a/packages/cli/test/tui.fixture.test.ts b/packages/cli/test/tui.fixture.test.ts
--- a/packages/cli/test/tui.fixture.test.ts
+++ b/packages/cli/test/tui.fixture.test.ts
@@ -389,7 +389,7 @@
 		// ctrl+a enables every selectable row — including tools the application
 		// never composed — and ctrl+x clears it; both are staged.
 		session.press(KeyCtrlA);
-		await session.waitForVisible(`/${selectable} enabled`, { timeoutMs: WAIT_MS });
+		await session.waitForVisible(`${selectable}/${selectable} enabled`, { timeoutMs: WAIT_MS });
 		await session.waitForStable(STABLE_MS, { timeoutMs: WAIT_MS });
 		const enabledAll = /(\d+)\/\d+ enabled/.exec(session.snapshot().visible);
 		assert.ok(enabledAll && Number(enabledAll[1]) > baseline, "ctrl+a grows the selection past the baseline");

You can send follow-ups to the cloud agent here.

Comment thread packages/cli/test/tui.fixture.test.ts
Three things, all real:

A row that is already installed now contributes the exact spec object in use
rather than a freshly built one. The CLI enables `javascript` on Anthropic's
native browser, and rebuilding the catalog from the menu's own specs would have
dropped `enable_javascript_exec` on a no-op apply or a reset-then-apply.

`ctrl+r` now restages like every other mutation. Toggle and the bulk actions
recompute availability against the staged set; reset did not, so rows could keep
availability from the previous stage while the footer already showed defaults.

The fixture's `ctrl+a` wait could resolve on the pre-keypress screen, since
`waitForVisible` is a substring check and the footer already contained
`/<selectable> enabled`. It now settles the screen and asserts the count grew.
@rgarcia

rgarcia commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit d0371a1. Configure here.

@rgarcia
rgarcia merged commit 0d225f4 into main Aug 14, 2026
10 of 11 checks passed
@rgarcia
rgarcia deleted the hypeship/tool-menu-selection branch August 14, 2026 02:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant