Replace the model allowlist with pi-ai catalog passthrough - #77
Merged
Conversation
CUA_MODEL_ANNOTATIONS decided which models were allowed to run. It was the last gate between a caller and pi-ai's catalog, and it refused models that needed no work at all: grok-4.3 was already in pi's registry and still rejected, and grok-4.6 could not be reached the day xAI shipped it. Every model pi-ai carries is now selectable — 37 providers, ~1,150 models — and an id the registry has not caught up with is synthesized from its nearest sibling. Only an unqualified ref or a provider pi-ai does not carry is refused. Two tables replace the allowlist, and neither decides whether a model may run: - CUA_NATIVE_SURFACES records which models have a provider-native computer or browser tool, so the tool menu can offer it. `cua models` shows it in a NATIVE column, and CuaModelInfo carries it alongside a vision flag. - CUA_MODEL_QUIRKS records request-shape limits, each carrying the documented limit or observed failure that justifies it. Capabilities default to permissive: a limit we cannot evidence becomes a provider-side error rather than a refusal cua invents. Synthesis follows the sibling sharing the longest id prefix, preferring the latest. Providers migrate transports mid-generation — xAI carries grok-4.3 on chat completions and grok-4.5 on Responses — so a new id has to follow its nearest, newest relative rather than whichever model comes first. Two consequences the opened catalog forces: - A bare model id that several providers carry resolves to the first-party provider, since gateways resell the same ids. It is a disambiguation preference for bare ids only, never a gate. - The CLI's API-key preflight runs only where CUA documents the variable names. For any other provider, pi resolves the credential when it streams; failing up front would refuse a model that works.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.
Bugbot Autofix prepared fixes for all 3 issues found in the latest run.
- ✅ Fixed: Google defaults fail catalog compile
- I gated non-native fallback tools by
acceptsComplexSchemasand now use a primitive browser set withoutbrowser_wait_forfor Google models that reject complex schemas.
- I gated non-native fallback tools by
- ✅ Fixed: Example tool policy drifted
- I rewrote
toolsForModelto mirror the CLI’s native-surface/capability policy so new providers and non-native Google models get the same valid defaults.
- I rewrote
- ✅ Fixed: Synthesis picks named siblings
- I changed sibling selection to prefer family-related IDs and then newer numeric recency on ties, preventing named variants from outranking their family root snapshots.
Or push these changes by commenting:
@cursor push 8ca6299b01
Preview (8ca6299b01)
diff --git a/packages/agent/examples/shared/tools.ts b/packages/agent/examples/shared/tools.ts
--- a/packages/agent/examples/shared/tools.ts
+++ b/packages/agent/examples/shared/tools.ts
@@ -1,8 +1,8 @@
import {
cua,
cuaModelCapabilities,
+ cuaNativeSurfaces,
getCuaModel,
- parseCuaModelRef,
type CuaModelRef,
} from "@onkernel/cua-ai";
import type { CuaAgentTool } from "../../src/index";
@@ -11,37 +11,25 @@
return [...cua.toolsets.browser(), cua.tools.browser.act()];
}
+function browserPrimitiveTools(): CuaAgentTool[] {
+ return cua.toolsets.browser().filter((tool) => tool.name !== "browser_wait_for");
+}
+
/**
* Interaction policy shared by the agent and harness provider matrices, mirroring
* the CLI defaults in `packages/cli/src/harness.ts`. Both examples read it from
* here so the two cannot drift apart.
*/
export function toolsForModel(model: CuaModelRef): CuaAgentTool[] {
- const { provider, model: modelId } = parseCuaModelRef(model);
- switch (provider) {
- case "openai":
- // Favor refs, semantic reads, and verified plans over coordinate-only computer use.
- return structuredBrowserTools();
- case "anthropic":
- // Claude 5 can use Anthropic's native browser tool; older models use portable
- // CUA tools plus the explicit semantic action-plan surface.
- return cua.providers.anthropic.supports.browser(modelId)
- ? [cua.providers.anthropic.tools.browser({ version: "20260701", javascript: true })]
- : structuredBrowserTools();
- case "google":
- // Current Gemini computer-use models expect Google's predefined browser actions.
- return cua.providers.google.toolsets.browser();
- case "xai":
- // No first-party native browser surface exists, so use CUA browser primitives
- // plus verified dependent plans.
- return structuredBrowserTools();
- case "moonshotai":
- case "openrouter":
- // Same as xai, minus browser_act where the model rejects that tool's
- // oversized schema. OpenRouter fronts several model families, so ask
- // the model rather than the provider.
- return cuaModelCapabilities(getCuaModel(model)).acceptsLargeSchemas
- ? structuredBrowserTools()
- : cua.toolsets.browser();
+ const resolved = getCuaModel(model);
+ if (cuaNativeSurfaces(resolved).includes("browser")) {
+ if (resolved.provider === "anthropic") {
+ return [cua.providers.anthropic.tools.browser({ version: "20260701", javascript: true })];
+ }
+ if (resolved.provider === "google") return cua.providers.google.toolsets.browser();
}
+ const capabilities = cuaModelCapabilities(resolved);
+ if (capabilities.acceptsLargeSchemas) return structuredBrowserTools();
+ if (capabilities.acceptsComplexSchemas) return cua.toolsets.browser();
+ return browserPrimitiveTools();
}
diff --git a/packages/agent/test/example-provider-matrix.test.ts b/packages/agent/test/example-provider-matrix.test.ts
--- a/packages/agent/test/example-provider-matrix.test.ts
+++ b/packages/agent/test/example-provider-matrix.test.ts
@@ -18,8 +18,10 @@
"anthropic:claude-opus-5",
"anthropic:claude-sonnet-5",
"google:gemini-3.6-flash",
+ "google:gemini-3.7-flash",
"openrouter:meta/muse-spark-1.1",
"xai:grok-4.5",
+ "groq:llama-3.3-70b-versatile",
"moonshotai:kimi-k3",
"openrouter:moonshotai/kimi-k3",
];
@@ -49,4 +51,11 @@
expect(toolsForModel(model).map((tool) => tool.name), model).toContain("browser_act");
}
});
+
+ it("uses primitive CUA browser tools for non-native Google models", () => {
+ const names = toolsForModel("google:gemini-3.7-flash").map((tool) => tool.name);
+ expect(names).toContain("browser_snapshot");
+ expect(names).not.toContain("browser_wait_for");
+ expect(names).not.toContain("browser_act");
+ });
});
diff --git a/packages/ai/src/models.ts b/packages/ai/src/models.ts
--- a/packages/ai/src/models.ts
+++ b/packages/ai/src/models.ts
@@ -219,23 +219,62 @@
throw new Error(`provider "${provider}" carries no models to infer "${modelId}" from`);
}
let template = siblings[0]!;
- let bestPrefix = -1;
- siblings.forEach((sibling, index) => {
- const prefix = sharedPrefixLength(sibling.id.toLowerCase(), modelId.toLowerCase());
- if (prefix >= bestPrefix) {
- bestPrefix = prefix;
- template = siblings[index]!;
+ let bestScore = siblingSimilarityScore(template.id, modelId);
+ for (let index = 1; index < siblings.length; index += 1) {
+ const sibling = siblings[index]!;
+ const score = siblingSimilarityScore(sibling.id, modelId);
+ if (isBetterSibling(score, sibling.id, bestScore, template.id)) {
+ bestScore = score;
+ template = sibling;
}
- });
+ }
return { ...template, id: modelId, name: modelId };
}
+function siblingSimilarityScore(candidateId: string, targetId: string): { familyRelated: number; prefix: number } {
+ const candidate = candidateId.toLowerCase();
+ const target = targetId.toLowerCase();
+ return {
+ familyRelated: isCuaFamilyMatch(target, candidate) || isCuaFamilyMatch(candidate, target) ? 1 : 0,
+ prefix: sharedPrefixLength(candidate, target),
+ };
+}
+
+function isBetterSibling(
+ candidate: { familyRelated: number; prefix: number },
+ candidateId: string,
+ current: { familyRelated: number; prefix: number },
+ currentId: string,
+): boolean {
+ if (candidate.familyRelated !== current.familyRelated) return candidate.familyRelated > current.familyRelated;
+ if (candidate.prefix !== current.prefix) return candidate.prefix > current.prefix;
+ const recency = compareModelRecency(candidateId, currentId);
+ if (recency !== 0) return recency > 0;
+ return candidateId.localeCompare(currentId) > 0;
+}
+
function sharedPrefixLength(a: string, b: string): number {
let length = 0;
while (length < a.length && length < b.length && a[length] === b[length]) length += 1;
return length;
}
+function compareModelRecency(a: string, b: string): number {
+ const aParts = numericParts(a);
+ const bParts = numericParts(b);
+ const limit = Math.min(aParts.length, bParts.length);
+ for (let i = 0; i < limit; i += 1) {
+ const diff = (aParts[i] ?? 0) - (bParts[i] ?? 0);
+ if (diff !== 0) return diff;
+ }
+ if (aParts.length !== bParts.length) return aParts.length - bParts.length;
+ return a.localeCompare(b);
+}
+
+function numericParts(id: string): number[] {
+ return (id.match(/\d+/g) ?? []).map((part) => Number.parseInt(part, 10));
+}
+
/** Return the provider id for a concrete model. */
export function providerForModel(model: Model<Api>): CuaProvider {
return model.provider;
diff --git a/packages/ai/test/models.test.ts b/packages/ai/test/models.test.ts
--- a/packages/ai/test/models.test.ts
+++ b/packages/ai/test/models.test.ts
@@ -131,6 +131,8 @@
expect(snapshot.provider).toBe("openai");
expect(snapshot.api).toBe(getCuaModel("openai:gpt-5.5").api);
expect(snapshot.baseUrl).toBe(getCuaModel("openai:gpt-5.5").baseUrl);
+ expect(snapshot.contextWindow).toBe(getCuaModel("openai:gpt-5.5").contextWindow);
+ expect(snapshot.maxTokens).toBe(getCuaModel("openai:gpt-5.5").maxTokens);
// The motivating case: a model the provider has shipped and models.dev
// has not picked up yet.
diff --git a/packages/cli/src/harness.ts b/packages/cli/src/harness.ts
--- a/packages/cli/src/harness.ts
+++ b/packages/cli/src/harness.ts
@@ -94,6 +94,11 @@
return [...cua.toolsets.browser(), cua.tools.browser.act()];
}
+/** CDP browser tools that avoid complex schemas unsupported by some providers. */
+function browserPrimitiveTools(): CuaCliTool[] {
+ return cua.toolsets.browser().filter((tool) => tool.name !== "browser_wait_for");
+}
+
/**
* CLI interaction policy, asked of the model rather than switched on its
* provider: a model with a provider-native browser surface gets that surface,
@@ -112,9 +117,10 @@
}
if (provider === "google") return cua.providers.google.toolsets.browser();
}
- return cuaModelCapabilities(resolved).acceptsLargeSchemas
- ? structuredBrowserTools()
- : cua.toolsets.browser();
+ const capabilities = cuaModelCapabilities(resolved);
+ if (capabilities.acceptsLargeSchemas) return structuredBrowserTools();
+ if (capabilities.acceptsComplexSchemas) return cua.toolsets.browser();
+ return browserPrimitiveTools();
}
function composeSystemPrompt(skills: Skill[], contextFiles: ContextFile[]): string {
diff --git a/packages/cli/test/harness-assembly.test.ts b/packages/cli/test/harness-assembly.test.ts
--- a/packages/cli/test/harness-assembly.test.ts
+++ b/packages/cli/test/harness-assembly.test.ts
@@ -4,6 +4,7 @@
InMemorySessionRepo,
type Skill,
} from "@onkernel/cua-agent";
+import { compileCuaToolCatalog } from "@onkernel/cua-ai";
import { tmpdir } from "node:os";
import { mkdtempSync } from "node:fs";
import { join } from "node:path";
@@ -23,6 +24,13 @@
const googleNames = defaultInteractionTools("google:gemini-3.6-flash").map((tool) => tool.name);
expect(googleNames).toContain("take_screenshot");
expect(googleNames).not.toContain("browser_act");
+ const fallbackGoogleModel = "google:gemini-3.7-flash";
+ const fallbackGoogleNames = defaultInteractionTools(fallbackGoogleModel).map((tool) => tool.name);
+ expect(fallbackGoogleNames).not.toContain("browser_wait_for");
+ expect(fallbackGoogleNames).not.toContain("browser_act");
+ expect(
+ () => compileCuaToolCatalog({ model: fallbackGoogleModel, requestedTools: defaultInteractionTools(fallbackGoogleModel) }),
+ ).not.toThrow();
for (const model of ["xai:grok-4.5", "openrouter:meta/muse-spark-1.1"] as const) {
const tools = defaultInteractionTools(model);
expect(tools[0]).toMatchObject({ name: "browser_snapshot", origin: "cua" });You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit 94adeb8. Configure here.
The skill existed to maintain the model allowlist: enumerate every provider's models, smoke-test each one, decide whether it counted as CUA-supported, and write the verdict into a table. With the allowlist gone there is no verdict to reach and no table to sweep, so the periodic audit it prescribed has nothing to service. What remains is reactive and belongs next to the tables it describes, so `supported-models.md` gains a short section covering the four cases: a provider released a model (nothing to do), the catalog looks stale (bump pi-ai), a provider changed a native tool (probe it, update the adapter, adjust the surface entry), and a model rejects a tool we send (add a quirk with the error as its reason). `native-action-probe.ts` survives as `packages/ai/scripts/` — eliciting what a model actually emits is how you find out an adapter broke. The other three scripts were evidence-gathering for allowlist entries: enumerating provider metadata, cloning provider example repos, and diffing documented action names against local constants. Also documents that cua does not read pi's `models.json`, since that is a pi-coding-agent config file and cua builds its collection from pi-ai directly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 2.
CUA_MODEL_ANNOTATIONSdecided which models were allowed to run — the last gate between a caller and pi-ai's catalog. It refused models that needed no work at all:grok-4.3 was already in pi's registry. grok-4.6 shipped and couldn't be reached. Neither needed a line of transport work — only an entry in a hand-maintained table.
Now: 37 providers, ~1,153 models, all selectable. Only an unqualified ref or a provider pi-ai does not carry is refused.
Two tables replace the allowlist
Neither decides whether a model may run.
CUA_NATIVE_SURFACES— which models have a provider-native computer or browser tool, so the menu can offer it.cua modelsshows it:CUA_MODEL_QUIRKS— request-shape limits, each carrying the evidence that justifies it. Capabilities default to permissive, so a limit we can't evidence becomes a provider-side error rather than a refusal cua invents. The two kept here were both observed live in this work: Gemini rejectingbrowser_wait_for's schema, Kimi K3 rejectingbrowser_act's.Synthesis picks the nearest, newest sibling
A ref pi's registry lacks is built from the sibling sharing the longest id prefix, preferring the latest. This matters more than it looks: xAI carries
grok-4.3on chat completions andgrok-4.5on Responses, so taking "the provider's first model" sent a new Grok to the wrong transport. Caught while verifying, fixed, and pinned by a test.Two consequences the opened catalog forces
gpt-5.6-solis carried by six providers (azure, cloudflare gateway, copilot, openai, codex, opencode). A bare id now resolves to the first-party provider, since gateways resell the same ids. It's a disambiguation preference for bare ids only, never a gate — any provider is still reachable with a qualified ref.Gate removal
CuaProvideris a provider id string;CUA_PROVIDERS,isCuaProvider,findCuaAnnotation, andsupportsCuaProviderare gone.providerForModelno longer throws. Removed the refusals intool-catalog.ts(called purely for its throw) andCuaAgentHarness'sresolveModelFromCollection.Testing
npm run typecheckclean; 88 / 296 / 139 tests pass.The
update-modelsskill was rewritten around what's left — its whole job was maintaining the allowlist; now it detects native-surface drift and new quirks, and says explicitly that a newly released model needs no repo change.Note
Medium Risk
Broadens selectable models and changes default CLI toolsets and API-key preflight behavior; synthesis picks transport from sibling models, which is tested but affects any newly shipped model ids before registry updates.
Overview
Breaking (0.14.0 / agent 0.14.0 / cli 0.13.0): CUA no longer gates which models may run.
listCuaModels()andgetCuaModel()expose all pi-ai models (~37 providers); missing registry ids are synthesized from the longest-prefix, newest sibling so new provider ids work before models.dev updates.CUA_MODEL_ANNOTATIONS,CUA_PROVIDERS,isCuaProvider, andfindCuaAnnotationare removed.CUA_NATIVE_SURFACESandCUA_MODEL_QUIRKSreplace the allowlist for menus and tool policy only: native computer/browser offerings and documented request-shape limits (cuaNativeSurfaces,cuaModelQuirks, permissive-defaultcuaModelCapabilities). Catalog listings addnativeSurfacesandvision.Agent:
CuaAgentHarnessno longer throws on refs absent from a customModelscollection—it falls back togetCuaModel()/ synthesis.CLI:
cua modelslists the full catalog with a NATIVE column;-maccepts any pi-ai model; bare ids prefer first-party providers when ambiguous; API-key preflight runs only for documented env vars; default interaction tools followcuaNativeSurfaces+ quirks per model, not per-provider switches.Docs, tests, and the
update-modelsskill are updated for the two-table maintenance model.Reviewed by Cursor Bugbot for commit 94adeb8. Bugbot is set up for automated code reviews on this repo. Configure here.
The update-models skill is retired
Its whole job was maintaining the allowlist: enumerate every provider's models, smoke-test each, decide whether it counted as supported, write the verdict into the table. With no verdict to reach, the periodic audit it prescribed has nothing to service. −1,583 lines.
discover-models.tsaudit-official-examples.tsprovider-doc-drift.tsSKILL.md,report-schema.md,README.mdnative-action-probe.tssurvives aspackages/ai/scripts/native-action-probe.ts. Eliciting what a model actually emits is how you find out an adapter broke, and that stays true.What replaces it is a short section in
supported-models.md, next to the tables it describes, covering the four reactive cases: a provider released a model (nothing to do), the catalog looks stale (bump pi-ai), a provider changed a native tool (probe, update the adapter, adjust the surface entry), a model rejects a tool we send (add a quirk with the error as its reason).It also documents something worth knowing: cua does not read pi's
models.json. That's a pi-coding-agent config file loaded from the agent dir; cua builds itsModelscollection from pi-ai directly, andCreateModelsOptionsonly accepts credentials, a models store, and an auth context. So a provider pi-ai doesn't ship still needs registering inproviders.ts. Wiringmodels.jsonwould make custom and self-hosted providers a user-config action rather than a repo change — the last "you must edit cua to use your model" case left — but that's a separate change, not smuggled in here.