Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
1cdc116
feat(ai-harness): first-party plugins in @tanstack/ai-harness/plugins
AlemTuzlak Oct 13, 2026
84c6153
docs(ai-harness): say that a tool with no rule runs in plan mode
AlemTuzlak Oct 13, 2026
d323e93
Merge remote-tracking branch 'origin/feat/harness-core' into feat/har…
AlemTuzlak Oct 13, 2026
3188a70
feat(ai-code-mode): codeMode() harness plugin in @tanstack/ai-code-mo…
AlemTuzlak Oct 13, 2026
cee77b5
Merge remote-tracking branch 'origin/feat/harness-core' into feat/har…
AlemTuzlak Oct 13, 2026
da0bd96
Merge branch 'feat/harness-plugins' into feat/harness-code-mode
AlemTuzlak Oct 13, 2026
eb69ad0
Merge remote-tracking branch 'origin/feat/harness-core' into feat/har…
AlemTuzlak Oct 14, 2026
9bdd860
fix(ai-harness): adopt object params from #1779
AlemTuzlak Oct 14, 2026
b113c45
docs(ai-harness): answer permission questions with onEvent
AlemTuzlak Oct 14, 2026
aea1f8a
ci: apply automated fixes
autofix-ci[bot] Oct 14, 2026
1c60d05
Merge remote-tracking branch 'origin/feat/harness-plugins' into feat/…
AlemTuzlak Oct 14, 2026
5225695
docs(ai-harness): answer each open permission question once
AlemTuzlak Oct 14, 2026
cecb4c3
Merge remote-tracking branch 'origin/feat/harness-plugins' into feat/…
AlemTuzlak Oct 14, 2026
d890fa3
fix(ai-code-mode): tell the model what execute_typescript returns (#1…
jherr Oct 15, 2026
e4d3727
Merge remote-tracking branch 'origin/feat/harness-core' into feat/har…
AlemTuzlak Oct 15, 2026
572f3e3
test(ai-code-mode): cover the code mode allow rules and the prompt reset
AlemTuzlak Oct 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/code-mode-harness.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tanstack/ai-code-mode': minor
---

Add `@tanstack/ai-code-mode/harness` with `codeMode({ driver })`: a harness plugin that gives the model one `execute_typescript` tool. The model writes a program that calls several tools, and the program runs in the isolate of any `@tanstack/ai-isolate-*` driver. By default, only tools that are safe to run without a question move into code mode: server tools with no `needsApproval` that the permission rules allow in plan mode, and never a tool that declares permission resources. `include` picks the tools yourself, a safe tool with `metadata.codeMode: true` always moves, and `lazy: true` lists only the tool names and adds `discover_tools`. Tool names that are not JavaScript identifiers get a safe name inside the program. `@tanstack/ai-harness` is an optional peer dependency.
5 changes: 5 additions & 0 deletions .changeset/code-mode-result-shape.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tanstack/ai-code-mode': patch
---

The `execute_typescript` tool description and the code-mode system prompt now tell the model that the tool returns `{ success, result, logs, error }`, where `result` is the value its code returned.
1 change: 1 addition & 0 deletions docs/code-mode/code-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -418,3 +418,4 @@ pnpm eval -- --no-judge # skip Anthropic-based judging
- [Showing Code Mode in the UI](./client-integration) — Display execution progress in your React app
- [Code Mode with Snippets](./code-mode-with-snippets) — Add persistent, reusable snippet libraries
- [Isolate Drivers](./code-mode-isolates): compare Node, QuickJS, QuickJS Bun, Cloudflare, Daytona, and E2B.
- [Code mode in a harness](../harness/code-mode): add code mode to a harness session as a plugin.
5 changes: 5 additions & 0 deletions docs/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -1036,6 +1036,11 @@
"addedAt": "2026-10-13",
"updatedAt": "2026-10-14"
},
{
"label": "Code mode in a harness",
"to": "harness/code-mode",
"addedAt": "2026-10-13"
},
{
"label": "Run agents from a harness",
"to": "harness/subagents",
Expand Down
137 changes: 137 additions & 0 deletions docs/harness/code-mode.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
---
title: Code mode in a harness
id: harness-code-mode
order: 13
description: "Let the harness model write one TypeScript program that calls many tools, and run it in an isolate. Any TanStack AI isolate driver plugs in."
keywords:
- tanstack ai
- harness
- code mode
- isolate
- quickjs
---

Your agent has many read-only tools, and it calls them one at a time. Each call is a model round trip, and every tool schema goes into every request. With code mode, the model writes one TypeScript program that calls the tools it needs, and the program runs in an isolate. The read-only tools leave the tool list and become functions in that program.

## 1. Add the plugin

<!-- ::start:tabs variant="package-manager" mode="install" -->

react: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs
vue: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs
solid: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs
svelte: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs
preact: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs
angular: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs
octane: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs
vanilla: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs

<!-- ::end:tabs -->

```ts group=harness-code-mode
import { toolDefinition } from '@tanstack/ai'
import { codeMode } from '@tanstack/ai-code-mode/harness'
import { defineHarness } from '@tanstack/ai-harness'
import { createQuickJSIsolateDriver } from '@tanstack/ai-isolate-quickjs'
import { openaiText } from '@tanstack/ai-openai'
import { z } from 'zod'

const issues = [{ id: 'ENG-1', title: 'Fix login', state: 'open' }]

const listIssues = toolDefinition({
name: 'list_issues',
description: 'List the issues of the team',
inputSchema: z.object({ state: z.enum(['open', 'closed']) }),
}).server(async ({ state }) => issues.filter((issue) => issue.state === state))

const getIssue = toolDefinition({
name: 'get_issue',
description: 'Get one issue by id',
inputSchema: z.object({ id: z.string() }),
}).server(async ({ id }) => issues.find((issue) => issue.id === id) ?? null)

export const assistant = defineHarness({
name: 'acme/assistant',
adapter: openaiText('gpt-6.1-sol'),
tools: [listIssues, getIssue],
plugins: () => [codeMode({ driver: createQuickJSIsolateDriver() })],
})
```

The model now has an `execute_typescript` tool. Inside the program, each moved tool is an `external_*` function, for example `external_list_issues({ state: 'open' })`.

## 2. Pick the isolate

`driver` takes any isolate driver. Change one line to run the programs somewhere else:

| Driver | Package | Runs in |
| --- | --- | --- |
| `createQuickJSIsolateDriver()` | `@tanstack/ai-isolate-quickjs` | WebAssembly, on any runtime |
| `createQuickJSBunIsolateDriver()` | `@tanstack/ai-isolate-quickjs-bun` | Native QuickJS in Bun |
| `createNodeIsolateDriver()` | `@tanstack/ai-isolate-node` | A V8 isolate in Node |
| `createCloudflareIsolateDriver()` | `@tanstack/ai-isolate-cloudflare` | A Cloudflare Worker |
| `createDaytonaIsolateDriver()` | `@tanstack/ai-isolate-daytona` | A Daytona sandbox |
| `createE2BIsolateDriver()` | `@tanstack/ai-isolate-e2b` | An E2B sandbox |

[Code mode isolates](../code-mode/code-mode-isolates) has the options of each driver.

## 3. Choose which tools move

A call inside the program does not stop for approval. So by default, a tool moves into code mode only when it is safe to run without a question:

- The tool runs on the server and does not set `needsApproval`.
- The permission rules allow it in plan mode. A tool with an `edit` or `execute` rule stays.
- The tool does not declare `PermissionResources`. Calls inside the program skip the permission checks, so such a tool stays a normal tool call.

Where the permission rules come from:

- With [`permissions()`](./permissions) in the plugin list, code mode asks it about each tool. The answer is the one a call gets in plan mode: your `rules` win, and a tool with no rule gets your `default`. So with `default: 'ask'`, a tool with no rule stays.
- Without `permissions()`, code mode reads the rules that tool plugins add.

Every other tool stays a normal tool call, with its approvals. With this default pick, code mode also adds `allow` rules for `execute_typescript` and `discover_tools`, so a program does not ask when every tool in it is safe. Your own `rules` still win.

To pick the tools yourself, pass `include`:

```ts group=harness-code-mode
export const onlyIssues = codeMode({
driver: createQuickJSIsolateDriver(),
include: (tool) => tool.name.endsWith('_issue') || tool.name.endsWith('_issues'),
})
```

- A tool that `include` picks moves even when it needs approval, and its calls inside the program do not ask. Pick only tools that are safe to run without a question.
- With `include`, a call of `execute_typescript` gets the permission rules like any tool, for example a question with `default: 'ask'`.
- A safe tool with `metadata.codeMode: true` moves even when `include` does not pick it.

The other options of `createCodeMode` pass through: `timeout`, `memoryLimit`, `lazyToolsConfig`, and more.

## 4. Send only the tool names

The prompt of code mode lists each moved tool with its full TypeScript type. With 100 tools, that is a lot of tokens on every turn, and most turns use a few tools. Set `lazy: true`, and the prompt lists only the names:

```ts group=harness-code-mode
export const lazyCodeMode = codeMode({
driver: createQuickJSIsolateDriver(),
lazy: true,
lazyToolsConfig: { includeDescription: 'first-sentence' },
})
```

The model then works in three steps:

1. It reads the tool names in the prompt.
2. It calls `discover_tools` with the names it needs, and gets their TypeScript signatures.
3. It calls those tools in `execute_typescript`.

`includeDescription` sets what the prompt shows with each name: `'none'` (the default, the name only), `'first-sentence'`, or `'full'`. Tools that stay outside code mode keep their full schema.

## Tools that appear while the session runs

The plugin chooses the tools again before every turn. A plugin that finds tools at run time with `discoverTools` (for example after a sign-in) adds them first, and the safe ones move into code mode on the next turn. Tool names that are not JavaScript identifiers, for example `notion-search`, get a safe name inside the program: `external_notion_search`. When two tools get the same safe name, only the first one moves. The other one stays a normal tool call.

## What you have now

- One `execute_typescript` tool in place of many read-only tools.
- With `lazy: true`, a prompt that lists only the tool names.
- Programs that run in the isolate you choose.
- Approvals kept for every tool that stays a normal tool call.
1 change: 1 addition & 0 deletions docs/harness/permissions.md
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,7 @@ export const planTools = definePlugin({
- Pass what a call touches as a third argument, for example `decide('read_file', 'default', { paths: [file] })`. Then the path rules and `root` apply too.
- Read it when the plugin runs, not in `setup`. Then `permissions()` can come before or after your plugin.
- Saved `always` answers are not part of it. They can only allow, so a check that needs `'allow'` stays strict.
- [Code mode](./code-mode) uses this answer to pick the tools that move into a program. With `default: 'ask'`, a tool with no rule stays a normal tool call.
- `PermissionRules`, `PermissionResources`, `PermissionDecisionCapability`, and `decidePermission` come from `@tanstack/ai-harness`. They load no Node module, so a plugin for an edge runtime can use them. The plugins themselves come from `@tanstack/ai-harness/plugins`, which is Node only.

## Known limits
Expand Down
12 changes: 12 additions & 0 deletions packages/ai-code-mode/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@
".": {
"types": "./dist/esm/index.d.ts",
"import": "./dist/esm/index.js"
},
"./harness": {
"types": "./dist/esm/harness.d.ts",
"import": "./dist/esm/harness.js"
}
},
"sideEffects": false,
Expand Down Expand Up @@ -65,10 +69,18 @@
},
"peerDependencies": {
"@tanstack/ai": "workspace:^",
"@tanstack/ai-harness": "workspace:^",
"zod": "^3.0.0 || ^4.0.0"
},
"peerDependenciesMeta": {
"@tanstack/ai-harness": {
"optional": true
}
},
"devDependencies": {
"@tanstack/ai": "workspace:*",
"@tanstack/ai-harness": "workspace:*",
"@tanstack/ai-persistence": "workspace:*",
"@vitest/coverage-v8": "4.1.10",
"zod": "^4.2.0"
}
Expand Down
3 changes: 2 additions & 1 deletion packages/ai-code-mode/src/create-code-mode-tool.ts
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,7 @@ function buildToolDescription(tools: Array<CodeModeTool>): string {
`Execute TypeScript code in a secure sandbox environment. ` +
`The code can use these external API functions: ${externalFunctions}.${discoverable} ` +
`All external_* calls are async and must be awaited. ` +
`Return a value to pass results back. Use console.log() for debugging.`
`Return a value to pass results back. Use console.log() for debugging. ` +
`The tool returns { success, result, logs, error }, where result is the value your code returned.`
)
}
2 changes: 1 addition & 1 deletion packages/ai-code-mode/src/create-system-prompt.ts
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ return { warmestCity: warmest.location, temperature: warmest.temperature };
### Important Notes

- All \`external_*\` calls are async - always use \`await\`
- Return a value to pass results back to you
- Return a value to pass results back to you. The tool returns \`{ success, result, logs, error }\`, where \`result\` is the value your code returned
- Use \`console.log()\` for debugging (logs are captured)
- The sandbox is isolated - no network access or file system
- Each execution is independent (no shared state between calls)
Expand Down
152 changes: 152 additions & 0 deletions packages/ai-code-mode/src/harness.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
import {
PermissionDecisionCapability,
PermissionResources,
PermissionRules,
decidePermission,
definePlugin,
} from '@tanstack/ai-harness'
import { createCodeMode } from './create-code-mode'
import type { AnyTool } from '@tanstack/ai'
import type { CodeModeTool, CodeModeToolConfig } from './types'

export interface CodeModePluginOptions extends Omit<
CodeModeToolConfig,
'tools'
> {
/**
* Which tools move into code mode. Default: every server tool that does
* not need approval and that `permissions()` allows in plan mode (so no
* edits, no commands, nothing that asks first). Without `permissions()`,
* the rules of tool plugins decide. A safe tool with
* `metadata.codeMode: true`, for example a tool of an `mcp()` server with
* `codeMode: true`, moves even when `include` does not pick it.
*/
include?: (tool: CodeModeTool) => boolean
/**
* Make each tool that moves into code mode lazy. The prompt then lists only
* the tool names, and `lazyToolsConfig.includeDescription` adds part of
* each description. The model calls `discover_tools` for the signatures of
* the tools it needs, then calls them in `execute_typescript`. This saves
* tokens on each turn when there are many tools. Default `false`.
*/
lazy?: boolean
}

/**
* A name that works as a JavaScript identifier. Code mode turns each tool
* into an `external_<name>` function, and MCP tool names often have `-`.
*/
function identifierOf(name: string): string {
const safe = name.replace(/[^A-Za-z0-9_$]/g, '_')
return /^[0-9]/.test(safe) ? `_${safe}` : safe
}

/** A tool that runs on the server: not a client tool, and it has `execute`. */
function isServerTool(tool: AnyTool): tool is CodeModeTool {
const side = '__toolSide' in tool ? tool.__toolSide : 'server'
return side === 'server' && typeof tool.execute === 'function'
}

/**
* A harness plugin that gives the model an `execute_typescript` tool. The
* model writes one TypeScript program that calls several tools, and the
* program runs in the isolate of `driver` (any `@tanstack/ai-isolate-*`
* driver). The tools it calls leave the model's tool list, including tools
* that plugins find at run time, such as MCP tools after `/connect`.
*
* Calls inside the isolate do not stop for approval, so by default only
* tools that are safe to run without a question move into code mode. The
* other tools stay normal tool calls. A tool with `metadata.codeMode: true`
* moves even when `include` does not pick it, but only when it is safe in
* the same way. Calls inside the isolate also skip the permission checks,
* so a tool that declares `PermissionResources` never moves. With
* `lazy: true`, the model gets only the names of the moved tools and asks
* `discover_tools` for the signatures.
*
* @example
* ```ts
* import { codeMode } from '@tanstack/ai-code-mode/harness'
* import { createQuickJSIsolateDriver } from '@tanstack/ai-isolate-quickjs'
*
* defineHarness({
* name: 'acme/agent',
* adapter,
* plugins: () => [codeMode({ driver: createQuickJSIsolateDriver() })],
* })
* ```
*/
export function codeMode(options: CodeModePluginOptions) {
const { include, lazy = false, ...config } = options
return definePlugin({
name: 'tanstack/code-mode',
optionalRequires: [PermissionDecisionCapability],
setup: (ctx) => {
const rules = ctx.collect(PermissionRules)
const resources = ctx.collect(PermissionResources)
// The system prompt for the tools of the current turn.
let prompt = ''
// Without `include` or snippets, only tools that plan mode allows
// move, so a program is as safe as its tools. Allow the code mode
// tools, so `default: 'ask'` does not ask for (or plan mode deny)
// every program. The user's rules come later and still win.
const isDefaultPick = !include && !config.getSnippetBindings
const allowCodeMode = isDefaultPick
? ['execute_typescript', 'discover_tools'].map((tool) =>
PermissionRules.item({ tool, decision: 'allow', kind: 'read' }),
)
: []
return {
prompts: [{ id: 'tanstack/code-mode', text: () => prompt }],
contribute: allowCodeMode,
prepareTools: ({ tools }) => {
// A failed run below must not leave the prompt of a past turn.
prompt = ''
// Ask permissions() when it is mounted: it knows all rules in its
// order and its default. Read here, so it can come after code mode.
const decide = ctx.getOptional(PermissionDecisionCapability)
const safe = (tool: CodeModeTool) =>
!tool.needsApproval &&
(decide?.(tool.name, 'plan') ??
decidePermission(rules, tool.name, 'plan')) === 'allow'
const moves = (tool: CodeModeTool) => {
// The permission checks run on tool calls of the model. A call
// inside the isolate skips them, so such a tool stays a tool call.
const hasResources = resources.some((map) =>
Object.hasOwn(map, tool.name),
)
if (hasResources) return false
// A marked tool still needs the checks of the default pick.
const isMarked = tool.metadata?.codeMode === true
if (isMarked && safe(tool)) return true
return include ? include(tool) : safe(tool)
}
// Two tools that map to the same identifier: the first one moves.
const byIdentifier = new Map<string, CodeModeTool>()
for (const tool of tools.filter(isServerTool)) {
if (!moves(tool)) continue
const identifier = identifierOf(tool.name)
if (!byIdentifier.has(identifier))
byIdentifier.set(identifier, tool)
}
if (byIdentifier.size === 0) return tools
const created = createCodeMode({
...config,
tools: [...byIdentifier].map(([identifier, tool]) =>
identifier === tool.name && !lazy
? tool
: { ...tool, name: identifier, ...(lazy ? { lazy } : {}) },
),
})
prompt = created.systemPrompt
const moved = new Set(
[...byIdentifier.values()].map((tool) => tool.name),
)
return [
...tools.filter((tool) => !moved.has(tool.name)),
...created.tools,
]
},
}
},
})
}
1 change: 1 addition & 0 deletions packages/ai-code-mode/tests/create-code-mode-tool.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ describe('createCodeModeTool', () => {
})
expect(tool.description).toContain('external_fetchWeather')
expect(tool.description).toContain('external_dbQuery')
expect(tool.description).toContain('{ success, result, logs, error }')
})

it('tool description says discover_tools is a separate tool', () => {
Expand Down
1 change: 1 addition & 0 deletions packages/ai-code-mode/tests/create-system-prompt.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ describe('createCodeModeSystemPrompt', () => {
expect(prompt).toContain('When to Use')
expect(prompt).toContain('Available External APIs')
expect(prompt).toContain('Important Notes')
expect(prompt).toContain('`{ success, result, logs, error }`')
})

it('documents multiple tools', () => {
Expand Down
Loading
Loading