diff --git a/.changeset/ai-mcp-harness.md b/.changeset/ai-mcp-harness.md new file mode 100644 index 0000000000..e3f5f39956 --- /dev/null +++ b/.changeset/ai-mcp-harness.md @@ -0,0 +1,12 @@ +--- +'@tanstack/ai-mcp': minor +--- + +Use MCP in a TanStack AI harness. + +- `@tanstack/ai-mcp/harness` adds `createHarnessMcpServer`. It serves a harness as an MCP server, so Claude Code, Claude Desktop, Cursor, or another agent can use it. The tools are `chat`, `steer`, `cancel`, `approve`, `reject`, `resolve`, `answer`, and `status`, plus `agent_` for each exposed agent and `command_` for each exposed command. Every tool takes an optional `threadId`. With `approvals: 'ask'` (the default), the client asks the user about each approval when it supports elicitation. Otherwise the approvals come back in the result. `approvals: 'auto'` approves every tool call. `resolve` answers every kind of interrupt in one call. +- `@tanstack/ai-mcp/harness` also adds the `mcp()` harness plugin. It connects stdio and HTTP servers by name, gives the model their tools as `_`, and adds a `/mcp` status command. `oauth: true` signs in to a server with `/connect `. +- `@tanstack/ai-mcp/connector` adds `mcpConnector`: one OAuth MCP server as a harness plugin, with `needsApproval`, `prefix`, `scopes`, and `clientName`. The sign-in passes the `iss` of the callback on (RFC 9207). +- `mcp()` and `mcpConnector` answer the `elicitation/create` requests of MCP spec 2025 servers, and the input requests of spec 2026-07-28 servers, as session questions. One tool call can ask up to 5 times. + +`@tanstack/ai-harness` is an optional peer dependency. Only the `./harness` and `./connector` entries need it. diff --git a/docs/config.json b/docs/config.json index 02a28d1429..f918cb1a55 100644 --- a/docs/config.json +++ b/docs/config.json @@ -1012,6 +1012,16 @@ "label": "Run agents from a harness", "to": "harness/subagents", "addedAt": "2026-10-13" + }, + { + "label": "Use MCP servers", + "to": "harness/mcp", + "addedAt": "2026-10-13" + }, + { + "label": "Use from any MCP client", + "to": "harness/mcp-server", + "addedAt": "2026-10-13" } ] }, diff --git a/docs/harness/mcp-server.md b/docs/harness/mcp-server.md new file mode 100644 index 0000000000..25c2e414a5 --- /dev/null +++ b/docs/harness/mcp-server.md @@ -0,0 +1,243 @@ +--- +title: Use a harness from any MCP client +id: harness-mcp-server +order: 9 +description: "Serve your harness as an MCP server. Chat with it, answer its approvals, and run its agents and commands from Claude Code, Claude Desktop, Cursor, or another agent." +keywords: + - tanstack ai + - harness + - mcp + - mcp server + - claude code + - cursor +--- + +You built a harness, but you do your work in Claude Code, Cursor, or another agent. You want to give work to your harness from there. These clients speak MCP. `createHarnessMcpServer` serves your harness as an MCP server. The client can then chat with the harness, answer its approvals, and run its agents and commands. + +## 1. Install the MCP package + + + +react: @tanstack/ai-harness @tanstack/ai-mcp @tanstack/ai-persistence +vue: @tanstack/ai-harness @tanstack/ai-mcp @tanstack/ai-persistence +solid: @tanstack/ai-harness @tanstack/ai-mcp @tanstack/ai-persistence +svelte: @tanstack/ai-harness @tanstack/ai-mcp @tanstack/ai-persistence +preact: @tanstack/ai-harness @tanstack/ai-mcp @tanstack/ai-persistence +angular: @tanstack/ai-harness @tanstack/ai-mcp @tanstack/ai-persistence +octane: @tanstack/ai-harness @tanstack/ai-mcp @tanstack/ai-persistence +vanilla: @tanstack/ai-harness @tanstack/ai-mcp @tanstack/ai-persistence + + + +## 2. Serve the harness on stdio + +Put this in `mcp.ts`. It opens sessions on your host, and reads MCP messages on stdin and writes the answers on stdout: + +```ts group=harness-mcp-server +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { createHarnessMcpServer } from '@tanstack/ai-mcp/harness' +import { serveMCPStdio } from '@tanstack/ai-mcp/server/stdio' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +const host = createHarnessHost({ persistence: memoryPersistence() }) + +export const server = await createHarnessMcpServer({ + host, + harness: assistant, +}) + +serveMCPStdio(server) +``` + +For Claude Code, run this command one time: + +```bash +claude mcp add --env OPENAI_API_KEY=your-key assistant -- npx tsx /path/to/mcp.ts +``` + +For Claude Desktop, Cursor, and other clients with a JSON file, add the same command under `mcpServers`: + +```json +{ + "mcpServers": { + "assistant": { + "command": "npx", + "args": ["tsx", "/path/to/mcp.ts"], + "env": { "OPENAI_API_KEY": "your-key" } + } + } +} +``` + +- Claude Desktop keeps this in `claude_desktop_config.json`. +- Cursor keeps this in `.cursor/mcp.json`. + +The client starts the process when it needs the harness. The process stops when the client closes stdin. In this mode, stdout carries only MCP messages. Write your own logs with `console.error`. + +## 3. Use the tools + +Ask the client to use the harness, for example: "Ask assistant to summarize my open tickets." The client calls these tools. + +To talk to the harness: + +- `chat`: send a message and wait for the answer. While a turn runs, the message waits in the queue. To send files too, see [Send files and get media](#send-files-and-get-media). +- `steer`: add a message to the running turn. +- `cancel`: cancel the running turn. +- `status`: show the status, the interrupts and questions that wait, the background agents, and the queued turns. + +To answer the harness: + +- `approve` and `reject`: answer every approval that waits. +- `resolve`: answer every interrupt that waits, in one call. +- `answer`: answer a question from a command or a plugin. + +To run what the harness has: + +- `agent_`: one tool for each agent in `expose.agents`. It takes the agent input and returns the agent result. +- `command_`: one tool for each command in `expose.commands`. The command `connect:notion` becomes the tool `command_connect_notion`. + +The harness exposes nothing by default. Add the agents and commands that an MCP client can run to `expose` in `defineHarness`. See [Choose what clients can change](./connect#choose-what-clients-can-change). + +Two names can give the same tool name, for example `connect:notion` and `connect_notion`. Then the later one in name order gets a number: `command_connect_notion_2`. The description of each of these tools names its command or agent. + +Every tool takes an optional `threadId`. Each thread is its own conversation. Without a `threadId`, a tool uses the `threadId` option of the server (default `main`). + +`chat` answers with JSON: + +```json +{ + "status": "completed", + "text": "You have three open tickets.", + "interrupts": [], + "questions": [] +} +``` + +## Interrupts and questions + +A turn can stop for an interrupt. The turn then waits for your answer. Each interrupt has a `kind`: + +- `approval`: a tool call that needs your yes or no. +- `client-tool`: a tool that the client runs. The turn waits for the tool output. +- `generic`: a question from a middleware. The answer must match its `responseSchema`. + +For an approval, the client asks you in its own window (MCP elicitation). Answer `yes` to run the tool. Any other answer rejects it. + +In these cases, `chat` returns the interrupts with `status: "interrupted"`: + +- The client cannot ask you. +- You close the question without an answer. +- An interrupt of another kind also waits. + +```json +{ + "status": "interrupted", + "text": "Here is the plan.", + "interrupts": [ + { + "id": "interrupt-1", + "kind": "generic", + "message": "Review the plan", + "responseSchema": { + "type": "object", + "properties": { "note": { "type": "string" } }, + "required": ["note"] + } + } + ], + "questions": [] +} +``` + +An approval and a client tool also have `tool` and `args`, the tool call that waits. + +Then the client calls `approve`, `reject`, or `resolve`. That call waits for the turn to continue and returns the same JSON as `chat`. `approve` and `reject` answer approvals only. When an interrupt of another kind also waits, they refuse, and the client uses `resolve`. + +`resolve` takes one decision for each interrupt that waits. This example answers a turn with one approval and one generic interrupt: + +```json +{ + "decisions": [ + { "interruptId": "approval-1", "approved": true }, + { "interruptId": "interrupt-1", "payload": { "note": "Ship it." } } + ] +} +``` + +- For an approval, set `approved`. `true` runs the tool, and `false` rejects it. +- For a client tool, set `payload` to the tool output. +- For a generic interrupt, set `payload` to a value that matches its `responseSchema`. + +`resolve` refuses a list that does not answer every interrupt. It also refuses a decision without the field that its kind needs. + +To approve every tool call without a question, set `approvals: 'auto'`. Use it only with tools that you trust. It answers approvals only, so other interrupts still come back in the result. + +A command or a plugin can also ask you a question. The result then has `status: "waiting"` and the question in `questions`. The client calls `answer` with the question `id` as `questionId` and your `value`. `answer` waits for the work that asked, then returns its result. + +## Send files and get media + +`chat` takes `attachments`, the files to send with the message. Each attachment is one of these: + +- `{ path }`: a file on the machine of the server. The file must be inside a folder that the server can read. +- `{ url, mimeType }`: a URL that goes to the model as it is. The server does not download it. `mimeType` is optional when the URL ends in a known file extension. +- `{ data, mimeType }`: the bytes of the file in base64. + +Each attachment also takes an optional `name`. This `chat` call sends a screenshot: + +```json +{ + "message": "What is wrong in this screenshot?", + "attachments": [{ "path": "./bug.png" }] +} +``` + +The server reads a `path` only inside the `filePaths` option. By default it reads no files, so send `url` or `data`. + +When the work makes media, the results of `chat`, `approve`, `reject`, `resolve`, and `answer` include it: + +- An image or an audio file up to 5 MB comes back inline, as MCP image or audio content. +- Any other file comes back as a `resource_link` to `harness-media:///`. The client reads the bytes with `resources/read`. +- The JSON result lists each file in `media`, with its `id`, `kind`, `name`, `mimeType`, `size`, and `uri`. + +## Serve it over HTTP + +The server also has a `fetch` handler. Mount it on a route of your app, for example `/mcp`, and check a token first: + +```ts group=harness-mcp-server +export async function handleMcp(request: Request) { + const token = request.headers.get('authorization') + if (token !== `Bearer ${process.env.MCP_TOKEN}`) { + return new Response('Unauthorized', { status: 401 }) + } + return server.fetch(request) +} +``` + +The server opens sessions with no principal. Every caller that passes your check can use every thread and its media, so give each person their own server and token. + +Then add it to Claude Code: + +```bash +claude mcp add --transport http assistant https://your-app.example.com/mcp --header "Authorization: Bearer your-token" +``` + +`createHarnessMcpServer` takes these options next to `host` and `harness`: + +- `threadId`: the conversation of a tool call without a `threadId`. Default `main`. +- `approvals`: `'ask'` (default) asks in the client, or returns the approvals when the client cannot ask. `'auto'` approves every tool call. Both answer approvals only. +- `filePaths`: the folders that a `path` attachment can read from. The real path of the file, after every symlink, must be inside one of them. Default: none, so the server refuses every `path` attachment. +- `name` and `version`: the MCP server name and version. Default: the harness name and `1.0.0`. + +## What you have now + +- Your harness in Claude Code, Claude Desktop, Cursor, or any other MCP client. +- Approvals that the client asks you about, or answers with `approve` and `reject`. +- Every kind of interrupt, answered with one `resolve` call. +- The agents and commands of your harness as MCP tools. +- Files that the client sends with a message, and the media of each turn in the result. diff --git a/docs/harness/mcp.md b/docs/harness/mcp.md new file mode 100644 index 0000000000..611a16889b --- /dev/null +++ b/docs/harness/mcp.md @@ -0,0 +1,216 @@ +--- +title: Use MCP servers +id: harness-mcp +order: 8 +description: "Connect a harness to MCP servers: a local server that runs as a process, a remote server, or Notion and Linear with an OAuth sign-in. The model gets the tools of each server, with its name as a prefix." +keywords: + - tanstack ai + - harness + - mcp + - connectors + - oauth + - stdio + - notion + - linear +--- + +Your agent needs tools from MCP servers. Some servers run on your machine. Some are remote and take an API key. Notion and Linear sign in with OAuth. `mcp()` connects each server by name, and gives the model the tools of each server. + +This page connects your harness to MCP servers. To use your harness from an MCP client such as Claude Code, read [Use a harness from any MCP client](./mcp-server). + +## 1. Add the servers + + + +react: @tanstack/ai-harness @tanstack/ai-mcp +vue: @tanstack/ai-harness @tanstack/ai-mcp +solid: @tanstack/ai-harness @tanstack/ai-mcp +svelte: @tanstack/ai-harness @tanstack/ai-mcp +preact: @tanstack/ai-harness @tanstack/ai-mcp +angular: @tanstack/ai-harness @tanstack/ai-mcp +octane: @tanstack/ai-harness @tanstack/ai-mcp +vanilla: @tanstack/ai-harness @tanstack/ai-mcp + + + +```ts group=harness-mcp +import { defineHarness } from '@tanstack/ai-harness' +import { mcp } from '@tanstack/ai-mcp/harness' +import { openaiText } from '@tanstack/ai-openai' + +export const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [ + mcp({ + servers: { + files: { + type: 'stdio', + command: 'npx', + args: ['-y', '@modelcontextprotocol/server-filesystem', '.'], + }, + docs: { + type: 'http', + url: 'https://mcp.example.com/mcp', + headers: { authorization: `Bearer ${process.env.DOCS_TOKEN}` }, + }, + linear: { type: 'http', url: 'https://mcp.linear.app/mcp', oauth: true }, + }, + }), + ], +}) +``` + +The name of a server is the prefix of its tools. A `search` tool of the `docs` server is `docs_search` for the model. + +- `stdio`: the server runs as a child process on this machine (Node only). It stops when the session closes. +- `http`: a server that speaks Streamable HTTP. Send an API key in `headers`. +- `oauth: true`: the user signs in with `/connect linear`. See [Sign in to an OAuth server](#3-sign-in-to-an-oauth-server). + +## 2. Check the servers + +The servers connect in the background when the session opens. The first turn waits for them. Each server connects on its own, so a server that fails does not stop the tools of the others. + +`/mcp` shows each server and its status: + +```text +files: connected (14 tools) +docs: failed: Failed to connect to MCP server: Version negotiation probe failed: connection refused +linear: failed: Not signed in. Run /connect linear. +``` + +In your own UI, read the same list from `session.snapshot().plugins['tanstack/mcp'].servers`. Each entry is an `McpServerStatus`: + +- `name`: the name of the server. +- `status`: `connecting`, `connected`, or `failed`. A `failed` entry has an `error`. +- `toolCount`: how many tools the server gives the model. + +## 3. Sign in to an OAuth server + +1. Run `/connect linear` in the CLI. +2. The browser opens the Linear consent page. Approve it. +3. The CLI prints `Connected to linear.` +4. Ask for your issues. The next turn has the Linear tools. + +Before sign-in, the model knows that Linear is not connected. If the user asks for Linear, the model tells them to run `/connect linear`. + +The sign-in follows the MCP authorization spec: + +- The connector reads the server's OAuth metadata and registers a client for this sign-in. +- It signs in with PKCE and a random `state`, and listens on `127.0.0.1` for one callback. +- The token and the registered client go into `stores.credentials`. The next start of the harness uses them, so the user signs in once. +- A new `/connect linear` replaces them. Nothing from the old sign-in stays. +- `/disconnect linear` deletes them. + +Sometimes the server no longer accepts the sign-in, for example after the user removes the app in Linear. Then the turn runs without the Linear tools, and clients get a `harness.auth_required` event. The CLI prints `Sign in to linear. Run /connect linear.`, and the model tells the user to run `/connect linear`. If the server refuses the saved refresh token, the connector deletes the saved sign-in. + +The model never sees the token. To keep sign-ins after a restart, give the host a credential store. [Auth and connectors](./auth) shows one. With `oauth: true`, `headers` and `authProvider` do not apply. + +## Choose which tools ask first + +- A tool of an `oauth: true` server asks for approval before it runs, unless the server marks it read-only. +- The tools of the other servers run without a question. + +## Answer a question from a server + +An MCP server can ask the user for input in the middle of a tool call. MCP calls this elicitation. The session asks the user, and the tool call waits for the answer. + +- A form request becomes a question with the `message` and the requested `schema`. +- A URL request becomes a question with a `url`. The user opens the page, then answers. The session does not open the page. +- `mcp()` and `mcpConnector` both ask this way, for servers on MCP spec 2025 and on spec 2026-07-28. +- One tool call can ask more than once. On spec 2026-07-28, one call can ask up to 5 times. One more request fails the tool call. + +Answer with `session.answer(questionId, value)`, or from a [`HarnessClient`](./connect). The value tells the server what the user did: + +| Value | The server gets | +| --- | --- | +| The form content, for example `{ city: 'Paris' }` | `accept` with that content | +| Any other value, for a URL request | `accept` | +| `{ action: 'decline' }` | `decline` | +| `{ action: 'cancel' }` | `cancel` | + +This code answers the first open question with a city: + +```ts group=harness-mcp +import type { HarnessSession } from '@tanstack/ai-harness' + +export async function answerCity(session: HarnessSession) { + const [question] = session.snapshot().pendingQuestions + if (question) await session.answer(question.questionId, { city: 'Paris' }) +} +``` + +## Set a time limit + +`timeoutMs` limits how long one server can take to connect, to send its tool list, and to answer each tool call: + +```ts group=harness-mcp +export const quick = mcp({ + servers: { + docs: { type: 'http', url: 'https://mcp.example.com/mcp', timeoutMs: 10_000 }, + }, +}) +``` + +- The default is the MCP SDK default, 60 seconds. +- A server that is too slow gets the status `failed`. The tools of the other servers still work. +- If a server does not connect in time, its open request closes, or its child process stops. +- A `stdio` server can take up to two times `timeoutMs` to connect. When the version check runs out of time, the SDK tries the older handshake, with its own `timeoutMs`. + +An `http` server that does not connect in time shows this in `/mcp`: + +```text +docs: failed: Failed to connect to MCP server: Version negotiation probe timed out after 10000ms +``` + +## One OAuth server with more options + +`mcpConnector` adds one OAuth server, with options that `mcp()` does not have. The sign-in works as in [step 3](#3-sign-in-to-an-oauth-server): + +```ts group=harness-mcp +import { mcpConnector } from '@tanstack/ai-mcp/connector' + +export const linearWithoutWrites = mcpConnector({ + id: 'linear', + label: 'Linear', + url: 'https://mcp.linear.app/mcp', + // Ask before every tool whose name starts with "save". + needsApproval: (tool) => tool.name.startsWith('save'), +}) +``` + +Add it to the plugins like any plugin: `plugins: () => [linearWithoutWrites]`. The CLI then prints `Connected to Linear.`, with the `label`. + +- `needsApproval`: which tools ask first. Default: every tool that the server does not mark read-only. +- `prefix`: the tool name prefix. Default: the `id`. +- `scopes`: the OAuth scopes to ask for. Default: what the server offers. +- `clientName`: the app name on the consent page. Default: `TanStack AI Harness`. + +## Tools that appear later + +A plugin can give tools that it finds at run time, like the tools of a server that the user signed in to after the session started. Return them from `discoverTools`. The session asks each plugin before every turn: + +```ts group=harness-mcp +import { definePlugin } from '@tanstack/ai-harness' +import type { AnyTool } from '@tanstack/ai' + +const found: Array = [] + +export const lateTools = definePlugin({ + name: 'acme/late-tools', + setup: () => ({ + discoverTools: () => found, + }), +}) +``` + +- A tool whose name is already taken is skipped. The first tool keeps the name. +- If `discoverTools` throws, the turn runs without that plugin's tools, and clients get a `harness.plugin.warning` event. + +## What you have now + +- Tools from local, remote, and OAuth MCP servers, with the server name as a prefix. +- A status for each server, in `/mcp` and in your UI. +- One browser sign-in per service, kept in your credential store. +- A question before the tools that change data. +- Server requests for input, asked as session questions. diff --git a/packages/ai-mcp/package.json b/packages/ai-mcp/package.json index ad7268685a..c2df5435c9 100644 --- a/packages/ai-mcp/package.json +++ b/packages/ai-mcp/package.json @@ -52,6 +52,14 @@ "./server/stdio": { "types": "./dist/esm/server/stdio.d.ts", "import": "./dist/esm/server/stdio.js" + }, + "./connector": { + "types": "./dist/esm/connector.d.ts", + "import": "./dist/esm/connector.js" + }, + "./harness": { + "types": "./dist/esm/harness.d.ts", + "import": "./dist/esm/harness.js" } }, "files": [ @@ -76,7 +84,17 @@ "@tanstack/ai": "workspace:^", "jose": "^6.2.12" }, + "peerDependencies": { + "@tanstack/ai-harness": "workspace:^" + }, + "peerDependenciesMeta": { + "@tanstack/ai-harness": { + "optional": true + } + }, "devDependencies": { + "@tanstack/ai-harness": "workspace:*", + "@tanstack/ai-persistence": "workspace:*", "@vitest/coverage-v8": "4.1.10", "jiti": "^2.4.2", "json-schema-to-typescript": "^15.0.4", diff --git a/packages/ai-mcp/skills/ai-mcp/SKILL.md b/packages/ai-mcp/skills/ai-mcp/SKILL.md index d5b580c6b4..397bc4f7d6 100644 --- a/packages/ai-mcp/skills/ai-mcp/SKILL.md +++ b/packages/ai-mcp/skills/ai-mcp/SKILL.md @@ -17,6 +17,7 @@ sources: - 'TanStack/ai:packages/ai-mcp/src/transport.ts' - 'TanStack/ai:packages/ai-mcp/src/server/create-server.ts' - 'TanStack/ai:packages/ai-mcp/src/server/stdio.ts' + - 'TanStack/ai:packages/ai-mcp/src/harness-plugin.ts' --- # `@tanstack/ai-mcp` @@ -54,6 +55,12 @@ The package has these subpaths: - `./server` exports `createMCPServer`. - `./server/stdio` exports `serveMCPStdio`. - `./apps` exports `createMcpAppCallHandler`. +- `./harness` exports `createHarnessMcpServer` and the `mcp()` harness + plugin. `createHarnessMcpServer` serves a TanStack AI harness as an MCP + server. See `docs/harness/mcp-server.md`, and "`mcp()`: MCP servers in a + harness" below. +- `./connector` exports `mcpConnector`: one OAuth MCP server as a harness + plugin. See `docs/harness/mcp.md`. Import `./stdio` and `./server/stdio` only from Node code. Those entries use Node I/O. @@ -818,6 +825,65 @@ await using pool = await createMCPClients({ }) ``` +## `mcp()`: MCP servers in a harness + +In a TanStack AI harness, use the `mcp()` plugin. It connects each server, +gives the model the tools as `_`, and adds a `/mcp` command +that shows the status of each server. + +```typescript +import { defineHarness } from '@tanstack/ai-harness' +import { openaiText } from '@tanstack/ai-openai' +import { mcp } from '@tanstack/ai-mcp/harness' + +const agent = defineHarness({ + name: 'acme/agent', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [ + mcp({ + servers: { + files: { + type: 'stdio', + command: 'npx', + args: ['-y', '@modelcontextprotocol/server-filesystem', '.'], + }, + docs: { + type: 'http', + url: 'https://mcp.example.com/mcp', + headers: { authorization: `Bearer ${process.env.DOCS_TOKEN}` }, + timeoutMs: 10_000, + }, + notion: { + type: 'http', + url: 'https://mcp.notion.com/mcp', + oauth: true, + }, + }, + }), + ], +}) +``` + +How it works: + +- Each server connects on its own, in the background. The first turn waits + for the connections. +- A server that fails gets the status `failed` with its error. The tools of + the other servers still work. +- The status of a server is `connecting`, `connected`, or `failed`. `/mcp` + prints it. A UI reads `snapshot().plugins['tanstack/mcp'].servers`. +- A `stdio` server runs as a child process (Node only). It stops when the + session closes. +- A server request for input (MCP elicitation, spec 2025 and 2026) becomes + a session question. The tool call waits for `session.answer()`. + +Options for each server: + +- `timeoutMs`: the limit for the connect, the tool list, and each tool call + of that server. +- `oauth: true` (`http` only): sign in like `mcpConnector`. The user runs + `/connect `, and the tools come after the sign-in. + ## Abort signal — cancelling in-flight MCP calls TanStack AI stops waiting for MCP tool calls when the chat run's diff --git a/packages/ai-mcp/src/client.ts b/packages/ai-mcp/src/client.ts index 64726c9c3b..1ad773326c 100644 --- a/packages/ai-mcp/src/client.ts +++ b/packages/ai-mcp/src/client.ts @@ -37,6 +37,8 @@ import type { } from './types' import type { ClientOptions, + ElicitRequest, + ElicitResult, GetPromptResult, McpSubscription, Prompt, @@ -190,6 +192,7 @@ class MCPClientImpl< transport?: TransportConfig, clientOptions?: ClientOptions, policy: ToolPolicy = {}, + onElicit?: OnElicit, ) { this.prefix = prefix this.#transport = transport @@ -216,6 +219,12 @@ class MCPClientImpl< }, }, ) + // A spec 2025 server asks for input with a request to the client. + if (onElicit) { + this.#client.setRequestHandler('elicitation/create', (request) => + onElicit(request.params), + ) + } } getInfo(): { @@ -591,9 +600,18 @@ export async function createMCPClient( return connectTransport(options) } -async function connectTransport< +/** Answers a spec 2025 `elicitation/create` request. */ +export type OnElicit = ( + params: ElicitRequest['params'], +) => Promise + +/** + * Connects a client to a transport. The harness passes `onElicit` to answer + * the `elicitation/create` requests of spec 2025 servers. + */ +export async function connectTransport< TServer extends ServerDescriptor = AutomaticDescriptor, ->(options: MCPClientOptions) { +>(options: MCPClientOptions, onElicit?: OnElicit) { const transport = await resolveTransport(options.transport) const impl = new MCPClientImpl( options.prefix, @@ -609,6 +627,7 @@ async function connectTransport< toolName: options.toolName, requestOptions: options.requestOptions, }, + onElicit, ) await impl.connect(transport) return impl diff --git a/packages/ai-mcp/src/connector.ts b/packages/ai-mcp/src/connector.ts new file mode 100644 index 0000000000..f30f2cebc0 --- /dev/null +++ b/packages/ai-mcp/src/connector.ts @@ -0,0 +1,379 @@ +import { randomBytes } from 'node:crypto' +import { auth } from '@modelcontextprotocol/client' +import { + AuthRequiredError, + defineCommand, + definePlugin, + startLoopbackReceiver, +} from '@tanstack/ai-harness' +import { connectTransport } from './client' +import { MCPConnectionError } from './errors' +import { askForInput, elicitFor, harnessClientOptions } from './harness-plugin' +import type { + OAuthClientProvider, + OAuthDiscoveryState, + StoredOAuthClientInformation, +} from '@modelcontextprotocol/client' +import type { AnyTool } from '@tanstack/ai' +import type { CredentialsAccess, Principal } from '@tanstack/ai-harness' +import type { MCPClient } from './client' +import type { MCPClientOptions } from './types' + +export interface McpConnectorOptions { + /** A short id, for example `'notion'`. Commands are `connect:` and `disconnect:`. */ + id: string + label: string + /** The MCP server URL (Streamable HTTP), for example `https://mcp.notion.com/mcp`. */ + url: string + /** Tool name prefix. Default: the id. Tools are named `_` unless `toolName` is set. */ + prefix?: string + /** + * The name the model sees for each tool. Wins over `prefix`. + * See `MCPClientOptions.toolName`. + */ + toolName?: MCPClientOptions['toolName'] + /** Sent with the connect handshake, tool lists, tool calls, resources, and prompts. See `MCPClientOptions.requestOptions`. */ + requestOptions?: MCPClientOptions['requestOptions'] + /** OAuth scopes to ask for. Default: what the server offers. */ + scopes?: ReadonlyArray + /** The client name shown on the consent screen. Default `'TanStack AI Harness'`. */ + clientName?: string + /** + * Ask for approval before tools that can change data. Default: every tool + * the server does not mark `readOnlyHint`. + */ + needsApproval?: (tool: { + name: string + annotations?: { readOnlyHint?: boolean } + }) => boolean + /** Test hook for the OAuth and MCP requests. */ + fetch?: typeof fetch +} + +/** + * An `OAuthClientProvider` for the MCP SDK that keeps tokens (and the client + * that dynamic registration made) in the harness credential store. + */ +function credentialProvider( + id: string, + credentials: CredentialsAccess, + options: { + redirectUri: string + clientName: string + scopes: ReadonlyArray | undefined + state?: string + onRedirect?: (url: URL) => void + }, +) { + const { state } = options + let client: StoredOAuthClientInformation | undefined + let verifier = '' + let discovery: OAuthDiscoveryState | undefined + const stored = async () => { + const credential = await credentials.get(id) + return credential?.type === 'oauth' ? credential : undefined + } + return { + get redirectUrl() { + return options.redirectUri + }, + get clientMetadata() { + return { + client_name: options.clientName, + redirect_uris: [options.redirectUri], + grant_types: ['authorization_code', 'refresh_token'], + response_types: ['code'], + token_endpoint_auth_method: 'none', + ...(options.scopes?.length ? { scope: options.scopes.join(' ') } : {}), + } + }, + ...(state ? { state: () => state } : {}), + clientInformation: async () => { + if (client) return client + // A new sign-in registers a client for its own loopback port. + if (options.onRedirect) return undefined + const saved = (await stored())?.client + if (!saved) return undefined + // Kept, so the tokens saved after a refresh keep this client. + client = { + client_id: saved.clientId, + ...(saved.clientSecret ? { client_secret: saved.clientSecret } : {}), + ...(saved.issuer ? { issuer: saved.issuer } : {}), + } + return client + }, + saveClientInformation: (information) => { + client = information + }, + tokens: async () => { + if (options.onRedirect) return undefined + const credential = await stored() + if (!credential) return undefined + return { + access_token: credential.accessToken, + token_type: 'Bearer', + ...(credential.refreshToken + ? { refresh_token: credential.refreshToken } + : {}), + ...(credential.expiresAt + ? { + expires_in: Math.max( + 0, + Math.floor((credential.expiresAt - Date.now()) / 1000), + ), + } + : {}), + // The SDK checks that the tokens come from the same server (SEP-2352). + ...(credential.client?.issuer + ? { issuer: credential.client.issuer } + : {}), + } + }, + saveTokens: async (tokens) => { + const information = client + // `auth()` reads or registers the client before it saves tokens. + if (!information) { + throw new Error('The MCP SDK saved OAuth tokens before the client.') + } + // A new sign-in replaces the old one, with all its tokens. On a + // refresh, the SDK passes the old refresh token itself. + await credentials.set(id, { + type: 'oauth', + accessToken: tokens.access_token, + ...(tokens.refresh_token ? { refreshToken: tokens.refresh_token } : {}), + ...(tokens.expires_in + ? { expiresAt: Date.now() + tokens.expires_in * 1000 } + : {}), + ...(tokens.scope ? { scopes: tokens.scope.split(' ') } : {}), + client: { + clientId: information.client_id, + ...(information.client_secret + ? { clientSecret: information.client_secret } + : {}), + redirectUri: options.redirectUri, + issuer: information.issuer, + }, + }) + }, + // The SDK calls this with 'client' or 'tokens' when the server refuses + // the saved client or refresh token, for example a revoked refresh token. + // Both are in one saved credential, so either scope deletes it, and the + // user signs in again. A new sign-in keeps the old one until it works. + invalidateCredentials: async () => { + if (!options.onRedirect) await credentials.delete(id) + }, + redirectToAuthorization: (url) => { + // Outside `/connect`, a missing or revoked sign-in stops the tool. + if (!options.onRedirect) throw new AuthRequiredError(id) + options.onRedirect(url) + }, + saveCodeVerifier: (value) => { + verifier = value + }, + codeVerifier: () => verifier, + // Kept like the code verifier: `/connect` checks that the code comes + // from the authorization server it found first (SEP-2352). + saveDiscoveryState: (value) => { + discovery = value + }, + discoveryState: () => discovery, + } satisfies OAuthClientProvider +} + +/** The MCP sign-in of one sender, with its client and tools once found. */ +interface Connection { + connected: boolean + client?: MCPClient + tools?: ReadonlyArray +} + +/** + * A plugin that signs the user in to a remote MCP server (for example Notion + * or Linear) and gives the model its tools, the way Claude Code connects to + * MCP servers: + * + * - `/connect ` finds the server's OAuth settings, registers a client, + * and opens the browser (PKCE, loopback on `127.0.0.1`). + * - The tokens stay in the credential store. The model never sees them. + * - After sign-in, the next turn has the server's tools, named + * `_` by default. `toolName` can change the names. + * Tools that can change data ask for approval. + * + * @example + * ```ts + * const notion = mcpConnector({ id: 'notion', label: 'Notion', url: 'https://mcp.notion.com/mcp' }) + * ``` + */ +export function mcpConnector(options: McpConnectorOptions) { + const { id, label, url } = options + const prefix = options.prefix ?? id + const clientName = options.clientName ?? 'TanStack AI Harness' + return definePlugin({ + name: `connector/${id}`, + setup: async (ctx) => { + // One sign-in per sender: in a shared thread, one person's sign-in, + // client, and tools never serve another person's turn. + const connections = new Map() + const keyOf = (principal: Principal | undefined) => + principal + ? JSON.stringify([principal.tenantId ?? null, principal.id]) + : '' + const close = async (connection: Connection | undefined) => { + await connection?.client?.close().catch(() => {}) + } + /** Set the sign-in of `principal`, and drop its old client and tools. */ + const signedIn = async ( + principal: Principal | undefined, + connected: boolean, + ) => { + const key = keyOf(principal) + const old = connections.get(key) + connections.set(key, { connected }) + await close(old) + } + await ctx.resources.acquire( + () => undefined, + () => Promise.all([...connections.values()].map(close)), + ) + + return { + prompts: [ + { + id: `connector/${id}:status`, + // Read after `discoverTools`, which finds the sender's state. + text: () => + connections.get(keyOf(ctx.session.principal))?.connected + ? '' + : `${label} is not connected. If the user asks for ${label}, tell them to run /connect ${id}.`, + }, + ], + // Runs in a turn, so `ctx.session.principal` and `ctx.credentials` + // are the turn's sender. + discoverTools: async () => { + const key = keyOf(ctx.session.principal) + let connection = connections.get(key) + if (!connection) { + connection = { connected: (await ctx.credentials.get(id)) !== null } + connections.set(key, connection) + } + if (!connection.connected) return [] + if (!connection.tools) { + // The SDK only refreshes tokens for a client with a redirect URL. + // Without one it treats the client as machine-to-machine. A new + // browser sign-in still happens only through /connect. + const saved = await ctx.credentials.get(id) + const redirectUri = + (saved?.type === 'oauth' + ? saved.client?.redirectUri + : undefined) ?? 'http://127.0.0.1/callback' + try { + connection.client = await connectTransport( + { + transport: { + type: 'http', + url, + authProvider: credentialProvider(id, ctx.credentials, { + redirectUri, + clientName, + scopes: options.scopes, + }), + ...(options.fetch ? { fetch: options.fetch } : {}), + }, + prefix, + toolName: options.toolName, + requestOptions: options.requestOptions, + needsApproval: + options.needsApproval ?? + ((tool) => tool.annotations?.readOnlyHint !== true), + clientOptions: harnessClientOptions, + }, + elicitFor(ctx.session), + ) + } catch (error) { + // The server wants a new sign-in: ask the user to run /connect. + // Other failures, for example a server that is down, stay + // connection errors. + const needsSignIn = + error instanceof MCPConnectionError && + error.cause instanceof AuthRequiredError + if (!needsSignIn) throw error + connection.connected = false + ctx.session.authRequired({ connector: id }) + throw error.cause + } + const client = connection.client + // A failed tool list closes the client: the next turn connects + // again, and must not leave this one open. + const listed = await client + .tools() + .catch(async (error: unknown) => { + connection.client = undefined + await client.close().catch(() => {}) + throw error + }) + connection.tools = askForInput( + listed as ReadonlyArray, + ctx.session, + ) + } + return connection.tools + }, + commands: { + [`connect:${id}`]: defineCommand({ + description: `Sign in to ${label}`, + // The sign-in belongs to the user who runs the command. + run: async (_input, { credentials, principal }) => { + const receiver = await startLoopbackReceiver() + const state = randomBytes(16).toString('base64url') + try { + const provider = credentialProvider(id, credentials, { + redirectUri: receiver.redirectUri, + clientName, + scopes: options.scopes, + state, + onRedirect: (authorizationUrl) => + ctx.session.authRequired({ + connector: id, + url: authorizationUrl.href, + }), + }) + const fetchFn = options.fetch ? { fetchFn: options.fetch } : {} + // A new sign-in has no tokens, so this always ends in a + // redirect to the browser. + await auth(provider, { serverUrl: url, ...fetchFn }) + // The error names the service: a sign-in can end minutes later. + const { code, iss } = await receiver + .waitForCode(state) + .catch((error: unknown) => { + throw new Error( + `${label}: ${error instanceof Error ? error.message : String(error)}`, + ) + }) + // A server that sends `iss` (RFC 9207), such as Linear, gets + // it checked. Without it, the SDK refuses the code. + await auth(provider, { + serverUrl: url, + authorizationCode: code, + ...(iss ? { iss } : {}), + ...fetchFn, + }) + } finally { + receiver.close() + } + await signedIn(principal, true) + return `Connected to ${label}.` + }, + }), + [`disconnect:${id}`]: defineCommand({ + description: `Sign out of ${label}`, + run: async (_input, { credentials, principal }) => { + await credentials.delete(id) + await signedIn(principal, false) + return `Disconnected from ${label}.` + }, + }), + }, + } + }, + }) +} diff --git a/packages/ai-mcp/src/harness-plugin.ts b/packages/ai-mcp/src/harness-plugin.ts new file mode 100644 index 0000000000..545300c145 --- /dev/null +++ b/packages/ai-mcp/src/harness-plugin.ts @@ -0,0 +1,349 @@ +import { defineCommand, definePlugin } from '@tanstack/ai-harness' +import { connectTransport } from './client' +import type { ElicitResult } from '@modelcontextprotocol/client' +import type { AnyTool, JSONSchema } from '@tanstack/ai' +import type { + AnyCommand, + PluginPrompt, + PluginSessionApi, +} from '@tanstack/ai-harness' +import type { OnElicit } from './client' +import type { AskInput } from './tools' +import type { HttpTransportConfig, StdioTransportConfig } from './transport' + +/** One server of {@link mcp}, keyed by its name. */ +export type McpServerConfig = ( + | StdioTransportConfig + | (HttpTransportConfig & { + /** + * Sign in with OAuth, like `mcpConnector`. This adds `/connect ` + * and `/disconnect `. The tools of the server come after the + * sign-in, and the tools that are not read-only ask for approval. + * `headers` and `authProvider` do not apply then. + */ + oauth?: boolean + }) +) & { + /** + * The time in milliseconds that this server can take to connect, to send + * its tool list, or to answer a tool call. Default: the MCP SDK default, + * 60,000. + */ + timeoutMs?: number + /** + * Give the tools of this server to code mode: the `codeMode()` plugin of + * `@tanstack/ai-code-mode` takes them out of the tool list and lets the + * model call them from `execute_typescript`. Without that plugin, nothing + * changes. + */ + codeMode?: boolean +} + +/** + * The status of one server. UIs read the list from + * `snapshot().plugins['tanstack/mcp'].servers`. + */ +export type McpServerStatus = { + name: string + /** How many tools the server gives the model. */ + toolCount: number +} & ( + | { status: 'connecting' | 'connected' } + | { status: 'failed'; error: string } +) + +/** + * A harness plugin that connects MCP servers and gives the model their + * tools, named `_`. + * + * - Each server connects on its own. A server that fails does not stop the + * others: it gets the status `failed` with the error. + * - The servers connect in the background. The first turn waits for them. + * - `/mcp` lists the servers and their status. UIs read the same list from + * the plugin state. + * - A `stdio` server runs as a child process (Node only). It stops when the + * session closes. + * - An `http` server with `oauth: true` signs in through `/connect `. + * + * @param options.servers - The servers, keyed by name. The name is the tool + * prefix. + * + * @example + * ```ts + * import { mcp } from '@tanstack/ai-mcp/harness' + * + * plugins: () => [ + * mcp({ + * servers: { + * files: { type: 'stdio', command: 'npx', args: ['-y', '@modelcontextprotocol/server-filesystem', '.'] }, + * docs: { type: 'http', url: 'https://mcp.example.com/mcp', timeoutMs: 10_000 }, + * notion: { type: 'http', url: 'https://mcp.notion.com/mcp', oauth: true }, + * }, + * }), + * ] + * ``` + */ +export function mcp(options: { + servers: Readonly> +}) { + const servers = Object.entries(options.servers) + return definePlugin({ + name: 'tanstack/mcp', + setup: async (ctx) => { + const statuses = new Map() + for (const [name] of servers) { + statuses.set(name, { name, status: 'connecting', toolCount: 0 }) + } + const state = ctx.state({ servers: [...statuses.values()] }) + const setStatus = async (status: McpServerStatus) => { + const old = JSON.stringify(statuses.get(status.name)) + statuses.set(status.name, status) + if (old === JSON.stringify(status)) return + await state.update(() => ({ servers: [...statuses.values()] })) + } + // State from an earlier run of this session is out of date. + await state.update(() => ({ servers: [...statuses.values()] })) + + /** Connects one server, in the background. Never rejects. */ + const connect = async (name: string, server: McpServerConfig) => { + const opened = await ctx.resources + .acquire( + async () => { + // Loaded here: the stdio transport needs node:child_process. + const transport = + server.type === 'stdio' + ? (await import('./stdio')).stdioTransport(server) + : server + const client = await connectTransport( + { + transport, + prefix: name, + requestOptions: requestOptionsOf(server), + clientOptions: harnessClientOptions, + }, + elicitFor(ctx.session), + ) + try { + const tools = askForInput( + forCodeMode(server, await client.tools()), + ctx.session, + ) + return { client, tools } + } catch (error) { + // Connected, but no tool list: the server failed. + await client.close().catch(() => undefined) + throw error + } + }, + ({ client }) => client.close(), + ) + .catch((error: unknown) => new Error(errorText(error))) + const failed = opened instanceof Error + const status: McpServerStatus = failed + ? { name, status: 'failed', error: opened.message, toolCount: 0 } + : { name, status: 'connected', toolCount: opened.tools.length } + // After the session closed, the state store can be gone. `/mcp` + // reads `statuses`, which is set first. + await setStatus(status).catch(() => undefined) + return failed ? [] : opened.tools + } + + const commands: Record = { + mcp: defineCommand({ + description: 'Show the MCP servers and their status', + run: () => [...statuses.values()].map(statusLine).join('\n'), + }), + } + const prompts: Array = [] + // One tool list for each server, in the order of `servers`. + const lists = await Promise.all( + servers.map(async (entry) => { + const [name, server] = entry + const isOAuth = server.type === 'http' && server.oauth === true + if (!isOAuth) { + const connecting = connect(name, server) + return () => connecting + } + // Loaded here: the connector needs node:crypto and node:http. + const { mcpConnector } = await import('./connector') + const connector = await mcpConnector({ + id: name, + label: name, + url: server.url, + requestOptions: requestOptionsOf(server), + ...(server.fetch ? { fetch: server.fetch } : {}), + }).setup(ctx) + Object.assign(commands, connector.commands) + prompts.push(...connector.prompts) + // The sign-in belongs to the sender of the turn, so the status is + // the one of the last turn. + return async () => { + try { + const tools = await connector.discoverTools() + const signedIn = + tools.length > 0 || (await ctx.credentials.get(name)) !== null + await setStatus( + signedIn + ? { name, status: 'connected', toolCount: tools.length } + : { + name, + status: 'failed', + error: `Not signed in. Run /connect ${name}.`, + toolCount: 0, + }, + ) + return forCodeMode(server, tools) + } catch (error) { + // One server that fails must not hide the tools of the others. + await setStatus({ + name, + status: 'failed', + error: errorText(error), + toolCount: 0, + }) + return [] + } + } + }), + ) + + return { + commands, + prompts, + discoverTools: async () => { + const found = await Promise.all(lists.map((list) => list())) + return found.flat() + }, + } + }, + }) +} + +/** + * The harness asks the user for form and URL elicitations, so its clients + * declare both. `chat()` clients keep the default: form only. + */ +export const harnessClientOptions = { + capabilities: { elicitation: { form: {}, url: {} } }, +} + +/** + * Ask the session user when an MCP tool asks for input (a form or a URL + * elicitation). The tool call waits for the answer, then sends it to the + * server. On spec 2026, a tool call can ask up to 5 times. The answer is the + * form content, or an MCP result such as `{ action: 'decline' }` or + * `{ action: 'cancel' }`. + */ +export function askForInput( + tools: ReadonlyArray, + session: PluginSessionApi, +) { + const askInput: AskInput = async (request) => ({ + status: 'resolved', + payload: await ask(session, request), + }) + return tools.map((tool): AnyTool => { + const execute = tool.execute + if (!execute) return tool + return { + ...tool, + execute: (args, context) => + execute(args, { ...(isRecord(context) ? context : {}), askInput }), + } + }) +} + +/** Answers the `elicitation/create` requests of a spec 2025 server. */ +export function elicitFor(session: PluginSessionApi): OnElicit { + return async (params) => elicitResult(await ask(session, params)) +} + +/** Asks the session user for one MCP elicitation request body. */ +async function ask(session: PluginSessionApi, request: unknown) { + const body = isRecord(request) ? request : {} + const url = + body.mode === 'url' && typeof body.url === 'string' ? body.url : undefined + const answer = await session.ask({ + message: + typeof body.message === 'string' + ? body.message + : 'The MCP server asks for input.', + schema: isJsonSchema(body.requestedSchema) + ? body.requestedSchema + : undefined, + ...(url ? { url } : {}), + }) + // A URL request has no content: the answer only says "done". + return url !== undefined && !(isRecord(answer) && 'action' in answer) + ? { action: 'accept' } + : answer +} + +/** Maps an answer to the MCP result: decline, cancel, or accept. */ +function elicitResult(answer: unknown): ElicitResult { + if ( + isRecord(answer) && + (answer.action === 'decline' || answer.action === 'cancel') + ) { + return { action: answer.action } + } + const content = + isRecord(answer) && answer.action === 'accept' ? answer.content : answer + return isElicitContent(content) + ? { action: 'accept', content } + : { action: 'accept' } +} + +// ponytail: the SDK checks the result against the schema of the request. +function isElicitContent( + value: unknown, +): value is NonNullable { + return isRecord(value) +} + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null +} + +// ponytail: the server owns the schema. The session passes it on as is. +function isJsonSchema(value: unknown): value is JSONSchema { + return isRecord(value) +} + +/** The SDK request options for `timeoutMs`. */ +function requestOptionsOf(server: McpServerConfig) { + return server.timeoutMs === undefined + ? undefined + : { timeout: server.timeoutMs } +} + +/** Marks the tools of a `codeMode: true` server for the `codeMode()` plugin. */ +function forCodeMode(server: McpServerConfig, tools: ReadonlyArray) { + if (server.codeMode !== true) return tools + return tools.map((tool) => ({ + ...tool, + metadata: { ...tool.metadata, codeMode: true }, + })) +} + +function statusLine(server: McpServerStatus) { + switch (server.status) { + case 'connecting': + return `${server.name}: connecting` + case 'connected': { + const unit = server.toolCount === 1 ? 'tool' : 'tools' + return `${server.name}: connected (${server.toolCount} ${unit})` + } + case 'failed': + return `${server.name}: failed: ${server.error}` + } +} + +/** The error text for the status. A connect error keeps its reason in `cause`. */ +function errorText(error: unknown) { + if (!(error instanceof Error)) return String(error) + const { cause } = error + return cause instanceof Error + ? `${error.message}: ${cause.message}` + : error.message +} diff --git a/packages/ai-mcp/src/harness.ts b/packages/ai-mcp/src/harness.ts new file mode 100644 index 0000000000..737a29955e --- /dev/null +++ b/packages/ai-mcp/src/harness.ts @@ -0,0 +1,973 @@ +import { + EventType, + convertSchemaToJsonSchema, + readUnopenedInterruptBinding, + toolDefinition, +} from '@tanstack/ai' +import { + HARNESS_EVENTS, + isMediaRecord, + kindOf, + mediaPart, + mimeTypeOf, +} from '@tanstack/ai-harness' +// The inferred return type names `MCPHandleOptions`. Without an import of +// `./server/index`, the .d.ts emit writes `./server.js`, which does not resolve. +import { createMCPServer, resourceDefinition } from './server/index' +import { toCallToolResult } from './server/tasks' +import type { ContentPart, Interrupt, JSONSchema } from '@tanstack/ai' +import type { + AnyHarness, + Cursor, + HarnessHost, + HarnessSession, + MediaRecord, + Operation, +} from '@tanstack/ai-harness' +import type { ContentBlock } from '@modelcontextprotocol/server' +import type { MCPToolContext } from './server/context' + +export { mcp } from './harness-plugin' +export type { McpServerConfig, McpServerStatus } from './harness-plugin' + +/** Options for {@link createHarnessMcpServer}. */ +export interface HarnessMcpServerOptions { + /** The host that runs the sessions, from `createHarnessHost`. */ + host: HarnessHost + /** The harness to serve. */ + harness: AnyHarness + /** The conversation of a tool call that names no `threadId`. Default `'main'`. */ + threadId?: string + /** + * What happens when a turn stops for a tool call that needs approval. + * - `'ask'` (default): ask the user in the MCP client (elicitation). A + * client without elicitation gets the approvals in the result. It answers + * them with `approve`, `reject`, or `resolve`. + * - `'auto'`: approve every tool call. + * + * Both modes answer tool approvals only. When the turn also waits for + * another kind of interrupt, every interrupt comes back in the result. The + * client answers them all with one `resolve`. + */ + approvals?: 'ask' | 'auto' + /** + * The folders that a `path` attachment of `chat` can read from. A path must + * resolve, after every symlink, to a file inside one of them. Leave it out, + * and every `path` attachment is refused. + * + * The stdio CLI (`--mcp`) passes the working folder. An HTTP server should + * pass none, because a remote client must not read files on the server. + */ + filePaths?: ReadonlyArray + /** The MCP server name. Default: the harness name. */ + name?: string + /** The MCP server version. Default `'1.0.0'`. */ + version?: string +} + +/** One decision of a `resolve` call, from the client. */ +type Decision = { interruptId: string; approved?: boolean; payload?: unknown } + +/** The answer to one interrupt, for `session.resolve`. */ +type Answer = { interruptId: string; payload: unknown } + +/** One file of a `chat` call, from the client. */ +type Attachment = + | { path: string; name?: string } + | { url: string; mimeType?: string; name?: string } + | { data: string; mimeType: string; name?: string } + +/** The URI of a media resource is `harness-media:///`. */ +const MEDIA_URI_PREFIX = 'harness-media://' + +/** The biggest image or audio file that a result carries inline. */ +const INLINE_MEDIA_MAX_BYTES = 5 * 1024 * 1024 + +const threadIdSchema: JSONSchema = { + type: 'string', + description: + 'The conversation id. Leave it out to use the default conversation.', +} + +const nameSchema: JSONSchema = { + type: 'string', + description: 'The file name. Optional.', +} + +const attachmentsSchema: JSONSchema = { + type: 'array', + description: 'Files to send with the message.', + items: { + anyOf: [ + { + type: 'object', + properties: { + path: { + type: 'string', + description: + 'A file path on the machine of the server. The file must be in a folder that the server allows.', + }, + name: nameSchema, + }, + required: ['path'], + }, + { + type: 'object', + properties: { + url: { + type: 'string', + description: + 'A URL of the file. The model reads it. The server does not download it.', + }, + mimeType: { + type: 'string', + description: + 'The MIME type, for example image/png. Needed when the URL does not end in a known file extension.', + }, + name: nameSchema, + }, + required: ['url'], + }, + { + type: 'object', + properties: { + data: { type: 'string', description: 'The file bytes in base64.' }, + mimeType: { + type: 'string', + description: 'The MIME type, for example image/png.', + }, + name: nameSchema, + }, + required: ['data', 'mimeType'], + }, + ], + }, +} + +/** + * Serves a harness as an MCP server, so any MCP client (Claude Code, Claude + * Desktop, Cursor, another agent) can use it. + * + * The tools are `chat`, `steer`, `cancel`, `approve`, `reject`, `resolve`, + * `answer`, and `status`, plus `agent_` for each agent in + * `harness.expose.agents` and `command_` for each plugin command in + * `harness.expose.commands`. + * Every tool takes an optional `threadId`. Sessions open with + * `host.open(harness, { threadId })`. + * + * Each interrupt in a result has a `kind`: `approval`, `client-tool`, or + * `generic`. `approve` and `reject` answer approvals only. `resolve` answers + * every kind: `approved` for an approval, and `payload` for the others. + * + * `chat` takes `attachments`: `{ path }` (only inside `filePaths`), + * `{ url, mimeType? }` (the model reads the URL, the server does not fetch + * it), or `{ data, mimeType }` (base64). A result lists the media that the + * work made in `media`. An image or audio file up to 5 MB comes back inline. + * Any other file comes back as a `resource_link` to + * `harness-media:///`, which `resources/read` returns. + * + * The result is the server from `createMCPServer`. Mount `server.fetch` on + * an HTTP route, or pass the server to `serveMCPStdio`. + * + * @param options - The host, the harness, the default thread, the approval mode, and the folders for `path` attachments + * + * @example + * ```ts + * const server = await createHarnessMcpServer({ host, harness: assistant }) + * serveMCPStdio(server) + * ``` + */ +export async function createHarnessMcpServer(options: HarnessMcpServerOptions) { + const { host, harness } = options + const defaultThread = options.threadId ?? 'main' + const approvals = options.approvals ?? 'ask' + const filePaths = options.filePaths ?? [] + // Work that stopped for a question, by question id. `answer` continues it. + const waiting = new Map>() + + const open = (args: unknown) => + host.open(harness, { + threadId: + isRecord(args) && typeof args.threadId === 'string' + ? args.threadId + : defaultThread, + }) + + // Answers the approvals with the approval mode. `undefined` means that the + // client must answer. One resolve answers every interrupt, and only the + // client can answer an interrupt that is not an approval. + async function decide(interrupts: Array, context: MCPToolContext) { + if (!interrupts.every(isApproval)) return undefined + if (approvals === 'auto') { + return interrupts.map((interrupt) => ({ + interruptId: interrupt.id, + payload: true, + })) + } + const answers: Array = [] + for (const interrupt of interrupts) { + try { + const answer = await context.requestInput({ + message: approvalQuestion(interrupt), + }) + answers.push({ interruptId: interrupt.id, payload: isYes(answer) }) + } catch { + // ponytail: no elicitation, a declined question, or spec 2026 (it runs + // the whole call again). The client answers with approve or reject. + return undefined + } + } + return answers + } + + // Waits for the work. A chat turn that stops for approvals continues while + // the approval mode decides them. The result gets the media of the work. + async function finish( + session: HarnessSession, + work: Operation, + workFrom: Cursor, + context: MCPToolContext, + ) { + let operation = work + let from = workFrom + // The media of every operation that this call waited for. + const media: Array = [] + while (true) { + const outcome = await settle(session, operation, from) + if (!outcome.done) { + waiting.set(outcome.questionId, operation) + return { status: 'waiting', ...pending(session) } + } + media.push(...(await mediaOf(operation))) + if (operation.kind !== 'chat') { + return withMedia(session, outcome.value, media) + } + const interrupts = session.snapshot().pendingInterrupts + const stopped = + operation.status() === 'interrupted' && interrupts.length > 0 + const answers = stopped ? await decide(interrupts, context) : undefined + if (answers === undefined) { + const result = { + status: operation.status(), + text: turnText(outcome.value), + ...pending(session), + } + return withMedia(session, result, media) + } + from = session.snapshot().cursor + operation = (await resolveAll(session, answers)).turn + } + } + + async function resume( + session: HarnessSession, + answers: Array, + context: MCPToolContext, + ) { + const from = session.snapshot().cursor + const { turn } = await resolveAll(session, answers) + return finish(session, turn, from, context) + } + + async function decideAll( + args: unknown, + approved: boolean, + context: MCPToolContext, + ) { + const session = await open(args) + const interrupts = session.snapshot().pendingInterrupts + const other = interrupts.find((interrupt) => !isApproval(interrupt)) + if (other !== undefined) { + throw new Error(`Use resolve: interrupt ${other.id} needs a payload.`) + } + if (interrupts.length === 0) throw new Error('No approvals are waiting.') + const answers = interrupts.map((interrupt) => ({ + interruptId: interrupt.id, + payload: approved, + })) + return resume(session, answers, context) + } + + const chat = toolDefinition({ + name: 'chat', + description: + 'Send a message to the harness and wait for its answer. While a turn runs, the message waits in the queue. The result has the answer text, the status, the interrupts and questions that wait for you, and the media that the turn made.', + inputSchema: withThreadId({ + type: 'object', + properties: { + message: { type: 'string', description: 'The message.' }, + attachments: attachmentsSchema, + }, + required: ['message'], + }), + }).server(async (args, ctx) => { + const session = await open(args) + const message = await userInputOf(session, args, filePaths) + const from = session.snapshot().cursor + const turn = session.prompt(message, { busy: 'queue' }) + return finish(session, turn, from, ctx.context) + }) + + const steer = toolDefinition({ + name: 'steer', + description: + 'Add a message to the running turn. The model reads it at its next step. With no running turn, the message starts a new turn.', + inputSchema: withThreadId({ + type: 'object', + properties: { + message: { type: 'string', description: 'The message.' }, + }, + required: ['message'], + }), + }).server(async (args) => (await open(args)).steer(textArg(args, 'message'))) + + const cancel = toolDefinition({ + name: 'cancel', + description: 'Cancel the running turn.', + inputSchema: withThreadId(undefined), + }).server(async (args) => (await open(args)).cancel()) + + const approve = toolDefinition({ + name: 'approve', + description: + 'Approve every tool call that waits for approval, then wait for the turn to continue. When an interrupt of another kind also waits, use resolve.', + inputSchema: withThreadId(undefined), + }).server(async (args, ctx) => + decideAll(args, true, ctx.context), + ) + + const reject = toolDefinition({ + name: 'reject', + description: + 'Reject every tool call that waits for approval, then wait for the turn to continue. When an interrupt of another kind also waits, use resolve.', + inputSchema: withThreadId(undefined), + }).server(async (args, ctx) => + decideAll(args, false, ctx.context), + ) + + const resolve = toolDefinition({ + name: 'resolve', + description: + 'Answer every interrupt that waits, then wait for the turn to continue. Give one decision for each interrupt. For an approval, set approved. For a client tool, set payload to the tool output. For a generic interrupt, set payload to a value that matches its responseSchema.', + inputSchema: withThreadId({ + type: 'object', + properties: { + decisions: { + type: 'array', + items: { + type: 'object', + properties: { + interruptId: { + type: 'string', + description: 'The id of the interrupt.', + }, + approved: { + type: 'boolean', + description: + 'For an approval: true runs the tool call, false rejects it.', + }, + payload: { + description: + 'The answer. For a client tool, the tool output. For a generic interrupt, a value that matches its responseSchema.', + }, + }, + required: ['interruptId'], + }, + }, + }, + required: ['decisions'], + }), + }).server(async (args, ctx) => { + const session = await open(args) + const interrupts = session.snapshot().pendingInterrupts + const answers = answersOf(decisionsOf(args), interrupts) + return resume(session, answers, ctx.context) + }) + + const answer = toolDefinition({ + name: 'answer', + description: + 'Answer a question from the harness, then wait for the work that asked it.', + inputSchema: withThreadId({ + type: 'object', + properties: { + questionId: { type: 'string', description: 'The id of the question.' }, + value: { description: 'The answer. It matches the question schema.' }, + }, + required: ['questionId', 'value'], + }), + }).server(async (args, ctx) => { + const session = await open(args) + const questionId = textArg(args, 'questionId') + const from = session.snapshot().cursor + const receipt = await session.answer( + questionId, + isRecord(args) ? args.value : undefined, + ) + if (receipt.status === 'rejected') { + throw new Error( + `The harness refused the answer: ${receipt.reason ?? 'no reason'}.`, + ) + } + const operation = waiting.get(questionId) + waiting.delete(questionId) + // A question that no tool call waits on: nothing to continue. + if (operation === undefined) { + return { status: 'answered', ...pending(session) } + } + return finish(session, operation, from, ctx.context) + }) + + const status = toolDefinition({ + name: 'status', + description: + 'Show what the harness does now: the status, the interrupts and questions that wait, the background agents, and the queued turns.', + inputSchema: withThreadId(undefined), + }).server(async (args) => { + const session = await open(args) + const snapshot = session.snapshot() + return { + status: snapshot.status, + ...pending(session), + agents: snapshot.activeOperations + .filter((operation) => operation.kind === 'agent') + .map(({ id, agent }) => ({ id, agent })), + queuedTurns: snapshot.queuedTurns, + } + }) + + // Agents and commands come from a session. Every thread has the same ones. + const session = await host.open(harness, { threadId: defaultThread }) + + const exposed: ReadonlyArray = harness.expose?.agents ?? [] + const agents = exposed.flatMap((name) => { + const agent = session.registry.get(name) + return agent === undefined + ? [] + : [{ name, description: agent.description, agent }] + }) + const agentTools = toolNames('agent', agents).map( + ({ item: { name, agent }, toolName, description }) => + toolDefinition({ + name: toolName, + description, + inputSchema: withThreadId(convertSchemaToJsonSchema(agent.inputSchema)), + }).server(async (args, ctx) => { + const target = await open(args) + const handle = target.agent(name) + if (handle === undefined) throw new Error(`Unknown agent: ${name}`) + const from = target.snapshot().cursor + return finish(target, handle.start(inputOf(args)), from, ctx.context) + }), + ) + + // A command can change the session, for example `/mode bypass`, so an MCP + // client runs only the commands in `expose.commands`, like a protocol client. + const exposedCommands: ReadonlyArray = harness.expose?.commands ?? [] + const commands = session + .describe() + .commands.filter(({ name }) => exposedCommands.includes(name)) + const commandTools = toolNames('command', commands).map( + ({ item: command, toolName, description }) => + toolDefinition({ + name: toolName, + description, + inputSchema: withThreadId( + isJsonSchema(command.input) ? command.input : undefined, + ), + }).server(async (args, ctx) => { + const target = await open(args) + const from = target.snapshot().cursor + const operation = target.command(command.name, inputOf(args)) + return finish(target, operation, from, ctx.context) + }), + ) + + // The bytes of a `resource_link` from a result. The link names its thread, + // and a thread reads only its own media. + const mediaResource = resourceDefinition({ + uriTemplate: `${MEDIA_URI_PREFIX}{threadId}/{id}`, + name: 'media', + mimeType: 'application/octet-stream', + argsSchema: { parse: mediaAddress }, + }).read(async (_uri, { threadId, id }) => { + const target = await host.open(harness, { threadId }) + const record = await target.getMedia(id) + if (record === null) { + throw new Error(`Media ${id} was not found in thread ${threadId}.`) + } + const blob = toBase64(await target.loadMedia(id)) + return { blob, mimeType: record.mimeType } + }) + + return createMCPServer({ + name: options.name ?? harness.name, + version: options.version ?? '1.0.0', + // `approvals: 'ask'` asks by elicitation, and elicitation needs a spec 2025 + // session. ponytail: memory sessions live in this process, like the + // harness sessions. Add a `sessions` option when a host runs many instances. + sessions: 'memory', + resources: [mediaResource], + tools: [ + chat, + steer, + cancel, + approve, + reject, + resolve, + answer, + status, + ...agentTools, + ...commandTools, + ], + }) +} + +// The harness needs one resolve that answers every open interrupt. An async +// function waits for an operation that it returns, so the turn is wrapped. +async function resolveAll(session: HarnessSession, answers: Array) { + const receipt = await session.resolve( + answers.map((answer) => ({ ...answer, status: 'resolved' })), + ) + const operation = + receipt.operationId === undefined + ? undefined + : session.operation(receipt.operationId) + if (operation === undefined) { + throw new Error( + `The harness refused the decisions: ${receipt.reason ?? 'no reason'}.`, + ) + } + return { turn: operation } +} + +/** + * Waits for `operation`. Stops early when the session asks a question after + * `from`, because the work waits for that answer. + */ +async function settle( + session: HarnessSession, + operation: Operation, + from: Cursor, +) { + const stop = new AbortController() + const done = Promise.resolve(operation).then((value) => ({ + done: true as const, + value, + })) + const asked = (async () => { + const questionId = await questionAfter(session, from, stop.signal) + // The feed closed without a question: only the work can end the wait. + if (questionId === undefined) return done + return { done: false as const, questionId } + })() + try { + return await Promise.race([done, asked]) + } finally { + stop.abort() + } +} + +async function questionAfter( + session: HarnessSession, + from: Cursor, + signal: AbortSignal, +) { + const events = session.events({ from, signal }) + for await (const entry of events) { + const event = entry.event + const isQuestion = + event.type === EventType.CUSTOM && event.name === HARNESS_EVENTS.question + if (!isQuestion || !isRecord(event.value)) continue + const questionId = event.value.questionId + if (typeof questionId === 'string') return questionId + } + return undefined +} + +function pending(session: HarnessSession) { + const snapshot = session.snapshot() + return { + interrupts: snapshot.pendingInterrupts.map((interrupt) => ({ + id: interrupt.id, + kind: interruptKind(interrupt), + tool: interrupt.metadata?.toolName, + args: interrupt.metadata?.input, + message: interrupt.message, + responseSchema: interrupt.responseSchema, + })), + questions: snapshot.pendingQuestions.map(({ questionId, ...question }) => ({ + id: questionId, + ...question, + })), + } +} + +/** + * The message of a `chat` call: the text alone, or the text and one part per + * attachment. A file goes into the media store of the thread first. A + * `MediaError` (too big, a type the harness does not take) stops the call. + */ +async function userInputOf( + session: HarnessSession, + args: unknown, + filePaths: ReadonlyArray, +) { + const message = textArg(args, 'message') + const attachments = attachmentsOf(args) + if (attachments.length === 0) return message + const parts: Array = [{ type: 'text', content: message }] + for (const attachment of attachments) { + parts.push(await attachmentPart(session, attachment, filePaths)) + } + return parts +} + +function attachmentsOf(args: unknown) { + const list = + isRecord(args) && Array.isArray(args.attachments) ? args.attachments : [] + const attachments = list.filter(isAttachment) + if (attachments.length !== list.length) { + throw new Error('Each attachment needs path, url, or data with mimeType.') + } + return attachments +} + +async function attachmentPart( + session: HarnessSession, + attachment: Attachment, + filePaths: ReadonlyArray, +) { + if ('path' in attachment) { + const file = await readAllowedFile(attachment.path, filePaths) + const name = attachment.name ?? file.name + return mediaPart( + await session.putMedia(file.bytes, { mimeType: file.mimeType, name }), + ) + } + if ('url' in attachment) return urlPart(attachment) + const bytes = Uint8Array.from(atob(attachment.data), (char) => + char.charCodeAt(0), + ) + const name = attachment.name ?? 'attachment' + return mediaPart( + await session.putMedia(bytes, { mimeType: attachment.mimeType, name }), + ) +} + +/** + * The bytes of `path` when its real path is inside one of `folders`. The real + * path follows every symlink and `..`, so neither can leave a folder. A + * missing file gets the same refusal, so a client cannot probe for files. + */ +async function readAllowedFile(path: string, folders: ReadonlyArray) { + if (folders.length === 0) { + throw new Error( + 'This server does not read files. Send the file as data or url.', + ) + } + // Loaded here, not at the top: the module must also load where node:fs is + // missing (an edge worker), and only a path attachment needs it. + const { readFile, realpath } = await import('node:fs/promises') + const { basename, isAbsolute, relative, sep } = await import('node:path') + const refusal = `Cannot read ${path}. It must be a file inside the folders this server may read.` + const real = await realpath(path).catch(() => undefined) + if (real === undefined) throw new Error(refusal) + const roots = await Promise.all(folders.map((folder) => realpath(folder))) + const isInside = roots.some((root) => { + const rest = relative(root, real) + const isOut = rest === '..' || rest.startsWith(`..${sep}`) + return rest !== '' && !isOut && !isAbsolute(rest) + }) + if (!isInside) throw new Error(refusal) + const name = basename(path) + const mimeType = mimeTypeOf(name) + if (mimeType === undefined) { + throw new Error( + `Unknown file type: ${name}. Send the file as data with its mimeType.`, + ) + } + // ponytail: reads the whole file, and the store refuses it after that when + // it is over `media.maxBytes`. Stream it when big local files matter. + return { bytes: await readFile(real), mimeType, name } +} + +/** + * A part that sends `attachment.url` to the model as it is. The server does + * not fetch it: a remote client must not make the server call an internal + * address. The kind comes from `mimeType`, or from the extension of the path. + */ +function urlPart(attachment: { url: string; mimeType?: string }) { + const { url } = attachment + const mimeType = attachment.mimeType ?? mimeTypeOf(new URL(url).pathname) + const kind = mimeType === undefined ? undefined : kindOf(mimeType) + if (mimeType === undefined || kind === undefined) { + throw new Error(`Cannot tell the kind of ${url}. Add its mimeType.`) + } + const part: ContentPart = { + type: kind, + source: { type: 'url', value: url, mimeType }, + } + return part +} + +/** The media records that the settled `operation` published, in order. */ +async function mediaOf(operation: Operation) { + const records: Array = [] + // The operation is settled, so its stream replays its events, then ends. + const events = operation.stream() + for await (const event of events) { + const isMedia = + event.type === EventType.CUSTOM && event.name === HARNESS_EVENTS.media + if (isMedia && isMediaRecord(event.value)) records.push(event.value) + } + return records +} + +/** + * The tool result of `value` with its media. Without media, `value` stays as + * it is. An object result also lists the media in `media`. Each file then + * follows the JSON text as its own content block. + */ +async function withMedia( + session: HarnessSession, + value: unknown, + media: Array, +) { + if (media.length === 0) return value + const listed = media.map((record) => ({ + id: record.id, + kind: record.kind, + name: record.name, + mimeType: record.mimeType, + size: record.size, + uri: mediaUri(record), + })) + const blocks = await Promise.all( + media.map((record) => mediaBlock(session, record)), + ) + const result = toCallToolResult( + isRecord(value) ? { ...value, media: listed } : value, + ) + return { ...result, content: [...result.content, ...blocks] } +} + +/** Image and audio up to 5 MB go inline. Anything else goes as a link. */ +async function mediaBlock(session: HarnessSession, record: MediaRecord) { + const { kind } = record + const isSmall = record.size <= INLINE_MEDIA_MAX_BYTES + if (isSmall && (kind === 'image' || kind === 'audio')) { + const data = toBase64(await session.loadMedia(record.id)) + const inline: ContentBlock = { type: kind, data, mimeType: record.mimeType } + return inline + } + const link: ContentBlock = { + type: 'resource_link', + uri: mediaUri(record), + name: record.name, + mimeType: record.mimeType, + } + return link +} + +function mediaUri(record: MediaRecord) { + const threadId = encodeURIComponent(record.threadId) + return `${MEDIA_URI_PREFIX}${threadId}/${encodeURIComponent(record.id)}` +} + +/** + * The thread and media id of a `harness-media:///` URI, from + * the template variables. `mediaUri` encodes both, so they are decoded here. + */ +function mediaAddress(variables: unknown) { + const threadId = isRecord(variables) ? variables.threadId : undefined + const id = isRecord(variables) ? variables.id : undefined + if (typeof threadId !== 'string' || typeof id !== 'string') { + throw new Error('A media URI is harness-media:///.') + } + return { + threadId: decodeURIComponent(threadId), + id: decodeURIComponent(id), + } +} + +// ponytail: the same as `toBase64` in `@tanstack/ai-harness/src/media.ts`, +// which the package does not export. `btoa`, not `Buffer`, for edge workers. +function toBase64(bytes: Uint8Array) { + const chunk = 0x8000 + let binary = '' + // Chunks keep `fromCharCode` under the engine's argument limit. + for (let offset = 0; offset < bytes.length; offset += chunk) { + binary += String.fromCharCode(...bytes.subarray(offset, offset + chunk)) + } + return btoa(binary) +} + +/** + * How the client answers `interrupt`. `chat()` puts a binding on each + * interrupt. The binding marks a tool approval (answer with `approved`) and a + * client tool (answer with the tool output). Any other interrupt is generic: + * answer with a value that matches its `responseSchema`. + */ +function interruptKind(interrupt: Interrupt) { + const binding = readUnopenedInterruptBinding(interrupt) + switch (binding?.kind) { + case 'tool-approval': + return 'approval' + case 'client-tool-execution': + return 'client-tool' + case 'generic': + case undefined: + return 'generic' + } +} + +function isApproval(interrupt: Interrupt) { + return interruptKind(interrupt) === 'approval' +} + +/** + * The answers of a `resolve` call. The decisions must answer every open + * interrupt, and only those. An approval takes `approved` (or `payload`). + * Every other interrupt takes `payload`. + */ +function answersOf(decisions: Array, interrupts: Array) { + const openIds = interrupts.map(({ id }) => id) + const decidedIds = decisions.map(({ interruptId }) => interruptId) + const missing = openIds.filter((id) => !decidedIds.includes(id)) + if (missing.length > 0) { + throw new Error( + `resolve must answer every open interrupt. Missing: ${missing.join(', ')}.`, + ) + } + const unknown = decidedIds.filter((id) => !openIds.includes(id)) + if (unknown.length > 0) { + throw new Error(`These interrupts are not open: ${unknown.join(', ')}.`) + } + const approvalIds = interrupts.filter(isApproval).map(({ id }) => id) + return decisions.map(({ interruptId, approved, payload }) => { + if (approvalIds.includes(interruptId)) { + const answer = approved ?? payload + if (answer === undefined) { + throw new Error( + `The decision for ${interruptId} needs approved or payload.`, + ) + } + return { interruptId, payload: answer } + } + if (payload === undefined) { + throw new Error( + `The decision for ${interruptId} needs a payload. It is not an approval.`, + ) + } + return { interruptId, payload } + }) +} + +function approvalQuestion(interrupt: Interrupt) { + const args = interrupt.metadata?.input + const shown = args === undefined ? '' : ` Arguments: ${JSON.stringify(args)}.` + const message = interrupt.message ?? 'A tool call needs approval' + return `${message}.${shown} Approve? Answer yes or no.` +} + +function isYes(answer: unknown) { + return typeof answer === 'string' && /^y(es)?$/i.test(answer.trim()) +} + +function turnText(value: unknown) { + return isRecord(value) && typeof value.text === 'string' ? value.text : '' +} + +/** + * Gives each item an MCP tool name, in name order. MCP tool names allow + * letters, digits, and underscores, so `connect:notion` becomes + * `command_connect_notion`. `connect_notion` then gets the same name, so the + * later one gets a number: `command_connect_notion_2`. Each tool of such a + * clash names its original name in the description. + */ +function toolNames( + prefix: string, + items: ReadonlyArray, +) { + const safeName = (name: string) => + `${prefix}_${name.replace(/[^A-Za-z0-9_]/g, '_')}` + const sorted = [...items].sort(byName) + const safeNames = sorted.map((item) => safeName(item.name)) + const used = new Set() + return sorted.map((item) => { + const base = safeName(item.name) + let toolName = base + for (let count = 2; used.has(toolName); count++) { + toolName = `${base}_${count}` + } + used.add(toolName) + const isShared = safeNames.filter((other) => other === base).length > 1 + const clashes = isShared || toolName !== base + const description = clashes + ? `${item.description} (${prefix} ${item.name})` + : item.description + return { item, toolName, description } + }) +} + +// Code unit order, so the order is the same in every locale. +function byName(left: { name: string }, right: { name: string }) { + if (left.name === right.name) return 0 + return left.name < right.name ? -1 : 1 +} + +function withThreadId(schema: JSONSchema | undefined) { + return { + ...schema, + type: 'object', + properties: { ...schema?.properties, threadId: threadIdSchema }, + } +} + +// The tool input without `threadId`, or `undefined` when nothing is left. +function inputOf(args: unknown) { + if (!isRecord(args)) return undefined + const entries = Object.entries(args).filter(([key]) => key !== 'threadId') + return entries.length > 0 ? Object.fromEntries(entries) : undefined +} + +function textArg(args: unknown, key: string) { + const value = isRecord(args) ? args[key] : undefined + if (typeof value !== 'string') throw new Error(`${key} must be a string.`) + return value +} + +function decisionsOf(args: unknown) { + const list = + isRecord(args) && Array.isArray(args.decisions) ? args.decisions : [] + return list.filter(isDecision) +} + +function isDecision(value: unknown): value is Decision { + return ( + isRecord(value) && + typeof value.interruptId === 'string' && + (value.approved === undefined || typeof value.approved === 'boolean') + ) +} + +function isAttachment(value: unknown): value is Attachment { + if (!isRecord(value) || !isOptionalString(value.name)) return false + if (typeof value.path === 'string') return true + if (typeof value.url === 'string') return isOptionalString(value.mimeType) + return typeof value.data === 'string' && typeof value.mimeType === 'string' +} + +function isOptionalString(value: unknown) { + return value === undefined || typeof value === 'string' +} + +// `session.describe()` gives each command input as JSON Schema. +function isJsonSchema(value: unknown): value is JSONSchema { + return isRecord(value) +} + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} diff --git a/packages/ai-mcp/src/tools.ts b/packages/ai-mcp/src/tools.ts index d0365f7a3d..1418c046b6 100644 --- a/packages/ai-mcp/src/tools.ts +++ b/packages/ai-mcp/src/tools.ts @@ -147,6 +147,8 @@ export function mcpContentToTanstack( * @param signal - Stops the wait when the caller aborts * @param inputResponse - The user's answer from an `mcp_input` interrupt * @param requestOptions - The client `requestOptions`, sent with tools/call + * @param askInput - Asks the user for each elicitation round on spec 2026. + * The harness sets it. Without it, a call answers one round. */ export async function callMcpTool( client: Client, @@ -156,6 +158,7 @@ export async function callMcpTool( signal?: AbortSignal, inputResponse?: ToolInputResponse, requestOptions?: MCPClientOptions['requestOptions'], + askInput?: AskInput, ) { signal?.throwIfAborted() const isModern = client.getProtocolEra() === 'modern' @@ -194,16 +197,44 @@ export async function callMcpTool( // ponytail: one input round per call. The next resume starts with no // requestState, so a second pause would ask round 1 again forever. // Carry requestState through the interrupt if servers need more rounds. - if (isInputRequiredResult(raw)) { + if (isInputRequiredResult(raw) && askInput === undefined) { throw new Error( `The MCP tool "${mcpName}" asked for input a second time. ` + 'This client answers one input request per tool call.', ) } } + // The harness asks the user for each elicitation round, up to a cap. + for ( + let round = 1; + isModern && askInput && isInputRequiredResult(raw); + round++ + ) { + const entry = firstInputRequest(raw.inputRequests) + if (entry?.method !== 'elicitation/create') break + if (round > maxInputRounds) { + throw new Error( + `The MCP tool "${mcpName}" asked for input more than ` + + `${maxInputRounds} times in one call.`, + ) + } + const answer = await askInput(isRecord(entry.params) ? entry.params : entry) + raw = await rawRequest( + client, + 'tools/call', + { ...params, ...retryParams(raw, answer) }, + signal, + requestOptions, + ) + } return finishToolCall(client, mcpName, raw, signal) } +/** Asks the user for one MCP elicitation request body. */ +export type AskInput = (request: unknown) => Promise + +const maxInputRounds = 5 + // Answers only the first input request. That is the one the interrupt shows. function retryParams( result: { inputRequests?: unknown; requestState?: string }, @@ -684,7 +715,11 @@ export function makeMcpExecute( ) { return async ( args: unknown, - ctx?: { abortSignal?: AbortSignal; inputResponse?: ToolInputResponse }, + ctx?: { + abortSignal?: AbortSignal + inputResponse?: ToolInputResponse + askInput?: AskInput + }, ) => { const result = await callMcpTool( client, @@ -694,6 +729,7 @@ export function makeMcpExecute( ctx?.abortSignal, ctx?.inputResponse, requestOptions, + ctx?.askInput, ) if (result.isError) { const text = Array.isArray(result.content) diff --git a/packages/ai-mcp/tests/connector-edges.test.ts b/packages/ai-mcp/tests/connector-edges.test.ts new file mode 100644 index 0000000000..690343a74a --- /dev/null +++ b/packages/ai-mcp/tests/connector-edges.test.ts @@ -0,0 +1,550 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' +import { EventType } from '@tanstack/ai' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { + HARNESS_EVENTS, + createHarnessHost, + defineHarness, +} from '@tanstack/ai-harness' +import { mcpConnector } from '../src/connector' +import { approveSignIns, recorder } from './connector-helpers' +import { startProtectedServer } from './protected-server' +import type { Credential } from '@tanstack/ai-persistence' +import type { + HarnessHost, + HarnessSession, + SessionEvent, +} from '@tanstack/ai-harness' + +const scope = { threadId: 't', userId: 'user-1' } +const cleanups: Array<() => unknown> = [] +afterEach(async () => { + for (const cleanup of cleanups.splice(0).reverse()) await cleanup() +}) + +async function setup( + credential?: Credential, + serverOptions?: Parameters[0], +) { + const server = await startProtectedServer(serverOptions) + cleanups.push(() => server.close()) + const persistence = memoryPersistence() + if (credential) + await persistence.stores.credentials.set(scope, 'demo', credential) + const host = createHarnessHost({ persistence }) + cleanups.push(() => host.close()) + return { server, persistence, host } +} + +/** A harness with only the `demo` connector, with default options. */ +function connectorHarness(url: string, model: ReturnType) { + return defineHarness({ + name: 'test/connector', + adapter: model.adapter, + plugins: () => [mcpConnector({ id: 'demo', label: 'Demo', url })], + }) +} + +/** Open a session of {@link connectorHarness} for `user-1`. */ +function openDemo( + host: HarnessHost, + url: string, + model = recorder(), + threadId = 't', +) { + return host.open(connectorHarness(url, model), { + threadId, + principal: { id: 'user-1' }, + }) +} + +/** Run one prompt and return every event the session published for it. */ +async function promptAndCollect(session: HarnessSession, text: string) { + const seen: Array = [] + const controller = new AbortController() + const reading = (async () => { + for await (const entry of session.events({ + from: '0', + signal: controller.signal, + })) + seen.push(entry) + })() + await session.prompt(text) + controller.abort() + await reading + return seen +} + +/** The values of the custom events named `name`, in order. */ +function customValues(seen: Array, name: string) { + return seen.flatMap(({ event }) => + event.type === EventType.CUSTOM && event.name === name ? [event.value] : [], + ) +} + +describe('mcpConnector with a saved sign-in', () => { + it('uses the saved token and client after a restart, without /connect', async () => { + const { server, host } = await setup({ + type: 'oauth', + accessToken: 'access-1', + refreshToken: 'refresh-1', + expiresAt: Date.now() + 3_600_000, + client: { + clientId: 'client-1', + clientSecret: 'secret-1', + redirectUri: 'http://127.0.0.1:1/callback', + }, + }) + const model = recorder('demo_echo') + const session = await openDemo(host, server.url, model) + await session.prompt('echo') + expect(model.toolNames(0)).toEqual(['demo_echo']) + expect(JSON.stringify(model.calls[1]?.messages)).toContain('echo: hi') + // The status prompt is empty once connected. + expect(JSON.stringify(model.calls[0]?.systemPrompts ?? [])).not.toContain( + 'is not connected', + ) + expect(server.seen.registrations).toHaveLength(0) + }) + + it.each([ + { + redirect: 'the saved client redirect URI', + client: { clientId: 'client-1', redirectUri: 'http://127.0.0.1:1/cb' }, + redirectUri: 'http://127.0.0.1:1/cb', + }, + { + redirect: 'the fallback redirect URI', + client: { clientId: 'client-1' }, + redirectUri: 'http://127.0.0.1/callback', + }, + ])( + 'refreshes an expired token with $redirect and keeps the old refresh token and client', + async ({ client, redirectUri }) => { + const { server, persistence, host } = await setup({ + type: 'oauth', + accessToken: 'stale', + refreshToken: 'refresh-1', + expiresAt: Date.now() - 1000, + client, + }) + const model = recorder('demo_echo') + const session = await openDemo(host, server.url, model) + await session.prompt('echo') + expect(model.toolNames(0)).toEqual(['demo_echo']) + expect( + server.seen.tokenRequests.map((form) => form.get('grant_type')), + ).toEqual(['refresh_token']) + expect(server.seen.registrations).toHaveLength(0) + // A sign-in saved without an issuer gets the issuer of this server. + expect(await persistence.stores.credentials.get(scope, 'demo')).toEqual({ + type: 'oauth', + accessToken: 'access-2', + refreshToken: 'refresh-1', + client: { clientId: 'client-1', redirectUri, issuer: server.issuer }, + }) + }, + ) + + it.each([ + { + failure: 'the refresh token is revoked', + credential: { + type: 'oauth', + accessToken: 'stale', + refreshToken: 'revoked', + }, + revoke: [], + // The server refused the refresh token, so it is deleted. + kept: null, + }, + { + failure: 'the server revokes a token that has no refresh token', + credential: { type: 'oauth', accessToken: 'access-1' }, + revoke: ['access-1'], + kept: 'access-1', + }, + ] satisfies Array<{ + failure: string + credential: Credential + revoke: Array + kept: string | null + }>)( + 'drops the tools and asks the user to run /connect when $failure', + async ({ credential, revoke, kept }) => { + const { server, persistence, host } = await setup(credential) + for (const token of revoke) server.revoke(token) + const model = recorder() + const session = await openDemo(host, server.url, model) + const seen = await promptAndCollect(session, 'hi') + expect(model.toolNames(0)).toEqual([]) + expect(customValues(seen, 'harness.plugin.warning')).toEqual([ + { + plugin: 'connector/demo', + message: 'Sign in to demo first. Run /connect demo.', + }, + ]) + // The notice has no URL: outside /connect, no browser sign-in starts. + expect(customValues(seen, HARNESS_EVENTS.authRequired)).toEqual([ + { connector: 'demo' }, + ]) + // The same turn tells the model that the user must run /connect. + expect(JSON.stringify(model.calls[0]?.systemPrompts)).toContain( + 'run /connect demo', + ) + const saved = await persistence.stores.credentials.get(scope, 'demo') + expect(saved?.type === 'oauth' ? saved.accessToken : null).toBe(kept) + }, + ) + + it('keeps the sign-in and a connection error when the network is down', async () => { + const { server, persistence, host } = await setup({ + type: 'oauth', + accessToken: 'access-1', + }) + const model = recorder() + const session = await host.open( + defineHarness({ + name: 'test/offline', + adapter: model.adapter, + plugins: () => [ + mcpConnector({ + id: 'demo', + label: 'Demo', + url: server.url, + // Each request fails the way `fetch` fails without a network. + fetch: async () => { + throw new TypeError('fetch failed') + }, + }), + ], + }), + { threadId: 't', principal: { id: 'user-1' } }, + ) + const seen = await promptAndCollect(session, 'hi') + expect(model.toolNames(0)).toEqual([]) + expect(customValues(seen, 'harness.plugin.warning')).toEqual([ + { plugin: 'connector/demo', message: 'Failed to connect to MCP server' }, + ]) + expect(customValues(seen, HARNESS_EVENTS.authRequired)).toEqual([]) + expect(JSON.stringify(model.calls[0]?.systemPrompts ?? [])).not.toContain( + 'is not connected', + ) + expect(await persistence.stores.credentials.get(scope, 'demo')).toEqual({ + type: 'oauth', + accessToken: 'access-1', + }) + }) +}) + +describe('mcpConnector /connect', () => { + it('asks for scopes, shows the client name, and uses the prefix and approval rule', async () => { + const { server, persistence, host } = await setup() + const model = recorder() + const session = await host.open( + defineHarness({ + name: 'test/options', + adapter: model.adapter, + plugins: () => [ + mcpConnector({ + id: 'demo', + label: 'Demo', + url: server.url, + prefix: 'd', + scopes: ['read'], + clientName: 'Acme Agent', + needsApproval: () => true, + fetch: (input, init) => fetch(input, init), + }), + ], + }), + { threadId: 't', principal: { id: 'user-1' } }, + ) + const browser = approveSignIns(session, 'code-1') + cleanups.push(browser.stop) + + expect(await session.command('connect:demo')).toBe('Connected to Demo.') + expect(browser.authorizationUrls[0]?.searchParams.get('scope')).toBe('read') + expect(server.seen.registrations[0]).toMatchObject({ + client_name: 'Acme Agent', + scope: 'read', + }) + expect( + await persistence.stores.credentials.get(scope, 'demo'), + ).toMatchObject({ + scopes: ['read', 'write'], + client: { + clientId: 'client-1', + redirectUri: expect.stringContaining('127.0.0.1'), + }, + }) + + await session.prompt('hi') + expect(model.toolNames(0)).toEqual(['d_echo']) + expect(model.calls[0]?.tools?.[0]?.needsApproval).toBe(true) + }) + + it.each([ + { before: 'no earlier sign-in', earlier: undefined }, + { + before: 'an earlier sign-in', + // Maybe another account, with a larger scope. None of it may stay. + earlier: { + type: 'oauth', + accessToken: 'old', + refreshToken: 'refresh-0', + expiresAt: Date.now() + 3_600_000, + scopes: ['admin'], + client: { clientId: 'client-0', redirectUri: 'http://127.0.0.1:1/cb' }, + }, + }, + ] satisfies Array<{ before: string; earlier: Credential | undefined }>)( + 'saves only the new sign-in, which gives no refresh token, after $before', + async ({ earlier }) => { + const { persistence, host, server } = await setup(earlier) + const session = await openDemo(host, server.url) + const browser = approveSignIns(session, 'code-2') + cleanups.push(browser.stop) + + expect(await session.command('connect:demo')).toBe('Connected to Demo.') + expect(await persistence.stores.credentials.get(scope, 'demo')).toEqual({ + type: 'oauth', + accessToken: 'access-1', + client: { + clientId: 'client-1', + redirectUri: expect.stringMatching(/^http:\/\/127\.0\.0\.1:\d+\//), + issuer: server.issuer, + }, + }) + }, + ) + + it('saves the issuer, and signs in and refreshes without SEP-2352 warnings', async () => { + // The SDK prints its SEP-2352 warnings with console.warn. + const warn = vi.spyOn(console, 'warn') + cleanups.push(() => warn.mockRestore()) + const { server, persistence, host } = await setup() + const model = recorder() + const session = await openDemo(host, server.url, model) + const browser = approveSignIns(session, 'code-1') + cleanups.push(browser.stop) + expect(await session.command('connect:demo')).toBe('Connected to Demo.') + + // The server stops taking the first token, so the next turn refreshes it. + server.revoke('access-1') + await session.prompt('hi') + expect(model.toolNames(0)).toEqual(['demo_echo']) + expect( + server.seen.tokenRequests.map((form) => form.get('grant_type')), + ).toEqual(['authorization_code', 'refresh_token']) + expect( + await persistence.stores.credentials.get(scope, 'demo'), + ).toMatchObject({ + accessToken: 'access-2', + refreshToken: 'refresh-1', + client: { clientId: 'client-1', issuer: server.issuer }, + }) + const warnings = warn.mock.calls.map(([message]) => String(message)) + expect(warnings.filter((message) => message.includes('SEP-2352'))).toEqual( + [], + ) + }) + + it('passes the iss of the callback on, for a server that sends one (RFC 9207)', async () => { + const { server, persistence, host } = await setup(undefined, { + issParameter: true, + }) + const session = await openDemo(host, server.url, recorder()) + const browser = approveSignIns(session, 'code-1', server.issuer) + cleanups.push(browser.stop) + expect(await session.command('connect:demo')).toBe('Connected to Demo.') + expect( + await persistence.stores.credentials.get(scope, 'demo'), + ).toMatchObject({ accessToken: 'access-1' }) + }) + + it.each([ + { before: 'no sign-in', earlier: undefined, tools: [], asks: true }, + { + before: 'the earlier sign-in', + earlier: { + type: 'oauth', + accessToken: 'access-1', + refreshToken: 'refresh-1', + client: { clientId: 'client-1', redirectUri: 'http://127.0.0.1:1/cb' }, + }, + tools: ['demo_echo'], + asks: false, + }, + ] satisfies Array<{ + before: string + earlier: Credential | undefined + tools: Array + asks: boolean + }>)( + 'keeps $before when a new sign-in fails', + async ({ earlier, tools, asks }) => { + const { persistence, host, server } = await setup(earlier) + const model = recorder() + const session = await openDemo(host, server.url, model) + const browser = approveSignIns(session, 'wrong-code') + cleanups.push(browser.stop) + + await expect(session.command('connect:demo')).rejects.toMatchObject({ + code: 'invalid_grant', + }) + expect(await persistence.stores.credentials.get(scope, 'demo')).toEqual( + earlier ?? null, + ) + await session.prompt('hi') + expect(model.toolNames(0)).toEqual(tools) + expect( + JSON.stringify(model.calls[0]?.systemPrompts ?? []).includes( + 'run /connect demo', + ), + ).toBe(asks) + }, + ) +}) + +describe('mcpConnector stored credential shapes', () => { + it('works with a bare token, reuses the tools, and signs out before any tool call', async () => { + const { server, persistence, host } = await setup({ + type: 'oauth', + accessToken: 'access-1', + }) + const model = recorder() + const session = await openDemo(host, server.url, model) + await session.prompt('one') + await session.prompt('two') + expect(model.toolNames(0)).toEqual(['demo_echo']) + expect(model.toolNames(1)).toEqual(['demo_echo']) + + const fresh = await openDemo(host, server.url, recorder(), 't2') + expect(await fresh.command('disconnect:demo')).toBe( + 'Disconnected from Demo.', + ) + expect(await persistence.stores.credentials.get(scope, 'demo')).toBeNull() + }) + + it('refreshes a token saved without a client, and keeps a client secret', async () => { + const bare = await setup({ + type: 'oauth', + accessToken: 'stale', + refreshToken: 'refresh-1', + }) + const model = recorder() + const session = await openDemo(bare.host, bare.server.url, model) + await session.prompt('hi') + expect(model.toolNames(0)).toEqual(['demo_echo']) + const saved = await bare.persistence.stores.credentials.get(scope, 'demo') + expect(saved).toMatchObject({ + accessToken: 'access-2', + refreshToken: 'refresh-1', + }) + // No saved client: the SDK registered one before it refreshed. + expect(bare.server.seen.registrations).toHaveLength(1) + expect(saved).toMatchObject({ client: { clientId: 'client-1' } }) + + const withSecret = await setup({ + type: 'oauth', + accessToken: 'stale', + refreshToken: 'refresh-1', + client: { clientId: 'client-1', clientSecret: 'secret-1' }, + }) + const other = await openDemo(withSecret.host, withSecret.server.url) + await other.prompt('hi') + expect( + await withSecret.persistence.stores.credentials.get(scope, 'demo'), + ).toMatchObject({ + accessToken: 'access-2', + client: { clientId: 'client-1', clientSecret: 'secret-1' }, + }) + }) + + it('treats a saved API key as no sign-in for the MCP server', async () => { + const { server, host } = await setup({ + type: 'api_key', + value: 'not-oauth', + }) + const model = recorder() + const session = await openDemo(host, server.url, model) + await session.prompt('hi') + expect(model.toolNames(0)).toEqual([]) + }) +}) + +describe('mcpConnector client options', () => { + it('names the tools with toolName', async () => { + const { server, host } = await setup({ + type: 'oauth', + accessToken: 'access-1', + }) + const model = recorder() + const session = await host.open( + defineHarness({ + name: 'test/tool-name', + adapter: model.adapter, + plugins: () => [ + mcpConnector({ + id: 'demo', + label: 'Demo', + url: server.url, + toolName: (tool) => `mcp__demo__${tool.name}`, + }), + ], + }), + { threadId: 't', principal: { id: 'user-1' } }, + ) + await session.prompt('hi') + expect(model.toolNames(0)).toEqual(['mcp__demo__echo']) + }) + + it('sends requestOptions with each MCP request', async () => { + const { server, host } = await setup({ + type: 'oauth', + accessToken: 'access-1', + }) + const model = recorder() + const held: Array> = [] + const session = await host.open( + defineHarness({ + name: 'test/request-options', + adapter: model.adapter, + plugins: () => [ + mcpConnector({ + id: 'demo', + label: 'Demo', + url: server.url, + requestOptions: { timeout: 50 }, + // Hold the tools/list answer for 500 ms, so only the 50 ms + // timeout can end that request. + fetch: (input, init) => { + const response = fetch(input, init) + const body = init?.body + if (typeof body !== 'string' || !body.includes('"tools/list"')) { + return response + } + const late = new Promise((resolve) => { + setTimeout(resolve, 500) + }).then(() => response) + held.push(late) + return late + }, + }), + ], + }), + { threadId: 't', principal: { id: 'user-1' } }, + ) + const seen = await promptAndCollect(session, 'hi') + // Let the held answer finish before the server closes. + await Promise.allSettled(held) + expect(model.toolNames(0)).toEqual([]) + expect(customValues(seen, 'harness.plugin.warning')).toEqual([ + { + plugin: 'connector/demo', + message: expect.stringContaining('timed out'), + }, + ]) + }) +}) diff --git a/packages/ai-mcp/tests/connector-helpers.ts b/packages/ai-mcp/tests/connector-helpers.ts new file mode 100644 index 0000000000..d7ce558e12 --- /dev/null +++ b/packages/ai-mcp/tests/connector-helpers.ts @@ -0,0 +1,246 @@ +import { z } from 'zod' +import { + McpServer, + WebStandardStreamableHTTPServerTransport, + createMcpHandler, + inputRequired, +} from '@modelcontextprotocol/server' +import { EventType } from '@tanstack/ai' +import { HARNESS_EVENTS } from '@tanstack/ai-harness' +import type { + ElicitRequestFormParams, + ElicitRequestURLParams, + InputRequest, +} from '@modelcontextprotocol/server' +import type { AnyTextAdapter, StreamChunk, TextOptions } from '@tanstack/ai' +import type { HarnessSession } from '@tanstack/ai-harness' + +/** A text adapter for the connector tests. Only `chatStream` changes. */ +export function mockTextAdapter(chatStream: AnyTextAdapter['chatStream']) { + const adapter: AnyTextAdapter = { + kind: 'text', + name: 'mock', + model: 'mock', + '~types': { + providerOptions: {}, + inputModalities: ['text'], + messageMetadataByModality: { + text: undefined, + image: undefined, + audio: undefined, + video: undefined, + document: undefined, + }, + toolCapabilities: [], + toolCallMetadata: undefined, + systemPromptMetadata: undefined, + }, + structuredOutput: async () => ({ data: {}, rawText: '{}' }), + chatStream, + } + return adapter +} + +/** A model that records the calls it gets, calls `tool` once, then stops. */ +export function recorder(tool?: string) { + const calls: Array = [] + const now = () => Date.now() + const adapter = mockTextAdapter((options) => { + calls.push(options) + const first = calls.length === 1 && tool !== undefined + return (async function* (): AsyncGenerator { + yield { + type: EventType.RUN_STARTED, + runId: 'r', + threadId: 't', + timestamp: now(), + } + if (first) { + yield { + type: EventType.TOOL_CALL_START, + toolCallId: 'c1', + toolCallName: tool, + timestamp: now(), + } + yield { + type: EventType.TOOL_CALL_ARGS, + toolCallId: 'c1', + delta: '{"text":"hi"}', + timestamp: now(), + } + yield { + type: EventType.TOOL_CALL_END, + toolCallId: 'c1', + timestamp: now(), + } + } + yield { + type: EventType.RUN_FINISHED, + runId: 'r', + threadId: 't', + timestamp: now(), + metadata: { + tanstack: { finishReason: first ? 'tool_calls' : 'stop' }, + }, + } + })() + }) + const toolNames = (index: number) => + (calls[index]?.tools ?? []).map((entry) => entry.name) + return { adapter, calls, toolNames } +} + +/** The sign-in URL of a `harness.auth_required` event, or `undefined`. */ +export function authorizationUrlOf(event: StreamChunk) { + if ( + event.type !== EventType.CUSTOM || + event.name !== HARNESS_EVENTS.authRequired + ) + return undefined + const value: unknown = event.value + if (typeof value !== 'object' || value === null || !('url' in value)) + return undefined + return typeof value.url === 'string' ? new URL(value.url) : undefined +} + +/** + * Act as the user's browser for `/connect`: approve each sign-in the session + * asks for by calling its loopback redirect with `code` and the same `state`. + */ +export function approveSignIns( + session: HarnessSession, + code: string, + /** The `iss` the fake server sends back with the code (RFC 9207). */ + iss?: string, +) { + const authorizationUrls: Array = [] + const controller = new AbortController() + void (async () => { + for await (const entry of session.events({ signal: controller.signal })) { + const authorizationUrl = authorizationUrlOf(entry.event) + if (!authorizationUrl) continue + authorizationUrls.push(authorizationUrl) + const redirect = new URL( + authorizationUrl.searchParams.get('redirect_uri') ?? '', + ) + redirect.searchParams.set('code', code) + if (iss) redirect.searchParams.set('iss', iss) + redirect.searchParams.set( + 'state', + authorizationUrl.searchParams.get('state') ?? '', + ) + await fetch(redirect) + } + })() + return { authorizationUrls, stop: () => controller.abort() } +} + +/** + * A spec 2026 server with one `book` tool. The first call asks for `input`. + * The retry records the answer in `answers` and ends the call. + */ +export function elicitServer(input: InputRequest) { + const answers: Array = [] + const handler = createMcpHandler( + () => { + const server = new McpServer({ name: 'trips', version: '1.0.0' }) + server.registerTool( + 'book', + { + description: 'Book a trip', + inputSchema: z.object({ text: z.string() }), + }, + (_args, ctx) => { + const responses = ctx.mcpReq.inputResponses + if (responses === undefined) { + return inputRequired({ inputRequests: { trip: input } }) + } + answers.push(responses.trip) + return { content: [{ type: 'text', text: 'booked' }] } + }, + ) + return server + }, + { legacy: 'reject', keepAliveMs: 0 }, + ) + return { answers, fetch: (request: Request) => handler.fetch(request) } +} + +/** The form the `book` tool of {@link elicitServer} asks for. */ +export const tripForm = { + message: 'Which city?', + requestedSchema: { + type: 'object' as const, + properties: { city: { type: 'string' as const } }, + required: ['city'], + }, +} + +/** + * A spec 2025 server with one `book` tool. The tool sends each of `asks` to + * the client as an `elicitation/create` request, and records each answer in + * `answers`. + */ +export async function legacyElicitServer( + asks: Array, +) { + const answers: Array = [] + const server = new McpServer({ name: 'trips', version: '1.0.0' }) + server.registerTool( + 'book', + { + description: 'Book a trip', + inputSchema: z.object({ text: z.string() }), + }, + async (_args, ctx) => { + for (const ask of asks) answers.push(await ctx.mcpReq.elicitInput(ask)) + return { content: [{ type: 'text', text: 'booked' }] } + }, + ) + const transport = new WebStandardStreamableHTTPServerTransport({ + sessionIdGenerator: () => crypto.randomUUID(), + }) + await server.connect(transport) + return { + answers, + fetch: (request: Request) => transport.handleRequest(request), + } +} + +/** + * A spec 2026 server with one `book` tool that asks for each of `inputs` in + * turn, one round each. `requestState` carries the round. The retries record + * the answers in `answers`. + */ +export function roundsServer(inputs: Array) { + const answers: Array = [] + const handler = createMcpHandler( + () => { + const server = new McpServer({ name: 'trips', version: '1.0.0' }) + server.registerTool( + 'book', + { + description: 'Book a trip', + inputSchema: z.object({ text: z.string() }), + }, + (_args, ctx) => { + const state: unknown = ctx.mcpReq.requestState() + const round = typeof state === 'string' ? Number(state) : 0 + const responses = ctx.mcpReq.inputResponses + if (responses !== undefined) answers.push(responses[`r${round - 1}`]) + const input = inputs[round] + if (input === undefined) { + return { content: [{ type: 'text', text: 'booked' }] } + } + return inputRequired({ + inputRequests: { [`r${round}`]: input }, + requestState: String(round + 1), + }) + }, + ) + return server + }, + { legacy: 'reject', keepAliveMs: 0 }, + ) + return { answers, fetch: (request: Request) => handler.fetch(request) } +} diff --git a/packages/ai-mcp/tests/connector.test.ts b/packages/ai-mcp/tests/connector.test.ts new file mode 100644 index 0000000000..93ddcf4d63 --- /dev/null +++ b/packages/ai-mcp/tests/connector.test.ts @@ -0,0 +1,395 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' +import { inputRequired } from '@modelcontextprotocol/server' +import { EventType } from '@tanstack/ai' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { mcpConnector } from '../src/connector' +import { + approveSignIns, + elicitServer, + legacyElicitServer, + mockTextAdapter, + recorder, + tripForm, +} from './connector-helpers' +import { startProtectedServer } from './protected-server' +import type { StreamChunk } from '@tanstack/ai' + +/** A model that calls `tool` once, then answers with the tool result it saw. */ +function modelCalling(tool: string, args: { text: string }) { + let call = 0 + const now = () => Date.now() + return mockTextAdapter((options) => + (async function* (): AsyncGenerator { + call += 1 + yield { + type: EventType.RUN_STARTED, + runId: 'r', + threadId: 't', + timestamp: now(), + } + if (call === 1) { + yield { + type: EventType.TOOL_CALL_START, + toolCallId: 'c1', + toolCallName: tool, + timestamp: now(), + } + yield { + type: EventType.TOOL_CALL_ARGS, + toolCallId: 'c1', + delta: JSON.stringify(args), + timestamp: now(), + } + yield { + type: EventType.TOOL_CALL_END, + toolCallId: 'c1', + timestamp: now(), + } + yield { + type: EventType.RUN_FINISHED, + runId: 'r', + threadId: 't', + timestamp: now(), + metadata: { tanstack: { finishReason: 'tool_calls' } }, + } + return + } + const last = options.messages.at(-1) + const text = `saw: ${typeof last?.content === 'string' ? last.content : JSON.stringify(last?.content)}` + yield { + type: EventType.TEXT_MESSAGE_START, + messageId: `m${call}`, + role: 'assistant', + timestamp: now(), + } + yield { + type: EventType.TEXT_MESSAGE_CONTENT, + messageId: `m${call}`, + delta: text, + timestamp: now(), + } + yield { + type: EventType.TEXT_MESSAGE_END, + messageId: `m${call}`, + timestamp: now(), + } + yield { + type: EventType.RUN_FINISHED, + runId: 'r', + threadId: 't', + timestamp: now(), + metadata: { tanstack: { finishReason: 'stop' } }, + } + })(), + ) +} + +const cleanups: Array<() => unknown> = [] +afterEach(async () => { + for (const cleanup of cleanups.splice(0).reverse()) await cleanup() +}) + +describe('mcpConnector', () => { + it('signs in with OAuth discovery, registration, and PKCE, then gives the model the server tools', async () => { + const protectedServer = await startProtectedServer() + cleanups.push(() => protectedServer.close()) + const persistence = memoryPersistence() + const host = createHarnessHost({ persistence }) + cleanups.push(() => host.close()) + const harness = defineHarness({ + name: 'test/mcp-connector', + adapter: modelCalling('demo_echo', { text: 'hello' }), + plugins: () => [ + mcpConnector({ id: 'demo', label: 'Demo', url: protectedServer.url }), + ], + }) + const session = await host.open(harness, { + threadId: 't', + principal: { id: 'user-1' }, + }) + const browser = approveSignIns(session, 'code-1') + cleanups.push(browser.stop) + + await expect(session.command('connect:demo')).resolves.toBe( + 'Connected to Demo.', + ) + const authorizationUrl = browser.authorizationUrls.at(-1) + expect(authorizationUrl?.searchParams.get('client_id')).toBe('client-1') + expect(authorizationUrl?.searchParams.get('code_challenge_method')).toBe( + 'S256', + ) + expect( + new URL(authorizationUrl?.searchParams.get('redirect_uri') ?? '') + .hostname, + ).toBe('127.0.0.1') + // No `scopes` option: the sign-in asks for no scope. + expect(authorizationUrl?.searchParams.has('scope')).toBe(false) + expect(protectedServer.seen.registrations).toHaveLength(1) + expect(protectedServer.seen.registrations[0]).toMatchObject({ + client_name: 'TanStack AI Harness', + }) + expect(protectedServer.seen.registrations[0]).not.toHaveProperty('scope') + expect( + protectedServer.seen.tokenRequests[0]?.get('code_verifier'), + ).toBeTruthy() + + const saved = await persistence.stores.credentials.get( + { threadId: 't', userId: 'user-1' }, + 'demo', + ) + expect(saved).toMatchObject({ + type: 'oauth', + accessToken: 'access-1', + refreshToken: 'refresh-1', + client: { clientId: 'client-1' }, + }) + + // The echo tool is read-only, so the default rule runs it without approval. + const turn = await session.prompt('echo hello') + expect(turn.text).toContain('echo: hello') + // The model never sees the token. + expect( + JSON.stringify(await persistence.stores.messages.loadThread('t')), + ).not.toContain('access-1') + + await session.command('disconnect:demo') + expect( + await persistence.stores.credentials.get( + { threadId: 't', userId: 'user-1' }, + 'demo', + ), + ).toBeNull() + }) + + it('keeps one sign-in per sender in a shared thread', async () => { + const protectedServer = await startProtectedServer() + cleanups.push(() => protectedServer.close()) + const model = recorder() + const persistence = memoryPersistence() + const host = createHarnessHost({ persistence }) + cleanups.push(() => host.close()) + const alice = { id: 'alice' } + const bob = { id: 'bob' } + const session = await host.open( + defineHarness({ + name: 'test/mcp-connector-shared', + adapter: model.adapter, + plugins: () => [ + mcpConnector({ id: 'demo', label: 'Demo', url: protectedServer.url }), + ], + }), + { threadId: 't', principal: alice }, + ) + const browser = approveSignIns(session, 'code-1') + cleanups.push(browser.stop) + const savedFor = (userId: string) => + persistence.stores.credentials.get({ threadId: 't', userId }, 'demo') + + await session.command('connect:demo', undefined, { principal: bob }) + expect(await savedFor('bob')).not.toBeNull() + expect(await savedFor('alice')).toBeNull() + + await session.prompt('hi', { principal: bob }) + await session.prompt('hi', { principal: alice }) + await session.prompt('hi', { principal: bob }) + // Only bob signed in, so only his turns get the server tools. + expect(model.toolNames(0)).toEqual(['demo_echo']) + expect(model.toolNames(1)).toEqual([]) + expect(JSON.stringify(model.calls[1]?.systemPrompts)).toContain( + 'run /connect demo', + ) + expect(model.toolNames(2)).toEqual(['demo_echo']) + + // Alice signs in for her own turns. + await session.command('connect:demo', undefined, { principal: alice }) + await session.prompt('hi', { principal: alice }) + expect(await savedFor('alice')).not.toBeNull() + expect(model.toolNames(3)).toEqual(['demo_echo']) + }) + + it('keeps colliding sender names separate in one shared thread', async () => { + const protectedServer = await startProtectedServer() + cleanups.push(() => protectedServer.close()) + const first = { tenantId: 'a', id: 'b:c' } + const second = { tenantId: 'a:b', id: 'c' } + const persistence = memoryPersistence() + for (const [principal, accessToken] of [ + [first, 'first-token'], + [second, 'second-token'], + ] as const) { + await persistence.stores.credentials.set( + { threadId: 't', tenantId: principal.tenantId, userId: principal.id }, + 'demo', + { + type: 'oauth', + accessToken, + client: { + clientId: 'client-1', + redirectUri: 'http://127.0.0.1/callback', + issuer: protectedServer.issuer, + }, + }, + ) + } + const initialized: Array = [] + const toolLists: Array = [] + const fetchForSender: typeof fetch = async (input, init) => { + const request = new Request(input, init) + const token = request.headers + .get('authorization') + ?.replace(/^Bearer /, '') + const raw = request.method === 'POST' ? await request.clone().text() : '' + const body: unknown = raw ? JSON.parse(raw) : undefined + const method = + typeof body === 'object' && body !== null && 'method' in body + ? body.method + : undefined + if (method === 'initialize') initialized.push(token ?? '') + if (method === 'tools/list') toolLists.push(token ?? '') + if (token) request.headers.set('authorization', 'Bearer access-1') + const response = await fetch(request) + if (method !== 'tools/list' || !response.ok) return response + const result: unknown = await response.json() + if ( + typeof result !== 'object' || + result === null || + !('result' in result) || + typeof result.result !== 'object' || + result.result === null || + !('tools' in result.result) || + !Array.isArray(result.result.tools) + ) { + throw new Error('The MCP server did not return tools.') + } + for (const tool of result.result.tools) { + if (typeof tool !== 'object' || tool === null || !('name' in tool)) { + throw new Error('The MCP server returned an invalid tool.') + } + tool.name = token === 'first-token' ? 'first_echo' : 'second_echo' + } + return new Response(JSON.stringify(result), { + status: response.status, + headers: response.headers, + }) + } + const model = recorder() + const host = createHarnessHost({ persistence }) + cleanups.push(() => host.close()) + const session = await host.open( + defineHarness({ + name: 'test/mcp-connector-colliding-senders', + adapter: model.adapter, + plugins: () => [ + mcpConnector({ + id: 'demo', + label: 'Demo', + url: protectedServer.url, + fetch: fetchForSender, + }), + ], + }), + { threadId: 't', principal: first }, + ) + await session.prompt('first', { principal: first }) + await session.prompt('second', { principal: second }) + await session.prompt('first again', { principal: first }) + expect(model.toolNames(0)).toEqual(['demo_first_echo']) + expect(model.toolNames(1)).toEqual(['demo_second_echo']) + expect(model.toolNames(2)).toEqual(['demo_first_echo']) + expect(initialized).toEqual(['first-token', 'second-token']) + // tools() walks tools/list two times on a spec 2025 server: a raw walk, + // then listTools() for the SDK's output checks. Each walk of a sender + // uses that sender's token. + expect(toolLists).toEqual([ + 'first-token', + 'first-token', + 'second-token', + 'second-token', + ]) + }) + + it('tells the model to ask for /connect before sign-in', async () => { + const model = recorder() + const host = createHarnessHost({ persistence: memoryPersistence() }) + cleanups.push(() => host.close()) + const session = await host.open( + defineHarness({ + name: 'test/mcp-connector-off', + adapter: model.adapter, + plugins: () => [ + mcpConnector({ + id: 'demo', + label: 'Demo', + url: 'http://127.0.0.1:9/mcp', + }), + ], + }), + { threadId: 't' }, + ) + await session.prompt('hi') + expect(JSON.stringify(model.calls[0]?.systemPrompts)).toContain( + 'run /connect demo', + ) + expect(model.toolNames(0)).toEqual([]) + expect(session.commands().map((command) => command.name)).toEqual([ + 'connect:demo', + 'disconnect:demo', + ]) + }) + + it.each([ + ['spec 2026', async () => elicitServer(inputRequired.elicit(tripForm))], + [ + 'spec 2025', + async () => legacyElicitServer([{ mode: 'form', ...tripForm }]), + ], + ])( + 'asks a form elicitation of a %s server as a session question and sends the answer', + async (_name, start) => { + const trips = await start() + const persistence = memoryPersistence() + // A saved sign-in, so the test skips the OAuth steps. + await persistence.stores.credentials.set( + { threadId: 't', userId: 'user-1' }, + 'trips', + { type: 'oauth', accessToken: 'access-1' }, + ) + const model = recorder('trips_book') + const host = createHarnessHost({ persistence }) + cleanups.push(() => host.close()) + const session = await host.open( + defineHarness({ + name: 'test/mcp-connector-elicit', + adapter: model.adapter, + plugins: () => [ + mcpConnector({ + id: 'trips', + label: 'Trips', + url: 'http://mcp.test/mcp', + needsApproval: () => false, + fetch: async (input, init) => + trips.fetch(new Request(input, init)), + }), + ], + }), + { threadId: 't', principal: { id: 'user-1' } }, + ) + const turn = session.prompt('book a trip') + await vi.waitFor(() => + expect(session.snapshot().pendingQuestions).toHaveLength(1), + ) + const [question] = session.snapshot().pendingQuestions + + expect(question).toMatchObject({ + message: 'Which city?', + schema: tripForm.requestedSchema, + }) + await session.answer(question?.questionId ?? '', { city: 'Rome' }) + await turn + expect(trips.answers).toEqual([ + { action: 'accept', content: { city: 'Rome' } }, + ]) + }, + ) +}) diff --git a/packages/ai-mcp/tests/harness-media.test.ts b/packages/ai-mcp/tests/harness-media.test.ts new file mode 100644 index 0000000000..4212406253 --- /dev/null +++ b/packages/ai-mcp/tests/harness-media.test.ts @@ -0,0 +1,479 @@ +import { mkdir, mkdtemp, rm, symlink, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join, sep } from 'node:path' +import { + Client, + StreamableHTTPClientTransport, +} from '@modelcontextprotocol/client' +import { EventType, defineAgent } from '@tanstack/ai' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { z } from 'zod' +import { createHarnessMcpServer } from '../src/harness' +import type { + AnyTextAdapter, + ImageAdapter, + ModelMessage, + StreamChunk, +} from '@tanstack/ai' +import type { AnyHarness } from '@tanstack/ai-harness' +import type { HarnessMcpServerOptions } from '../src/harness' + +const serverUrl = new URL('https://harness.example.com/mcp') +const now = () => Date.now() +/** The bytes "hello", and their base64. */ +const hello = new TextEncoder().encode('hello') +const helloBase64 = 'aGVsbG8=' +/** One byte over the 5 MB inline limit. */ +const bigImage = new Uint8Array(5 * 1024 * 1024 + 1) + +const mediaListShape = z.object({ + media: z.array(z.object({ id: z.string(), uri: z.string() })), +}) +const linkShape = z.object({ uri: z.string() }) + +function textTurn(text: string): Array { + return [ + { + type: EventType.RUN_STARTED, + runId: 'r', + threadId: 't', + timestamp: now(), + }, + { + type: EventType.TEXT_MESSAGE_START, + messageId: `m-${text}`, + role: 'assistant', + timestamp: now(), + }, + { + type: EventType.TEXT_MESSAGE_CONTENT, + messageId: `m-${text}`, + delta: text, + timestamp: now(), + }, + { + type: EventType.TEXT_MESSAGE_END, + messageId: `m-${text}`, + timestamp: now(), + }, + { + type: EventType.RUN_FINISHED, + runId: 'r', + threadId: 't', + timestamp: now(), + metadata: { tanstack: { finishReason: 'stop' } }, + }, + ] +} + +/** One model call that calls the tool `name` with no input. */ +function toolCallTurn(name: string): Array { + return [ + { + type: EventType.RUN_STARTED, + runId: 'r', + threadId: 't', + timestamp: now(), + }, + { + type: EventType.TOOL_CALL_START, + toolCallId: 'call-0', + toolCallName: name, + timestamp: now(), + }, + { + type: EventType.TOOL_CALL_ARGS, + toolCallId: 'call-0', + delta: '{}', + timestamp: now(), + }, + { type: EventType.TOOL_CALL_END, toolCallId: 'call-0', timestamp: now() }, + { + type: EventType.RUN_FINISHED, + runId: 'r', + threadId: 't', + timestamp: now(), + metadata: { tanstack: { finishReason: 'tool_calls' } }, + }, + ] +} + +/** A model that plays `turns` in order and keeps the messages of each call. */ +function scripted(turns: Array> = []) { + const calls: Array> = [] + const adapter: AnyTextAdapter = { + kind: 'text', + name: 'mock', + model: 'mock', + // `~types` holds types only. Its values are never read, so they are casts. + '~types': { + providerOptions: {} as Record, + inputModalities: ['text'] as readonly ['text'], + messageMetadataByModality: { + text: undefined as unknown, + image: undefined as unknown, + audio: undefined as unknown, + video: undefined as unknown, + document: undefined as unknown, + }, + toolCapabilities: [] as ReadonlyArray, + toolCallMetadata: undefined as unknown, + systemPromptMetadata: undefined as never, + }, + structuredOutput: async () => ({ data: {}, rawText: '{}' }), + chatStream: (options) => { + const turn = turns[calls.length] ?? textTurn('ok') + calls.push(options.messages) + return (async function* () { + yield* turn + })() + }, + } + return { adapter, calls } +} + +/** An image model that returns `bytes` as one image. */ +function imageModel(bytes: Uint8Array): ImageAdapter { + return { + kind: 'image', + name: 'fake-image', + model: 'fake-image-model', + '~types': { + providerOptions: {}, + modelProviderOptionsByName: {}, + modelSizeByName: {}, + modelInputModalitiesByName: {}, + }, + generateImages: async () => ({ + id: 'image-1', + model: 'fake-image-model', + images: [{ b64Json: Buffer.from(bytes).toString('base64') }], + }), + } +} + +/** An agent that paints one image with `bytes`. */ +function painter(bytes: Uint8Array) { + return defineAgent({ + name: 'painter', + description: 'Paints', + run: async (ctx) => { + await ctx.generateImage({ adapter: imageModel(bytes), prompt: 'a cat' }) + return 'painted' + }, + }) +} + +const cleanups: Array<() => Promise> = [] +afterEach(async () => { + for (const cleanup of cleanups.splice(0).reverse()) await cleanup() +}) + +type ServerOptions = Omit + +/** A real MCP client (spec 2026) on a harness MCP server. */ +async function serve(harness: AnyHarness, options: ServerOptions = {}) { + const host = createHarnessHost({ persistence: memoryPersistence() }) + cleanups.push(() => host.close()) + const server = await createHarnessMcpServer({ host, harness, ...options }) + const client = new Client( + { name: 'tester', version: '1.0.0' }, + { versionNegotiation: { mode: { pin: '2026-07-28' } } }, + ) + const transport = new StreamableHTTPClientTransport(serverUrl, { + fetch: async (input, init) => server.fetch(new Request(input, init)), + }) + await client.connect(transport) + cleanups.push(() => client.close()) + return client +} + +/** A chat harness on an MCP server. `calls` has the messages of each model call. */ +async function chatServer(options: ServerOptions = {}) { + const model = scripted() + const harness = defineHarness({ + name: 'test/mcp-media', + adapter: model.adapter, + }) + const client = await serve(harness, options) + return { client, calls: model.calls } +} + +function chat(client: Client, args: Record) { + return client.callTool({ name: 'chat', arguments: args }) +} + +/** The content of the last user message that a model call got. */ +function userContent(messages: Array | undefined) { + return messages?.findLast((message) => message.role === 'user')?.content +} + +/** The MCP result of a tool call that threw an error with `message`. */ +function toolError(message: string) { + return { isError: true, content: [{ type: 'text', text: message }] } +} + +/** The model content for the question "What is this?" with the image "hello". */ +const helloImageQuestion = [ + { type: 'text', content: 'What is this?' }, + { + type: 'image', + source: { type: 'data', value: helloBase64, mimeType: 'image/png' }, + }, +] + +/** + * A temp folder with `allowed/cat.png` ("hello"), `allowed/notes.xyz`, and + * `outside/secret.png`. + */ +async function folders() { + const root = await mkdtemp(join(tmpdir(), 'harness-mcp-media-')) + cleanups.push(() => rm(root, { recursive: true, force: true })) + const allowed = join(root, 'allowed') + const outside = join(root, 'outside') + await mkdir(allowed) + await mkdir(outside) + await writeFile(join(allowed, 'cat.png'), hello) + await writeFile(join(allowed, 'notes.xyz'), 'notes') + await writeFile(join(outside, 'secret.png'), 'secret') + return { allowed, outside } +} + +const outsideRefusal = (path: string) => + `Cannot read ${path}. It must be a file inside the folders this server may read.` + +describe('chat attachments', () => { + it('sends a data attachment to the model', async () => { + const { client, calls } = await chatServer() + + await chat(client, { + message: 'What is this?', + attachments: [ + { data: helloBase64, mimeType: 'image/png', name: 'cat.png' }, + ], + }) + + expect(userContent(calls[0])).toEqual(helloImageQuestion) + }) + + it('sends a path attachment inside filePaths to the model', async () => { + const { allowed } = await folders() + const { client, calls } = await chatServer({ filePaths: [allowed] }) + + await chat(client, { + message: 'What is this?', + attachments: [{ path: join(allowed, 'cat.png') }], + }) + + expect(userContent(calls[0])).toEqual(helloImageQuestion) + }) + + it.each([ + { + given: "a '..' path out of filePaths", + filePaths: true, + // Not `join`: it would remove the '..'. + path: (allowed: string) => + `${allowed}${sep}..${sep}outside${sep}secret.png`, + refusal: outsideRefusal, + }, + { + given: 'a path when the server has no filePaths', + filePaths: false, + path: (allowed: string) => join(allowed, 'cat.png'), + refusal: () => + 'This server does not read files. Send the file as data or url.', + }, + { + given: 'a file type it does not know', + filePaths: true, + path: (allowed: string) => join(allowed, 'notes.xyz'), + refusal: () => + 'Unknown file type: notes.xyz. Send the file as data with its mimeType.', + }, + ])('refuses $given', async ({ filePaths, path, refusal }) => { + const { allowed } = await folders() + const { client, calls } = await chatServer( + filePaths ? { filePaths: [allowed] } : {}, + ) + + const reply = await chat(client, { + message: 'Read this', + attachments: [{ path: path(allowed) }], + }) + + expect(reply).toMatchObject(toolError(refusal(path(allowed)))) + expect(calls).toEqual([]) + }) + + it('refuses a path through a symlink out of filePaths', async (ctx) => { + const { allowed, outside } = await folders() + // A junction needs no admin rights on Windows. Other systems make a + // normal symlink to the folder. + const linked = await symlink(outside, join(allowed, 'link'), 'junction') + .then(() => true) + .catch(() => false) + ctx.skip(!linked, 'This system cannot make a symlink.') + const { client, calls } = await chatServer({ filePaths: [allowed] }) + const path = join(allowed, 'link', 'secret.png') + + const reply = await chat(client, { + message: 'Read this', + attachments: [{ path }], + }) + + expect(reply).toMatchObject(toolError(outsideRefusal(path))) + expect(calls).toEqual([]) + }) + + it('passes a url attachment to the model and does not fetch it', async () => { + // A fetch fails, so the test never reaches the network. + const fetchSpy = vi + .spyOn(globalThis, 'fetch') + .mockRejectedValue(new Error('No network in this test.')) + cleanups.push(async () => fetchSpy.mockRestore()) + const { client, calls } = await chatServer() + + await chat(client, { + message: 'Look', + attachments: [{ url: 'https://cdn.example.com/cat.png' }], + }) + + expect(userContent(calls[0])).toEqual([ + { type: 'text', content: 'Look' }, + { + type: 'image', + source: { + type: 'url', + value: 'https://cdn.example.com/cat.png', + mimeType: 'image/png', + }, + }, + ]) + expect(fetchSpy).not.toHaveBeenCalled() + }) + + it('refuses a url when neither mimeType nor the path gives a kind', async () => { + const { client, calls } = await chatServer() + + const reply = await chat(client, { + message: 'Look', + attachments: [{ url: 'https://example.com/download' }], + }) + + expect(reply).toMatchObject( + toolError( + 'Cannot tell the kind of https://example.com/download. Add its mimeType.', + ), + ) + expect(calls).toEqual([]) + }) + + it('returns the refusal of the media store as a tool error', async () => { + const { client, calls } = await chatServer() + + const reply = await chat(client, { + message: 'Open this', + attachments: [{ data: helloBase64, mimeType: 'application/zip' }], + }) + + expect(reply).toMatchObject( + toolError('Files of type application/zip are not supported.'), + ) + expect(calls).toEqual([]) + }) +}) + +describe('media in results', () => { + it('chat returns a small image of the turn inline and lists it', async () => { + const lead = scripted([toolCallTurn('painter'), textTurn('Here it is.')]) + const client = await serve( + defineHarness({ + name: 'test/mcp-media', + adapter: lead.adapter, + subagents: { agents: [painter(hello)] }, + }), + ) + + const reply = await chat(client, { message: 'Paint a cat' }) + + expect(reply.structuredContent).toEqual({ + status: 'completed', + text: 'Here it is.', + interrupts: [], + questions: [], + media: [ + { + id: expect.any(String), + kind: 'image', + name: expect.any(String), + mimeType: 'image/png', + size: 5, + uri: expect.any(String), + }, + ], + }) + const [media] = mediaListShape.parse(reply.structuredContent).media + expect(media?.uri).toBe(`harness-media://main/${media?.id}`) + expect(reply.content).toEqual([ + { type: 'text', text: JSON.stringify(reply.structuredContent) }, + { type: 'image', data: helloBase64, mimeType: 'image/png' }, + ]) + }) + + /** Runs `agent_painter` with an image over 5 MB. `uri` is its media link. */ + async function paintBig() { + const client = await serve( + defineHarness({ + name: 'test/mcp-media', + adapter: scripted().adapter, + agents: [painter(bigImage)], + expose: { agents: ['painter'] }, + }), + ) + const reply = await client.callTool({ + name: 'agent_painter', + arguments: {}, + }) + const link = z.array(z.unknown()).parse(reply.content)[1] + return { client, reply, uri: linkShape.parse(link).uri } + } + + it('returns an image over 5 MB as a resource link', async () => { + const { reply } = await paintBig() + + expect(reply.content).toEqual([ + { type: 'text', text: 'painted' }, + { + type: 'resource_link', + uri: expect.stringMatching(/^harness-media:\/\/main\/[^/]+$/), + name: expect.any(String), + mimeType: 'image/png', + }, + ]) + }) + + it('reads the bytes of a media link with resources/read', async () => { + const { client, uri } = await paintBig() + + const read = await client.readResource({ uri }) + + const [contents] = read.contents + expect(contents).toMatchObject({ uri, mimeType: 'image/png' }) + const blob = z.object({ blob: z.string() }).parse(contents).blob + expect(Buffer.from(blob, 'base64').equals(Buffer.from(bigImage))).toBe(true) + }) + + it('refuses to read a media link with the thread of another conversation', async () => { + const { client, uri } = await paintBig() + const id = uri.slice('harness-media://main/'.length) + + const read = client.readResource({ uri: `harness-media://other/${id}` }) + + await expect(read).rejects.toThrow( + `Media ${id} was not found in thread other.`, + ) + }) +}) diff --git a/packages/ai-mcp/tests/harness-plugin.test.ts b/packages/ai-mcp/tests/harness-plugin.test.ts new file mode 100644 index 0000000000..fe72962a06 --- /dev/null +++ b/packages/ai-mcp/tests/harness-plugin.test.ts @@ -0,0 +1,425 @@ +import { fileURLToPath } from 'node:url' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { z } from 'zod' +import { McpServer, inputRequired } from '@modelcontextprotocol/server' +import { StdioServerTransport } from '@modelcontextprotocol/server/stdio' +import { toolDefinition } from '@tanstack/ai' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { mcp } from '../src/harness' +import { createMCPServer } from '../src/server/index' +import { + approveSignIns, + elicitServer, + legacyElicitServer, + recorder, + roundsServer, + tripForm, +} from './connector-helpers' +import { startProtectedServer } from './protected-server' +import type { McpServerConfig } from '../src/harness' + +// With this env var, this file is the stdio server of the stdio test. +const childEnv = 'TANSTACK_AI_MCP_PLUGIN_CHILD' + +const cleanups: Array<() => unknown> = [] + +/** A server with one `echo` tool, in this process. */ +function echoServer() { + const echo = toolDefinition({ + name: 'echo', + description: 'Echo text', + inputSchema: z.object({ text: z.string() }), + }).server(async ({ text }) => text) + return createMCPServer({ name: 'echo', version: '1.0.0', tools: [echo] }) +} + +/** An `http` server config whose requests go to `handle`, with no network. */ +function httpServer( + handle: (request: Request) => Promise, + options: Pick = {}, +) { + const config: McpServerConfig = { + type: 'http', + url: 'http://mcp.test/mcp', + fetch: async (input, init) => handle(new Request(input, init)), + ...options, + } + return config +} + +/** Opens a session with `mcp({ servers })` and the recording `model`. */ +async function openWith( + servers: Record, + model = recorder(), +) { + const host = createHarnessHost({ persistence: memoryPersistence() }) + cleanups.push(() => host.close()) + const harness = defineHarness({ + name: 'test/mcp-plugin', + adapter: model.adapter, + plugins: () => [mcp({ servers })], + }) + const session = await host.open(harness, { + threadId: 't', + principal: { id: 'user-1' }, + }) + return { host, session, model } +} + +function isRunning(pid: number) { + try { + process.kill(pid, 0) + return true + } catch { + return false + } +} + +if (process.env[childEnv] === '1') { + // An SDK server, like most published stdio servers. + const server = new McpServer({ name: 'local', version: '1.0.0' }) + server.registerTool( + 'pid', + { + description: 'Name the process of this server', + inputSchema: z.object({ text: z.string() }), + }, + async () => ({ + content: [{ type: 'text' as const, text: `pid:${process.pid}` }], + }), + ) + await server.connect(new StdioServerTransport()) +} else { + afterEach(async () => { + for (const cleanup of cleanups.splice(0).reverse()) await cleanup() + }) + + describe('mcp', () => { + it('connects each server on its own, so a failed server leaves the tools of the others', async () => { + const good = echoServer() + const { session, model } = await openWith({ + good: httpServer((request) => good.fetch(request)), + bad: httpServer(async () => { + throw new TypeError('connection refused') + }), + }) + await session.prompt('hi') + + expect(model.toolNames(0)).toEqual(['good_echo']) + const error = + 'Failed to connect to MCP server: Version negotiation probe failed: connection refused' + await expect(session.command('mcp')).resolves.toBe( + `good: connected (1 tool)\nbad: failed: ${error}`, + ) + expect(session.snapshot().plugins['tanstack/mcp']).toEqual({ + servers: [ + { name: 'good', status: 'connected', toolCount: 1 }, + { name: 'bad', status: 'failed', error, toolCount: 0 }, + ], + }) + }) + + it('stops a server that does not answer within its timeoutMs', async () => { + const slow = echoServer() + const { session, model } = await openWith({ + slow: httpServer( + async (request) => { + const body = + request.method === 'POST' ? await request.clone().text() : '' + // The tool list never comes, so only the timeout ends the wait. + if (body.includes('"tools/list"')) { + return new Promise(() => {}) + } + return slow.fetch(request) + }, + { timeoutMs: 100 }, + ), + }) + await session.prompt('hi') + + expect(model.toolNames(0)).toEqual([]) + await expect(session.command('mcp')).resolves.toMatch( + /^slow: failed: .*timed out/i, + ) + }) + + it('stops a server that does not connect within its timeoutMs, and closes its request', async () => { + const good = echoServer() + const signals: Array = [] + const { session, model } = await openWith({ + good: httpServer((request) => good.fetch(request)), + silent: { + type: 'http', + url: 'http://mcp.test/mcp', + timeoutMs: 100, + // The handshake never gets an answer, so only the timeout ends the + // wait. The signal shows if the request was closed. + fetch: (_input, init) => { + if (init?.signal) signals.push(init.signal) + return new Promise(() => {}) + }, + }, + }) + await session.prompt('hi') + + expect(model.toolNames(0)).toEqual(['good_echo']) + await expect(session.command('mcp')).resolves.toBe( + 'good: connected (1 tool)\nsilent: failed: Failed to connect to MCP server: Version negotiation probe timed out after 100ms', + ) + expect(signals.length).toBeGreaterThan(0) + expect(signals.every((signal) => signal.aborted)).toBe(true) + }) + + it('marks the tools of a codeMode server for code mode', async () => { + const plain = echoServer() + const coded = echoServer() + const { session, model } = await openWith({ + plain: httpServer((request) => plain.fetch(request)), + coded: httpServer((request) => coded.fetch(request), { + codeMode: true, + }), + }) + await session.prompt('hi') + + const marks = (model.calls[0]?.tools ?? []).map((tool) => [ + tool.name, + tool.metadata?.codeMode, + ]) + expect(marks).toEqual([ + ['plain_echo', undefined], + ['coded_echo', true], + ]) + }) + + it.each([ + [ + 'accepts', + { city: 'Paris' }, + { action: 'accept', content: { city: 'Paris' } }, + ], + ['declines', { action: 'decline' }, { action: 'decline' }], + ['cancels', { action: 'cancel' }, { action: 'cancel' }], + ])( + 'asks a form elicitation as a session question, and the user %s', + async (_name, value, expected) => { + const trips = elicitServer(inputRequired.elicit(tripForm)) + const model = recorder('trips_book') + const { session } = await openWith( + { trips: httpServer((request) => trips.fetch(request)) }, + model, + ) + const turn = session.prompt('book a trip') + await vi.waitFor(() => + expect(session.snapshot().pendingQuestions).toHaveLength(1), + ) + const [question] = session.snapshot().pendingQuestions + + expect(question?.message).toBe('Which city?') + expect(question?.schema).toMatchObject(tripForm.requestedSchema) + expect(question?.url).toBeUndefined() + await session.answer(question?.questionId ?? '', value) + await turn + expect(trips.answers).toEqual([expected]) + expect(JSON.stringify(model.calls[1]?.messages)).toContain('booked') + }, + ) + + it('asks a url elicitation as a session question with the url', async () => { + const trips = elicitServer( + inputRequired.elicitUrl({ + message: 'Pay for the trip, then come back.', + url: 'https://pay.example.com/trip', + }), + ) + const { session } = await openWith( + { trips: httpServer((request) => trips.fetch(request)) }, + recorder('trips_book'), + ) + const turn = session.prompt('book a trip') + await vi.waitFor(() => + expect(session.snapshot().pendingQuestions).toHaveLength(1), + ) + const [question] = session.snapshot().pendingQuestions + + expect(question).toMatchObject({ + message: 'Pay for the trip, then come back.', + url: 'https://pay.example.com/trip', + }) + await session.answer(question?.questionId ?? '', 'done') + await turn + expect(trips.answers).toEqual([{ action: 'accept' }]) + }) + + it.each([ + [ + 'accepts', + { city: 'Paris' }, + { action: 'accept', content: { city: 'Paris' } }, + ], + ['declines', { action: 'decline' }, { action: 'decline' }], + ['cancels', { action: 'cancel' }, { action: 'cancel' }], + ])( + 'asks an elicitation/create form request of a spec 2025 server, and the user %s', + async (_name, value, expected) => { + const trips = await legacyElicitServer([{ mode: 'form', ...tripForm }]) + const model = recorder('trips_book') + const { session } = await openWith( + { trips: httpServer((request) => trips.fetch(request)) }, + model, + ) + const turn = session.prompt('book a trip') + await vi.waitFor(() => + expect(session.snapshot().pendingQuestions).toHaveLength(1), + ) + const [question] = session.snapshot().pendingQuestions + + expect(question?.message).toBe('Which city?') + expect(question?.schema).toMatchObject(tripForm.requestedSchema) + await session.answer(question?.questionId ?? '', value) + await turn + expect(trips.answers).toEqual([expected]) + expect(JSON.stringify(model.calls[1]?.messages)).toContain('booked') + }, + ) + + it('asks an elicitation/create url request of a spec 2025 server', async () => { + const trips = await legacyElicitServer([ + { + mode: 'url', + message: 'Pay for the trip, then come back.', + url: 'https://pay.example.com/trip', + elicitationId: 'pay-1', + }, + ]) + const { session } = await openWith( + { trips: httpServer((request) => trips.fetch(request)) }, + recorder('trips_book'), + ) + const turn = session.prompt('book a trip') + await vi.waitFor(() => + expect(session.snapshot().pendingQuestions).toHaveLength(1), + ) + const [question] = session.snapshot().pendingQuestions + + expect(question).toMatchObject({ + message: 'Pay for the trip, then come back.', + url: 'https://pay.example.com/trip', + }) + await session.answer(question?.questionId ?? '', 'done') + await turn + expect(trips.answers).toEqual([{ action: 'accept' }]) + }) + + it('asks two elicitations of one spec 2026 tool call in turn', async () => { + const seat = { + message: 'Which seat?', + requestedSchema: { + type: 'object' as const, + properties: { seat: { type: 'string' as const } }, + required: ['seat'], + }, + } + const trips = roundsServer([ + inputRequired.elicit(tripForm), + inputRequired.elicit(seat), + ]) + const model = recorder('trips_book') + const { session } = await openWith( + { trips: httpServer((request) => trips.fetch(request)) }, + model, + ) + const turn = session.prompt('book a trip') + for (const [message, value] of [ + ['Which city?', { city: 'Paris' }], + ['Which seat?', { seat: '12A' }], + ] as const) { + await vi.waitFor(() => + expect(session.snapshot().pendingQuestions[0]?.message).toBe(message), + ) + const [question] = session.snapshot().pendingQuestions + await session.answer(question?.questionId ?? '', value) + } + await turn + expect(trips.answers).toEqual([ + { action: 'accept', content: { city: 'Paris' } }, + { action: 'accept', content: { seat: '12A' } }, + ]) + expect(JSON.stringify(model.calls[1]?.messages)).toContain('booked') + }) + + it('fails the tool call after 5 elicitations in one call', async () => { + const trips = roundsServer( + Array.from({ length: 6 }, () => inputRequired.elicit(tripForm)), + ) + const model = recorder('trips_book') + const { session } = await openWith( + { trips: httpServer((request) => trips.fetch(request)) }, + model, + ) + const turn = session.prompt('book a trip') + for (let round = 0; round < 5; round++) { + await vi.waitFor(() => + expect(session.snapshot().pendingQuestions).toHaveLength(1), + ) + const [question] = session.snapshot().pendingQuestions + await session.answer(question?.questionId ?? '', { city: 'Paris' }) + } + await turn + expect(trips.answers).toHaveLength(5) + expect(JSON.stringify(model.calls[1]?.messages)).toContain( + 'asked for input more than 5 times', + ) + }) + + it('runs a stdio server as a child process and stops it when the session closes', async () => { + const model = recorder('local_pid') + const { host, session } = await openWith( + { + local: { + type: 'stdio', + command: process.execPath, + args: ['--import', 'jiti/register', fileURLToPath(import.meta.url)], + cwd: fileURLToPath(new URL('..', import.meta.url)), + env: { [childEnv]: '1' }, + }, + }, + model, + ) + await session.prompt('which process runs the server?') + const seen = JSON.stringify(model.calls[1]?.messages) + const pid = Number(/pid:(\d+)/.exec(seen)?.[1]) + expect(isRunning(pid)).toBe(true) + + await host.close() + await vi.waitFor(() => expect(isRunning(pid)).toBe(false), { + timeout: 10_000, + }) + }, 120_000) + + it('signs in to an oauth server with /connect, then gives its tools', async () => { + const protectedServer = await startProtectedServer() + cleanups.push(() => protectedServer.close()) + const { session, model } = await openWith({ + demo: { type: 'http', url: protectedServer.url, oauth: true }, + }) + await session.prompt('before') + await expect(session.command('mcp')).resolves.toBe( + 'demo: failed: Not signed in. Run /connect demo.', + ) + + const browser = approveSignIns(session, 'code-1') + cleanups.push(browser.stop) + await expect(session.command('connect:demo')).resolves.toBe( + 'Connected to demo.', + ) + await session.prompt('after') + + expect(model.toolNames(0)).toEqual([]) + expect(model.toolNames(1)).toEqual(['demo_echo']) + await expect(session.command('mcp')).resolves.toBe( + 'demo: connected (1 tool)', + ) + }) + }) +} diff --git a/packages/ai-mcp/tests/harness.test.ts b/packages/ai-mcp/tests/harness.test.ts new file mode 100644 index 0000000000..8c925d80e9 --- /dev/null +++ b/packages/ai-mcp/tests/harness.test.ts @@ -0,0 +1,1378 @@ +import { + Client, + StreamableHTTPClientTransport, +} from '@modelcontextprotocol/client' +import { + EventType, + defineAgent, + defineChatMiddleware, + defineInterrupt, + toolDefinition, +} from '@tanstack/ai' +import { + HARNESS_EVENTS, + createHarnessHost, + defineCommand, + defineHarness, + definePlugin, +} from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { afterEach, describe, expect, it } from 'vitest' +import { z } from 'zod' +import { createHarnessMcpServer } from '../src/harness' +import type { + AnyTextAdapter, + InterruptBoundaryPhase, + ModelMessage, + StreamChunk, +} from '@tanstack/ai' +import type { AnyHarness, HarnessSession } from '@tanstack/ai-harness' +import type { HarnessMcpServerOptions } from '../src/harness' + +const serverUrl = new URL('https://harness.example.com/mcp') +const now = () => Date.now() + +const pendingShape = z.object({ + interrupts: z.array(z.object({ id: z.string() })), + questions: z.array(z.object({ id: z.string() })), +}) + +/** The response schema of an approval for `remove`, which takes `{ path }`. */ +const removeApprovalSchema = { + oneOf: [ + { + type: 'object', + properties: { + approved: { const: true }, + // zod writes the input schema, so extra keys stay allowed. + editedArgs: { + $schema: 'https://json-schema.org/draft/2020-12/schema', + type: 'object', + properties: { path: { type: 'string' } }, + required: ['path'], + }, + }, + required: ['approved'], + additionalProperties: false, + }, + { + type: 'object', + properties: { approved: { const: false } }, + required: ['approved'], + additionalProperties: false, + }, + ], +} + +/** A generic interrupt that asks for a review note. */ +const review = defineInterrupt({ + id: 'review-plan', + responseSchema: z.object({ note: z.string() }), +}) + +const reviewSchema = { + type: 'object', + properties: { note: { type: 'string' } }, + required: ['note'], +} + +const questionShape = z.object({ questionId: z.string() }) + +const controlTools = [ + 'answer', + 'approve', + 'cancel', + 'chat', + 'reject', + 'resolve', + 'status', + 'steer', +] + +function textTurn(text: string): Array { + return [ + { + type: EventType.RUN_STARTED, + runId: 'r', + threadId: 't', + timestamp: now(), + }, + { + type: EventType.TEXT_MESSAGE_START, + messageId: `m-${text}`, + role: 'assistant', + timestamp: now(), + }, + { + type: EventType.TEXT_MESSAGE_CONTENT, + messageId: `m-${text}`, + delta: text, + timestamp: now(), + }, + { + type: EventType.TEXT_MESSAGE_END, + messageId: `m-${text}`, + timestamp: now(), + }, + { + type: EventType.RUN_FINISHED, + runId: 'r', + threadId: 't', + timestamp: now(), + metadata: { tanstack: { finishReason: 'stop' } }, + }, + ] +} + +/** One model call that fails with `message`. */ +function errorTurn(message: string): Array { + return [ + { + type: EventType.RUN_STARTED, + runId: 'r', + threadId: 't', + timestamp: now(), + }, + { type: EventType.RUN_ERROR, message, timestamp: now() }, + ] +} + +/** One model call that asks to remove each path, one tool call per path. */ +function removeTurn(...paths: Array) { + return toolCallTurn( + paths.map((path) => ({ name: 'remove', input: { path } })), + ) +} + +/** One model call that makes each tool call. */ +function toolCallTurn( + toolCalls: Array<{ name: string; input: unknown }>, +): Array { + const calls = toolCalls.flatMap( + ({ name, input }, index): Array => [ + { + type: EventType.TOOL_CALL_START, + toolCallId: `call-${index}`, + toolCallName: name, + timestamp: now(), + }, + { + type: EventType.TOOL_CALL_ARGS, + toolCallId: `call-${index}`, + delta: JSON.stringify(input), + timestamp: now(), + }, + { + type: EventType.TOOL_CALL_END, + toolCallId: `call-${index}`, + timestamp: now(), + }, + ], + ) + return [ + { + type: EventType.RUN_STARTED, + runId: 'r', + threadId: 't', + timestamp: now(), + }, + ...calls, + { + type: EventType.RUN_FINISHED, + runId: 'r', + threadId: 't', + timestamp: now(), + metadata: { tanstack: { finishReason: 'tool_calls' } }, + }, + ] +} + +function deferred() { + let resolve: (value: void) => void = () => {} + const promise = new Promise((done) => { + resolve = done + }) + return { promise, resolve } +} + +/** + * A point that work reaches and then waits at. `reached` resolves when the + * work arrives. The work continues after `release()`. + */ +function hold() { + const reached = deferred() + const released = deferred() + return { + reached: reached.promise, + release: () => released.resolve(), + async wait() { + reached.resolve() + await released.promise + }, + } +} + +/** A model call that waits at its `hold`, then streams `chunks`. */ +function heldTurn(chunks: Array) { + return { hold: hold(), chunks } +} + +type ScriptedTurn = Array | ReturnType + +async function* play(turn: ScriptedTurn) { + if (Array.isArray(turn)) { + yield* turn + return + } + await turn.hold.wait() + yield* turn.chunks +} + +/** A model that plays `turns` in order and keeps the messages of each call. */ +function scripted(turns: Array) { + const calls: Array> = [] + const adapter: AnyTextAdapter = { + kind: 'text', + name: 'mock', + model: 'mock', + // `~types` holds types only. Its values are never read, so they are casts. + '~types': { + providerOptions: {} as Record, + inputModalities: ['text'] as readonly ['text'], + messageMetadataByModality: { + text: undefined as unknown, + image: undefined as unknown, + audio: undefined as unknown, + video: undefined as unknown, + document: undefined as unknown, + }, + toolCapabilities: [] as ReadonlyArray, + toolCallMetadata: undefined as unknown, + systemPromptMetadata: undefined as never, + }, + structuredOutput: async () => ({ data: {}, rawText: '{}' }), + chatStream: (options) => { + const turn = turns[calls.length] ?? textTurn('') + calls.push(options.messages) + return play(turn) + }, + } + return { adapter, calls } +} + +function userTexts(messages: Array | undefined) { + return (messages ?? []) + .filter((message) => message.role === 'user') + .map((message) => message.content) +} + +/** The MCP result of a tool call that threw an error with `message`. */ +function toolError(message: string) { + return { isError: true, content: [{ type: 'text', text: message }] } +} + +/** The ids of the interrupts that wait, from a tool result. */ +function interruptIds(result: { structuredContent?: unknown }) { + return pendingShape + .parse(result.structuredContent) + .interrupts.map(({ id }) => id) +} + +/** + * A middleware that stops the first run of a turn at `phase` with a + * `review` interrupt. `notes` keeps each answer that it gets. + */ +function reviewAt(phase: InterruptBoundaryPhase) { + const notes: Array = [] + const middleware = defineChatMiddleware({ + onInterruptBoundary(ctx) { + if (ctx.parentRunId || ctx.phase !== phase) return + return { + interrupts: [ + review.interrupt({ + key: 'plan', + reason: 'review', + message: 'Review the plan', + }), + ], + } + }, + onInterruptResolution(_ctx, resolutions) { + for (const resolution of resolutions.for(review)) { + if (resolution.status === 'resolved') notes.push(resolution.response) + } + return { toolResume: 'continue' } + }, + }) + return { middleware, notes } +} + +/** Waits until `session` asks a question, then returns its id. */ +async function nextQuestionId(session: HarnessSession) { + for await (const { event } of session.events()) { + const isQuestion = + event.type === EventType.CUSTOM && event.name === HARNESS_EVENTS.question + if (isQuestion) return questionShape.parse(event.value).questionId + } + throw new Error('The session closed before it asked a question') +} + +const pricer = defineAgent({ + name: 'pricer', + description: 'Prices a vendor', + inputSchema: z.object({ vendor: z.string() }), + run: async (ctx) => `${ctx.input.vendor}: twelve dollars`, +}) + +const broken = defineAgent({ + name: 'broken', + description: 'Always fails', + run: async () => { + throw new Error('The price list is gone.') + }, +}) + +const commands = definePlugin({ + name: 'test/commands', + setup: (ctx) => ({ + commands: { + 'connect:notion': defineCommand({ + description: 'Sign in to Notion', + run: () => 'Connected to Notion.', + }), + 'release/v2.1-beta': defineCommand({ + description: 'Release the beta', + run: () => 'Released.', + }), + confirm: defineCommand({ + description: 'Ask first', + run: async () => { + const sure = await ctx.session.ask({ + message: 'Really?', + schema: { type: 'boolean' }, + }) + return sure === true ? 'Confirmed.' : 'Stopped.' + }, + }), + greet: defineCommand({ + description: 'Greet a person', + input: z.object({ name: z.string() }), + run: (input) => `Hello, ${input.name}.`, + }), + fail: defineCommand({ + description: 'Always fails', + run: () => { + throw new Error('Notion is down.') + }, + }), + }, + }), +}) + +const cleanups: Array<() => Promise> = [] +afterEach(async () => { + for (const cleanup of cleanups.splice(0).reverse()) await cleanup() +}) + +type ServerOptions = Omit + +async function serve(harness: AnyHarness, options: ServerOptions = {}) { + const host = createHarnessHost({ persistence: memoryPersistence() }) + cleanups.push(() => host.close()) + const server = await createHarnessMcpServer({ host, harness, ...options }) + return { server, host } +} + +/** + * A harness on an MCP server. With `reviewAt`, a middleware stops the turn at + * that phase with a `review` interrupt. + */ +async function harnessServer( + turns: Array, + options: ServerOptions & { reviewAt?: InterruptBoundaryPhase } = {}, +) { + const { reviewAt: phase, ...serverOptions } = options + const model = scripted(turns) + const removed: Array = [] + const remove = toolDefinition({ + name: 'remove', + description: 'Remove a file', + needsApproval: true, + inputSchema: z.object({ path: z.string() }), + }).server(async ({ path }) => { + removed.push(path) + return `removed ${path}` + }) + // No `.server()`: the client runs it and sends the output. + const pickColor = toolDefinition({ + name: 'pick_color', + description: 'Ask the user for a color', + inputSchema: z.object({ hint: z.string() }), + outputSchema: z.object({ color: z.string() }), + }) + const reviewer = phase === undefined ? undefined : reviewAt(phase) + const agentHold = hold() + const waiter = defineAgent({ + name: 'waiter', + description: 'Waits until the test releases it', + run: async () => { + await agentHold.wait() + return 'Done waiting.' + }, + }) + const harness = defineHarness({ + name: 'test/mcp-harness', + adapter: model.adapter, + tools: [remove, pickColor], + agents: [pricer, broken, waiter], + expose: { + agents: ['pricer', 'broken', 'waiter'], + commands: [ + 'confirm', + 'connect:notion', + 'fail', + 'greet', + 'release/v2.1-beta', + ], + }, + plugins: () => [commands], + ...(reviewer === undefined + ? {} + : { middleware: [reviewer.middleware], interrupts: [review] }), + }) + const { server, host } = await serve(harness, serverOptions) + const session = () => host.open(harness, { threadId: 'main' }) + return { + server, + session, + removed, + notes: reviewer?.notes ?? [], + calls: model.calls, + agentHold, + } +} + +type ElicitReply = + | { action: 'accept'; content: { value: string } } + | { action: 'decline' } + +function accepted(value: string): ElicitReply { + return { action: 'accept', content: { value } } +} + +/** + * A real MCP client on `server.fetch`. With `reply`, the client supports + * elicitation and gives that reply to every question (spec 2025 sessions + * only). Without it, the client speaks spec 2026 and has no elicitation. + */ +async function connect( + server: { fetch: (request: Request) => Promise }, + options: { reply?: ElicitReply } = {}, +) { + const reply = options.reply + const client = new Client( + { name: 'tester', version: '1.0.0' }, + reply === undefined + ? { versionNegotiation: { mode: { pin: '2026-07-28' } } } + : { capabilities: { elicitation: { form: {} } } }, + ) + const asked: Array = [] + if (reply !== undefined) { + client.setRequestHandler('elicitation/create', async (request) => { + asked.push(request.params.message) + return reply + }) + } + const transport = new StreamableHTTPClientTransport(serverUrl, { + fetch: async (input, init) => server.fetch(new Request(input, init)), + }) + await client.connect(transport) + cleanups.push(() => client.close()) + return { client, asked } +} + +describe('createHarnessMcpServer', () => { + it('lists the control tools, one tool per exposed agent, and one per exposed command', async () => { + const { server } = await harnessServer([]) + const { client } = await connect(server) + + const listed = await client.listTools() + expect(listed.tools.map((tool) => tool.name).sort()).toEqual([ + 'agent_broken', + 'agent_pricer', + 'agent_waiter', + 'answer', + 'approve', + 'cancel', + 'chat', + 'command_confirm', + 'command_connect_notion', + 'command_fail', + 'command_greet', + 'command_release_v2_1_beta', + 'reject', + 'resolve', + 'status', + 'steer', + ]) + const agentTool = listed.tools.find((tool) => tool.name === 'agent_pricer') + expect(agentTool?.description).toBe('Prices a vendor') + expect(Object.keys(agentTool?.inputSchema.properties ?? {})).toEqual([ + 'vendor', + 'threadId', + ]) + const commandTool = listed.tools.find( + (tool) => tool.name === 'command_connect_notion', + ) + expect(commandTool?.description).toBe('Sign in to Notion') + const inputTool = listed.tools.find((tool) => tool.name === 'command_greet') + expect(Object.keys(inputTool?.inputSchema.properties ?? {})).toEqual([ + 'name', + 'threadId', + ]) + }) + + it('lists only the control tools for a harness that exposes no agents', async () => { + const { server } = await serve( + defineHarness({ name: 'test/bare', adapter: scripted([]).adapter }), + ) + const { client } = await connect(server) + + const listed = await client.listTools() + expect(listed.tools.map((tool) => tool.name).sort()).toEqual(controlTools) + }) + + it('gives command names that clash as tool names distinct tool names, in name order', async () => { + const clashing = definePlugin({ + name: 'test/clashing', + setup: () => ({ + commands: { + // Added first, but `connect:notion` comes first in name order. + connect_notion: defineCommand({ + description: 'Sign in with the new flow', + run: () => 'New flow.', + }), + 'connect:notion': defineCommand({ + description: 'Sign in to Notion', + run: () => 'Connected to Notion.', + }), + }, + }), + }) + const { server } = await serve( + defineHarness({ + name: 'test/clash', + adapter: scripted([]).adapter, + plugins: () => [clashing], + expose: { commands: ['connect_notion', 'connect:notion'] }, + }), + ) + const { client } = await connect(server) + + const listed = await client.listTools() + const commandTools = listed.tools + .filter((tool) => tool.name.startsWith('command_')) + .map(({ name, description }) => ({ name, description })) + .sort((left, right) => left.name.localeCompare(right.name)) + expect(commandTools).toEqual([ + { + name: 'command_connect_notion', + description: 'Sign in to Notion (command connect:notion)', + }, + { + name: 'command_connect_notion_2', + description: 'Sign in with the new flow (command connect_notion)', + }, + ]) + const plain = await client.callTool({ + name: 'command_connect_notion', + arguments: {}, + }) + expect(plain.content).toEqual([ + { type: 'text', text: 'Connected to Notion.' }, + ]) + const suffixed = await client.callTool({ + name: 'command_connect_notion_2', + arguments: {}, + }) + expect(suffixed.content).toEqual([{ type: 'text', text: 'New flow.' }]) + }) + + it('lists and runs only the commands in expose.commands', async () => { + const modes: Array = [] + const permissions = definePlugin({ + name: 'test/permissions', + setup: () => ({ + commands: { + mode: defineCommand({ + description: 'Set the permission mode', + input: z.object({ mode: z.string() }), + run: (input) => { + modes.push(input.mode) + return `Mode: ${input.mode}.` + }, + }), + undo: defineCommand({ + description: 'Undo the last change', + run: () => 'Undone.', + }), + }, + }), + }) + const { server } = await serve( + defineHarness({ + name: 'test/expose-commands', + adapter: scripted([]).adapter, + plugins: () => [permissions], + expose: { commands: ['undo'] }, + }), + ) + const { client } = await connect(server) + + const listed = await client.listTools() + const commandTools = listed.tools + .map((tool) => tool.name) + .filter((name) => name.startsWith('command_')) + expect(commandTools).toEqual(['command_undo']) + + await expect( + client.callTool({ name: 'command_mode', arguments: { mode: 'bypass' } }), + ).rejects.toThrow('Tool command_mode not found') + expect(modes).toEqual([]) + + const undone = await client.callTool({ + name: 'command_undo', + arguments: {}, + }) + expect(undone.content).toEqual([{ type: 'text', text: 'Undone.' }]) + }) + + it.each([ + { + options: {}, + expected: { name: 'test/mcp-harness', version: '1.0.0' }, + }, + { + options: { name: 'desk', version: '2.1.0' }, + expected: { name: 'desk', version: '2.1.0' }, + }, + ])( + 'names the server $expected.name $expected.version', + async ({ options, expected }) => { + const { server } = await harnessServer([], options) + const { client } = await connect(server) + + expect(client.getServerVersion()).toMatchObject(expected) + }, + ) + + it('chat returns the answer of the turn', async () => { + const { server } = await harnessServer([ + textTurn('Hello from the harness.'), + ]) + const { client } = await connect(server) + + const reply = await client.callTool({ + name: 'chat', + arguments: { message: 'hi' }, + }) + expect(reply.structuredContent).toEqual({ + status: 'completed', + text: 'Hello from the harness.', + interrupts: [], + questions: [], + }) + }) + + it('returns a failed model call as a tool error', async () => { + const { server } = await harnessServer([errorTurn('The model is down.')]) + const { client } = await connect(server) + + const reply = await client.callTool({ + name: 'chat', + arguments: { message: 'hi' }, + }) + expect(reply).toMatchObject(toolError('The model is down.')) + }) + + it('returns a tool that needs approval as pending, and approve runs it', async () => { + const { server, removed } = await harnessServer([ + removeTurn('a.txt'), + textTurn('Removed a.txt.'), + ]) + const { client } = await connect(server) + + const pending = await client.callTool({ + name: 'chat', + arguments: { message: 'remove a.txt' }, + }) + expect(pending.structuredContent).toEqual({ + status: 'interrupted', + text: '', + interrupts: [ + { + id: expect.any(String), + kind: 'approval', + tool: 'remove', + args: { path: 'a.txt' }, + message: 'Approval required to run remove', + responseSchema: removeApprovalSchema, + }, + ], + questions: [], + }) + expect(removed).toEqual([]) + + const status = await client.callTool({ name: 'status', arguments: {} }) + expect(status.structuredContent).toMatchObject({ + status: 'requires_action', + interrupts: [{ kind: 'approval', tool: 'remove' }], + agents: [], + queuedTurns: 0, + }) + + const approved = await client.callTool({ name: 'approve', arguments: {} }) + expect(approved.structuredContent).toEqual({ + status: 'completed', + text: 'Removed a.txt.', + interrupts: [], + questions: [], + }) + expect(removed).toEqual(['a.txt']) + }) + + it('reject continues the turn without running the tool', async () => { + const { server, removed } = await harnessServer([ + removeTurn('a.txt'), + textTurn('Kept a.txt.'), + ]) + const { client } = await connect(server) + + await client.callTool({ + name: 'chat', + arguments: { message: 'remove a.txt' }, + }) + const rejected = await client.callTool({ name: 'reject', arguments: {} }) + expect(rejected.structuredContent).toEqual({ + status: 'completed', + text: 'Kept a.txt.', + interrupts: [], + questions: [], + }) + expect(removed).toEqual([]) + }) + + it('refuses approve when no approval waits', async () => { + const { server } = await harnessServer([]) + const { client } = await connect(server) + + const refused = await client.callTool({ name: 'approve', arguments: {} }) + expect(refused).toMatchObject(toolError('No approvals are waiting.')) + }) + + it('refuses a resolve that does not answer every open interrupt', async () => { + const { server, removed } = await harnessServer([ + removeTurn('a.txt', 'b.txt'), + textTurn('Removed one file.'), + ]) + const { client } = await connect(server) + + const pending = await client.callTool({ + name: 'chat', + arguments: { message: 'remove both files' }, + }) + const ids = interruptIds(pending) + expect(ids).toHaveLength(2) + const [first, second] = ids + + const partial = await client.callTool({ + name: 'resolve', + arguments: { decisions: [{ interruptId: first, approved: true }] }, + }) + expect(partial).toMatchObject( + toolError( + `resolve must answer every open interrupt. Missing: ${second}.`, + ), + ) + expect(removed).toEqual([]) + + const mixed = await client.callTool({ + name: 'resolve', + arguments: { + decisions: [ + { interruptId: first, approved: true }, + { interruptId: second, approved: false }, + ], + }, + }) + expect(mixed.structuredContent).toMatchObject({ + status: 'completed', + text: 'Removed one file.', + }) + expect(removed).toEqual(['a.txt']) + }) + + it('refuses a resolve that names an interrupt that is not open', async () => { + const { server, removed } = await harnessServer([removeTurn('a.txt')]) + const { client } = await connect(server) + + const pending = await client.callTool({ + name: 'chat', + arguments: { message: 'remove a.txt' }, + }) + const [open] = interruptIds(pending) + + const refused = await client.callTool({ + name: 'resolve', + arguments: { + decisions: [ + { interruptId: open, approved: true }, + { interruptId: 'ghost', approved: true }, + ], + }, + }) + expect(refused).toMatchObject( + toolError('These interrupts are not open: ghost.'), + ) + expect(removed).toEqual([]) + }) + + it('returns the harness refusal of a resolve as a tool error', async () => { + const { server } = await harnessServer([]) + const { client } = await connect(server) + + const refused = await client.callTool({ + name: 'resolve', + arguments: { decisions: [] }, + }) + expect(refused).toMatchObject( + toolError('The harness refused the decisions: no_pending_interrupts.'), + ) + }) + + it("approves every tool call on its own with approvals 'auto'", async () => { + const { server, removed } = await harnessServer( + [removeTurn('a.txt'), textTurn('Removed a.txt.')], + { approvals: 'auto' }, + ) + const { client } = await connect(server) + + const reply = await client.callTool({ + name: 'chat', + arguments: { message: 'remove a.txt' }, + }) + expect(reply.structuredContent).toEqual({ + status: 'completed', + text: 'Removed a.txt.', + interrupts: [], + questions: [], + }) + expect(removed).toEqual(['a.txt']) + }) + + it('asks the user about an approval when the client supports elicitation', async () => { + const { server, removed } = await harnessServer([ + removeTurn('a.txt'), + textTurn('Removed a.txt.'), + ]) + const { client, asked } = await connect(server, { reply: accepted('yes') }) + + const reply = await client.callTool({ + name: 'chat', + arguments: { message: 'remove a.txt' }, + }) + expect(reply.structuredContent).toMatchObject({ + status: 'completed', + text: 'Removed a.txt.', + }) + expect(removed).toEqual(['a.txt']) + expect(asked).toEqual([ + 'Approval required to run remove. Arguments: {"path":"a.txt"}. Approve? Answer yes or no.', + ]) + }) + + it('rejects the tool call when the user answers no', async () => { + const { server, removed } = await harnessServer([ + removeTurn('a.txt'), + textTurn('Kept a.txt.'), + ]) + const { client } = await connect(server, { reply: accepted('no') }) + + const reply = await client.callTool({ + name: 'chat', + arguments: { message: 'remove a.txt' }, + }) + expect(reply.structuredContent).toEqual({ + status: 'completed', + text: 'Kept a.txt.', + interrupts: [], + questions: [], + }) + expect(removed).toEqual([]) + }) + + it('returns the approval as pending when the user declines the question', async () => { + const { server, removed } = await harnessServer([removeTurn('a.txt')]) + const { client, asked } = await connect(server, { + reply: { action: 'decline' }, + }) + + const reply = await client.callTool({ + name: 'chat', + arguments: { message: 'remove a.txt' }, + }) + expect(reply.structuredContent).toMatchObject({ + status: 'interrupted', + interrupts: [ + { kind: 'approval', tool: 'remove', args: { path: 'a.txt' } }, + ], + }) + expect(asked).toHaveLength(1) + expect(removed).toEqual([]) + }) + + it('returns a generic interrupt with its kind and schema, and resolve answers it with a payload', async () => { + const { server, notes } = await harnessServer( + [textTurn('Here is the plan.'), textTurn('Plan approved.')], + { reviewAt: 'afterModel' }, + ) + const { client } = await connect(server) + + const pending = await client.callTool({ + name: 'chat', + arguments: { message: 'make a plan' }, + }) + expect(pending.structuredContent).toEqual({ + status: 'interrupted', + text: 'Here is the plan.', + interrupts: [ + { + id: expect.any(String), + kind: 'generic', + message: 'Review the plan', + responseSchema: reviewSchema, + }, + ], + questions: [], + }) + const [id] = interruptIds(pending) + + const refused = await client.callTool({ name: 'approve', arguments: {} }) + expect(refused).toMatchObject( + toolError(`Use resolve: interrupt ${id} needs a payload.`), + ) + + const resolved = await client.callTool({ + name: 'resolve', + arguments: { + decisions: [{ interruptId: id, payload: { note: 'Ship it.' } }], + }, + }) + expect(resolved.structuredContent).toEqual({ + status: 'completed', + text: 'Plan approved.', + interrupts: [], + questions: [], + }) + expect(notes).toEqual([{ note: 'Ship it.' }]) + }) + + it("approvals 'auto' leaves a generic interrupt pending", async () => { + const { server, notes } = await harnessServer( + [textTurn('Here is the plan.')], + { approvals: 'auto', reviewAt: 'afterModel' }, + ) + const { client } = await connect(server) + + const reply = await client.callTool({ + name: 'chat', + arguments: { message: 'make a plan' }, + }) + expect(reply.structuredContent).toMatchObject({ + status: 'interrupted', + interrupts: [{ kind: 'generic', message: 'Review the plan' }], + }) + expect(notes).toEqual([]) + }) + + it('resolves an approval and a generic interrupt of one turn in one call', async () => { + const { server, removed, notes } = await harnessServer( + [removeTurn('a.txt'), textTurn('Removed a.txt.')], + { approvals: 'auto', reviewAt: 'beforeTools' }, + ) + const { client } = await connect(server) + + const pending = await client.callTool({ + name: 'chat', + arguments: { message: 'remove a.txt' }, + }) + // 'auto' answers approvals only. The review is open, so both wait. + expect(pending.structuredContent).toMatchObject({ + status: 'interrupted', + interrupts: [{ kind: 'approval', tool: 'remove' }, { kind: 'generic' }], + }) + expect(removed).toEqual([]) + const [approval, reviewId] = interruptIds(pending) + + const resolved = await client.callTool({ + name: 'resolve', + arguments: { + decisions: [ + { interruptId: approval, approved: true }, + { interruptId: reviewId, payload: { note: 'Go ahead.' } }, + ], + }, + }) + expect(resolved.structuredContent).toEqual({ + status: 'completed', + text: 'Removed a.txt.', + interrupts: [], + questions: [], + }) + expect(removed).toEqual(['a.txt']) + expect(notes).toEqual([{ note: 'Go ahead.' }]) + }) + + it.each([ + { + given: 'no approved and no payload for an approval', + approval: {}, + reviewDecision: { payload: { note: 'Go ahead.' } }, + refused: 0, + problem: 'needs approved or payload.', + }, + { + given: 'only approved for a generic interrupt', + approval: { approved: true }, + reviewDecision: { approved: true }, + refused: 1, + problem: 'needs a payload. It is not an approval.', + }, + ])( + 'refuses a decision with $given', + async ({ approval, reviewDecision, refused, problem }) => { + const { server, removed, notes } = await harnessServer( + [removeTurn('a.txt')], + { reviewAt: 'beforeTools' }, + ) + const { client } = await connect(server) + + const pending = await client.callTool({ + name: 'chat', + arguments: { message: 'remove a.txt' }, + }) + const ids = interruptIds(pending) + const [approvalId, reviewId] = ids + + const reply = await client.callTool({ + name: 'resolve', + arguments: { + decisions: [ + { interruptId: approvalId, ...approval }, + { interruptId: reviewId, ...reviewDecision }, + ], + }, + }) + expect(reply).toMatchObject( + toolError(`The decision for ${ids[refused]} ${problem}`), + ) + const status = await client.callTool({ name: 'status', arguments: {} }) + expect(status.structuredContent).toMatchObject({ + status: 'requires_action', + interrupts: [{ kind: 'approval' }, { kind: 'generic' }], + }) + expect(removed).toEqual([]) + expect(notes).toEqual([]) + }, + ) + + it('returns a client tool call as pending, and resolve sends its output', async () => { + const { server, calls } = await harnessServer([ + toolCallTurn([{ name: 'pick_color', input: { hint: 'sky' } }]), + textTurn('Blue it is.'), + ]) + const { client } = await connect(server) + + const pending = await client.callTool({ + name: 'chat', + arguments: { message: 'pick a color' }, + }) + expect(pending.structuredContent).toMatchObject({ + status: 'interrupted', + interrupts: [ + { + kind: 'client-tool', + tool: 'pick_color', + args: { hint: 'sky' }, + message: 'Client tool pick_color is ready to run', + }, + ], + }) + const [id] = interruptIds(pending) + + const resolved = await client.callTool({ + name: 'resolve', + arguments: { + decisions: [{ interruptId: id, payload: { color: 'blue' } }], + }, + }) + expect(resolved.structuredContent).toMatchObject({ + status: 'completed', + text: 'Blue it is.', + }) + const toolResults = (calls[1] ?? []).filter( + (message) => message.role === 'tool', + ) + expect(toolResults).toMatchObject([ + { toolCallId: 'call-0', content: '{"color":"blue"}' }, + ]) + }) + + it('answer resolves the question of a command and returns its result', async () => { + const { server } = await harnessServer([]) + const { client } = await connect(server) + + const waiting = await client.callTool({ + name: 'command_confirm', + arguments: {}, + }) + expect(waiting.structuredContent).toEqual({ + status: 'waiting', + interrupts: [], + questions: [ + { + id: expect.any(String), + message: 'Really?', + schema: { type: 'boolean' }, + }, + ], + }) + const [question] = pendingShape.parse(waiting.structuredContent).questions + if (question === undefined) throw new Error('The command asked nothing') + + const answered = await client.callTool({ + name: 'answer', + arguments: { questionId: question.id, value: true }, + }) + expect(answered.content).toEqual([{ type: 'text', text: 'Confirmed.' }]) + }) + + it('answer returns answered for a question that no tool call waits on', async () => { + const { server, session } = await harnessServer([]) + const { client } = await connect(server) + const live = await session() + const confirm = live.command('confirm') + const questionId = await nextQuestionId(live) + + const answered = await client.callTool({ + name: 'answer', + arguments: { questionId, value: true }, + }) + expect(answered.structuredContent).toEqual({ + status: 'answered', + interrupts: [], + questions: [], + }) + expect(await confirm).toBe('Confirmed.') + }) + + it('returns the harness refusal of an answer as a tool error', async () => { + const { server } = await harnessServer([]) + const { client } = await connect(server) + + const refused = await client.callTool({ + name: 'answer', + arguments: { questionId: 'q-unknown', value: true }, + }) + expect(refused).toMatchObject( + toolError('The harness refused the answer: unknown_question.'), + ) + }) + + it('agent_ runs the agent and returns its result', async () => { + const { server } = await harnessServer([]) + const { client } = await connect(server) + + const result = await client.callTool({ + name: 'agent_pricer', + arguments: { vendor: 'Acme' }, + }) + expect(result.content).toEqual([ + { type: 'text', text: 'Acme: twelve dollars' }, + ]) + }) + + it('returns the error of a failed agent as a tool error', async () => { + const { server } = await harnessServer([]) + const { client } = await connect(server) + + const result = await client.callTool({ + name: 'agent_broken', + arguments: {}, + }) + expect(result).toMatchObject(toolError('The price list is gone.')) + }) + + it('command_ runs the command under a tool-safe name', async () => { + const { server } = await harnessServer([]) + const { client } = await connect(server) + + const result = await client.callTool({ + name: 'command_connect_notion', + arguments: {}, + }) + expect(result.content).toEqual([ + { type: 'text', text: 'Connected to Notion.' }, + ]) + }) + + it('command_ takes the command input next to threadId', async () => { + const { server } = await harnessServer([]) + const { client } = await connect(server) + + const result = await client.callTool({ + name: 'command_greet', + arguments: { name: 'Ada', threadId: 'main' }, + }) + expect(result.content).toEqual([{ type: 'text', text: 'Hello, Ada.' }]) + }) + + it('returns the error of a failed command as a tool error', async () => { + const { server } = await harnessServer([]) + const { client } = await connect(server) + + const result = await client.callTool({ + name: 'command_fail', + arguments: {}, + }) + expect(result).toMatchObject(toolError('Notion is down.')) + }) + + it('steer starts a turn when no turn runs', async () => { + const { server, calls } = await harnessServer([ + textTurn('Tabs it is.'), + textTurn('Formatted.'), + ]) + const { client } = await connect(server) + + const steered = await client.callTool({ + name: 'steer', + arguments: { message: 'use tabs' }, + }) + expect(steered.structuredContent).toEqual({ + inputId: expect.any(String), + status: 'accepted', + operationId: expect.any(String), + }) + const reply = await client.callTool({ + name: 'chat', + arguments: { message: 'format it' }, + }) + expect(reply.structuredContent).toMatchObject({ text: 'Formatted.' }) + expect(userTexts(calls[0])).toEqual(['use tabs']) + expect(userTexts(calls[1])).toEqual(['use tabs', 'format it']) + }) + + it('cancel stops the running turn, and its chat call ends with an error', async () => { + const held = heldTurn(textTurn('Too late.')) + const { server } = await harnessServer([held]) + const { client } = await connect(server) + + const reply = client.callTool({ + name: 'chat', + arguments: { message: 'write a long story' }, + }) + await held.hold.reached + const cancelled = await client.callTool({ name: 'cancel', arguments: {} }) + expect(cancelled.structuredContent).toEqual({ + inputId: expect.any(String), + status: 'accepted', + operationId: expect.any(String), + }) + held.hold.release() + expect(await reply).toMatchObject(toolError('Cancelled.')) + }) + + it('returns the refusal of cancel when no turn runs', async () => { + const { server } = await harnessServer([]) + const { client } = await connect(server) + + const refused = await client.callTool({ name: 'cancel', arguments: {} }) + expect(refused.structuredContent).toEqual({ + inputId: expect.any(String), + status: 'rejected', + reason: 'not_running', + }) + }) + + it('status shows the running agents and the queued turns', async () => { + const held = heldTurn(textTurn('Busy.')) + const { server, session, agentHold } = await harnessServer([ + held, + textTurn('Follow-up done.'), + ]) + const { client } = await connect(server) + + const turn = client.callTool({ + name: 'chat', + arguments: { message: 'work' }, + }) + await held.hold.reached + await (await session()).followUp('then this') + const agent = client.callTool({ name: 'agent_waiter', arguments: {} }) + await agentHold.reached + + const status = await client.callTool({ name: 'status', arguments: {} }) + expect(status.structuredContent).toEqual({ + status: 'running', + interrupts: [], + questions: [], + agents: [{ id: expect.any(String), agent: 'waiter' }], + queuedTurns: 1, + }) + held.hold.release() + agentHold.release() + await Promise.all([turn, agent]) + }) + + it('keeps a separate conversation per threadId', async () => { + const { server, calls } = await harnessServer([ + textTurn('one'), + textTurn('two'), + textTurn('three'), + ]) + const { client } = await connect(server) + + const threads = [ + { message: 'first', threadId: 'a' }, + { message: 'second', threadId: 'b' }, + { message: 'third', threadId: 'a' }, + ] + for (const input of threads) { + await client.callTool({ name: 'chat', arguments: input }) + } + expect(userTexts(calls[1])).toEqual(['second']) + expect(userTexts(calls[2])).toEqual(['first', 'third']) + }) + + it.each([ + { options: {}, thread: 'main' }, + { options: { threadId: 'desk' }, thread: 'desk' }, + ])( + 'uses thread $thread for a call without threadId', + async ({ options, thread }) => { + const { server, calls } = await harnessServer( + [textTurn('one'), textTurn('two')], + options, + ) + const { client } = await connect(server) + + await client.callTool({ name: 'chat', arguments: { message: 'first' } }) + await client.callTool({ + name: 'chat', + arguments: { message: 'second', threadId: thread }, + }) + expect(userTexts(calls[1])).toEqual(['first', 'second']) + }, + ) +}) diff --git a/packages/ai-mcp/tests/protected-server.ts b/packages/ai-mcp/tests/protected-server.ts new file mode 100644 index 0000000000..a9b4e7ad59 --- /dev/null +++ b/packages/ai-mcp/tests/protected-server.ts @@ -0,0 +1,170 @@ +import { createServer } from 'node:http' +import { z } from 'zod' +import { + McpServer, + WebStandardStreamableHTTPServerTransport, +} from '@modelcontextprotocol/server' +import type { IncomingMessage, Server } from 'node:http' + +/** Copy the Node request headers into Web `Headers`. */ +function webHeaders(req: IncomingMessage) { + const headers = new Headers() + const entries = Object.entries(req.headersDistinct) + for (const [name, values] of entries) { + for (const value of values ?? []) headers.append(name, value) + } + return headers +} + +/** + * A local MCP server behind OAuth, like Notion's or Linear's: protected + * resource metadata, authorization server metadata, dynamic client + * registration, a token endpoint, and an `echo` tool. + * + * - The code `code-1` gives `access-1` and `refresh-1`. + * - The code `code-2` gives only `access-1` (no refresh token, expiry, or scope). + * - The refresh token `refresh-1` gives `access-2` (and no new refresh token). + * - `revoke(token)` makes the MCP endpoint refuse a token. + */ +export async function startProtectedServer( + options: { + /** Advertise that callbacks carry `iss` (RFC 9207), as Linear does. */ + issParameter?: boolean + } = {}, +) { + const seen = { + registrations: [] as Array>, + tokenRequests: [] as Array, + } + const valid = new Set(['access-1']) + let base = '' + const readBody = async (req: IncomingMessage) => { + const chunks: Array = [] + for await (const chunk of req) chunks.push(chunk as Buffer) + return Buffer.concat(chunks).toString('utf8') + } + const server: Server = createServer((req, res) => { + void (async () => { + const url = new URL(req.url ?? '/', base) + const json = (status: number, value: unknown) => { + res.writeHead(status, { 'Content-Type': 'application/json' }) + res.end(JSON.stringify(value)) + } + if (url.pathname.startsWith('/.well-known/oauth-protected-resource')) { + return json(200, { + resource: `${base}/mcp`, + authorization_servers: [base], + }) + } + if (url.pathname.startsWith('/.well-known/oauth-authorization-server')) { + return json(200, { + issuer: base, + authorization_endpoint: `${base}/authorize`, + token_endpoint: `${base}/token`, + registration_endpoint: `${base}/register`, + response_types_supported: ['code'], + grant_types_supported: ['authorization_code', 'refresh_token'], + code_challenge_methods_supported: ['S256'], + token_endpoint_auth_methods_supported: ['none'], + ...(options.issParameter + ? { authorization_response_iss_parameter_supported: true } + : {}), + }) + } + if (url.pathname === '/register' && req.method === 'POST') { + const metadata = JSON.parse(await readBody(req)) + seen.registrations.push(metadata) + return json(201, { ...metadata, client_id: 'client-1' }) + } + if (url.pathname === '/token' && req.method === 'POST') { + const form = new URLSearchParams(await readBody(req)) + seen.tokenRequests.push(form) + if ( + form.get('grant_type') === 'authorization_code' && + form.get('code') === 'code-1' + ) { + return json(200, { + access_token: 'access-1', + refresh_token: 'refresh-1', + token_type: 'Bearer', + expires_in: 3600, + scope: 'read write', + }) + } + if ( + form.get('grant_type') === 'authorization_code' && + form.get('code') === 'code-2' + ) { + return json(200, { access_token: 'access-1', token_type: 'Bearer' }) + } + if ( + form.get('grant_type') === 'refresh_token' && + form.get('refresh_token') === 'refresh-1' + ) { + valid.add('access-2') + return json(200, { access_token: 'access-2', token_type: 'Bearer' }) + } + return json(400, { error: 'invalid_grant' }) + } + if (url.pathname === '/mcp') { + const token = (req.headers.authorization ?? '').replace(/^Bearer /, '') + if (!valid.has(token)) { + res.writeHead(401, { + 'WWW-Authenticate': `Bearer resource_metadata="${base}/.well-known/oauth-protected-resource"`, + }) + res.end() + return + } + if (req.method !== 'POST') { + res.writeHead(405).end() + return + } + const mcp = new McpServer({ name: 'protected', version: '1.0.0' }) + mcp.registerTool( + 'echo', + { + description: 'Echo text', + inputSchema: { text: z.string() }, + annotations: { readOnlyHint: true }, + }, + async ({ text }) => ({ + content: [{ type: 'text' as const, text: `echo: ${text}` }], + }), + ) + const transport = new WebStandardStreamableHTTPServerTransport({ + sessionIdGenerator: undefined, + enableJsonResponse: true, + }) + await mcp.connect(transport) + const response = await transport.handleRequest( + new Request(url, { + method: req.method, + headers: webHeaders(req), + body: await readBody(req), + }), + ) + res.writeHead(response.status, Object.fromEntries(response.headers)) + res.end(await response.text()) + return + } + res.writeHead(404).end() + })() + }) + await new Promise((resolve) => + server.listen(0, '127.0.0.1', () => resolve()), + ) + const address = server.address() + base = `http://127.0.0.1:${typeof address === 'object' && address ? address.port : 0}` + return { + url: `${base}/mcp`, + /** The `issuer` of the authorization server metadata. */ + issuer: base, + seen, + revoke: (token: string) => valid.delete(token), + close: () => + new Promise((resolve) => { + server.closeAllConnections() + server.close(() => resolve()) + }), + } +} diff --git a/packages/ai-mcp/vite.config.ts b/packages/ai-mcp/vite.config.ts index 5b6c74c28c..a0d3ac34d4 100644 --- a/packages/ai-mcp/vite.config.ts +++ b/packages/ai-mcp/vite.config.ts @@ -31,6 +31,8 @@ export default mergeConfig( './src/apps/index.ts', './src/server/index.ts', './src/server/stdio.ts', + './src/connector.ts', + './src/harness.ts', ], srcDir: './src', cjs: false, diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 7df08d78ac..764e263bec 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -2749,6 +2749,12 @@ importers: specifier: ^6.2.12 version: 6.2.12 devDependencies: + '@tanstack/ai-harness': + specifier: workspace:* + version: link:../ai-harness + '@tanstack/ai-persistence': + specifier: workspace:* + version: link:../ai-persistence '@vitest/coverage-v8': specifier: 4.1.10 version: 4.1.10(vitest@4.1.11) diff --git a/testing/e2e/fixtures/harness/protocol-inputs.json b/testing/e2e/fixtures/harness/protocol-inputs.json index c8859ce995..4dccc0d312 100644 --- a/testing/e2e/fixtures/harness/protocol-inputs.json +++ b/testing/e2e/fixtures/harness/protocol-inputs.json @@ -248,6 +248,52 @@ "response": { "content": "Green." } + }, + { + "match": { + "userMessage": "[harness-elicit] what is the forecast", + "sequenceIndex": 0 + }, + "response": { + "toolCalls": [ + { + "name": "weather_ask_city", + "arguments": "{}" + } + ] + } + }, + { + "match": { + "userMessage": "[harness-elicit] what is the forecast", + "sequenceIndex": 1 + }, + "response": { + "content": "The forecast for Paris is sunny." + } + }, + { + "match": { + "userMessage": "[harness-elicit-2025] book a trip", + "sequenceIndex": 0 + }, + "response": { + "toolCalls": [ + { + "name": "trips_book_trip", + "arguments": "{}" + } + ] + } + }, + { + "match": { + "userMessage": "[harness-elicit-2025] book a trip", + "sequenceIndex": 1 + }, + "response": { + "content": "Your trip to Paris is booked, seat 12A." + } } ] } diff --git a/testing/e2e/src/routeTree.gen.ts b/testing/e2e/src/routeTree.gen.ts index 0a7e3fadbf..5faf38a2e9 100644 --- a/testing/e2e/src/routeTree.gen.ts +++ b/testing/e2e/src/routeTree.gen.ts @@ -106,6 +106,7 @@ import { Route as ApiMcpAppsCallRouteImport } from './routes/api.mcp-apps-call' import { Route as ApiMcpAppsChatRouteImport } from './routes/api.mcp-apps-chat' import { Route as ApiMcpAppsServerRouteImport } from './routes/api.mcp-apps-server' import { Route as ApiMcpInputServerRouteImport } from './routes/api.mcp-input-server' +import { Route as ApiMcpLegacyInputServerRouteImport } from './routes/api.mcp-legacy-input-server' import { Route as ApiMcpInputTestRouteImport } from './routes/api.mcp-input-test' import { Route as ApiMcpLifecycleTestRouteImport } from './routes/api.mcp-lifecycle-test' import { Route as ApiMcpManagedTestRouteImport } from './routes/api.mcp-managed-test' @@ -686,6 +687,11 @@ const ApiMcpInputServerRoute = ApiMcpInputServerRouteImport.update({ path: '/api/mcp-input-server', getParentRoute: () => rootRouteImport, } as any) +const ApiMcpLegacyInputServerRoute = ApiMcpLegacyInputServerRouteImport.update({ + id: '/api/mcp-legacy-input-server', + path: '/api/mcp-legacy-input-server', + getParentRoute: () => rootRouteImport, +} as any) const ApiMcpInputTestRoute = ApiMcpInputTestRouteImport.update({ id: '/api/mcp-input-test', path: '/api/mcp-input-test', @@ -1175,6 +1181,7 @@ export interface FileRoutesByFullPath { '/api/mcp-apps-chat': typeof ApiMcpAppsChatRoute '/api/mcp-apps-server': typeof ApiMcpAppsServerRoute '/api/mcp-input-server': typeof ApiMcpInputServerRoute + '/api/mcp-legacy-input-server': typeof ApiMcpLegacyInputServerRoute '/api/mcp-input-test': typeof ApiMcpInputTestRoute '/api/mcp-lifecycle-test': typeof ApiMcpLifecycleTestRoute '/api/mcp-managed-test': typeof ApiMcpManagedTestRoute @@ -1348,6 +1355,7 @@ export interface FileRoutesByTo { '/api/mcp-apps-chat': typeof ApiMcpAppsChatRoute '/api/mcp-apps-server': typeof ApiMcpAppsServerRoute '/api/mcp-input-server': typeof ApiMcpInputServerRoute + '/api/mcp-legacy-input-server': typeof ApiMcpLegacyInputServerRoute '/api/mcp-input-test': typeof ApiMcpInputTestRoute '/api/mcp-lifecycle-test': typeof ApiMcpLifecycleTestRoute '/api/mcp-managed-test': typeof ApiMcpManagedTestRoute @@ -1522,6 +1530,7 @@ export interface FileRoutesById { '/api/mcp-apps-chat': typeof ApiMcpAppsChatRoute '/api/mcp-apps-server': typeof ApiMcpAppsServerRoute '/api/mcp-input-server': typeof ApiMcpInputServerRoute + '/api/mcp-legacy-input-server': typeof ApiMcpLegacyInputServerRoute '/api/mcp-input-test': typeof ApiMcpInputTestRoute '/api/mcp-lifecycle-test': typeof ApiMcpLifecycleTestRoute '/api/mcp-managed-test': typeof ApiMcpManagedTestRoute @@ -1697,6 +1706,7 @@ export interface FileRouteTypes { | '/api/mcp-apps-chat' | '/api/mcp-apps-server' | '/api/mcp-input-server' + | '/api/mcp-legacy-input-server' | '/api/mcp-input-test' | '/api/mcp-lifecycle-test' | '/api/mcp-managed-test' @@ -1870,6 +1880,7 @@ export interface FileRouteTypes { | '/api/mcp-apps-chat' | '/api/mcp-apps-server' | '/api/mcp-input-server' + | '/api/mcp-legacy-input-server' | '/api/mcp-input-test' | '/api/mcp-lifecycle-test' | '/api/mcp-managed-test' @@ -2043,6 +2054,7 @@ export interface FileRouteTypes { | '/api/mcp-apps-chat' | '/api/mcp-apps-server' | '/api/mcp-input-server' + | '/api/mcp-legacy-input-server' | '/api/mcp-input-test' | '/api/mcp-lifecycle-test' | '/api/mcp-managed-test' @@ -2217,6 +2229,7 @@ export interface RootRouteChildren { ApiMcpAppsChatRoute: typeof ApiMcpAppsChatRoute ApiMcpAppsServerRoute: typeof ApiMcpAppsServerRoute ApiMcpInputServerRoute: typeof ApiMcpInputServerRoute + ApiMcpLegacyInputServerRoute: typeof ApiMcpLegacyInputServerRoute ApiMcpInputTestRoute: typeof ApiMcpInputTestRoute ApiMcpLifecycleTestRoute: typeof ApiMcpLifecycleTestRoute ApiMcpManagedTestRoute: typeof ApiMcpManagedTestRoute @@ -2970,6 +2983,13 @@ declare module '@tanstack/react-router' { preLoaderRoute: typeof ApiMcpInputServerRouteImport parentRoute: typeof rootRouteImport } + '/api/mcp-legacy-input-server': { + id: '/api/mcp-legacy-input-server' + path: '/api/mcp-legacy-input-server' + fullPath: '/api/mcp-legacy-input-server' + preLoaderRoute: typeof ApiMcpLegacyInputServerRouteImport + parentRoute: typeof rootRouteImport + } '/api/mcp-input-test': { id: '/api/mcp-input-test' path: '/api/mcp-input-test' @@ -3651,6 +3671,7 @@ const rootRouteChildren: RootRouteChildren = { ApiMcpAppsChatRoute: ApiMcpAppsChatRoute, ApiMcpAppsServerRoute: ApiMcpAppsServerRoute, ApiMcpInputServerRoute: ApiMcpInputServerRoute, + ApiMcpLegacyInputServerRoute: ApiMcpLegacyInputServerRoute, ApiMcpInputTestRoute: ApiMcpInputTestRoute, ApiMcpLifecycleTestRoute: ApiMcpLifecycleTestRoute, ApiMcpManagedTestRoute: ApiMcpManagedTestRoute, diff --git a/testing/e2e/src/routes/api.harness-protocol.$.ts b/testing/e2e/src/routes/api.harness-protocol.$.ts index f810e36f5a..2ada40511f 100644 --- a/testing/e2e/src/routes/api.harness-protocol.$.ts +++ b/testing/e2e/src/routes/api.harness-protocol.$.ts @@ -11,6 +11,7 @@ import { defineHarness, definePlugin, } from '@tanstack/ai-harness' +import { mcp } from '@tanstack/ai-mcp/harness' import { memoryLogStore, memoryPersistence } from '@tanstack/ai-persistence' import { z } from 'zod' import { @@ -118,6 +119,36 @@ function harnessFor(request: Request) { maxRetries: 0, }) : createTextAdapter('openai', undefined, port, testId).adapter + // `x-harness-mcp: 1` connects the `api.mcp-input-server` route. Its + // `ask_city` tool asks the user for a city by MCP elicitation. + // `x-harness-mcp: 2025` connects the spec 2025 `api.mcp-legacy-input-server` + // route. Its `book_trip` tool asks twice in one call. + const mcpMode = request.headers.get('x-harness-mcp') + const servers = + mcpMode === '1' + ? [ + mcp({ + servers: { + weather: { + type: 'http', + url: new URL('/api/mcp-input-server', request.url).href, + }, + }, + }), + ] + : mcpMode === '2025' + ? [ + mcp({ + servers: { + trips: { + type: 'http', + url: new URL('/api/mcp-legacy-input-server', request.url) + .href, + }, + }, + }), + ] + : [] const harness = defineHarness({ name: 'e2e/protocol', adapter, @@ -181,6 +212,7 @@ function harnessFor(request: Request) { commands: ['greet', 'release', 'undo'], }, plugins: () => [ + ...servers, // The sender of the running turn, as plugins see it. definePlugin({ name: 'e2e/whoami', diff --git a/testing/e2e/src/routes/api.mcp-legacy-input-server.ts b/testing/e2e/src/routes/api.mcp-legacy-input-server.ts new file mode 100644 index 0000000000..979316a9f2 --- /dev/null +++ b/testing/e2e/src/routes/api.mcp-legacy-input-server.ts @@ -0,0 +1,46 @@ +import { createFileRoute } from '@tanstack/react-router' +import { toolDefinition } from '@tanstack/ai' +import { createMCPServer } from '@tanstack/ai-mcp/server' +import type { MCPToolContext } from '@tanstack/ai-mcp/server' +import { z } from 'zod' + +/** + * A spec 2025 MCP server built with `createMCPServer`. + * + * `book_trip` asks the user twice in one call: a city, then a seat. On spec + * 2025 each question is an `elicitation/create` request to the client. + */ +const bookTrip = toolDefinition({ + name: 'book_trip', + description: 'Ask the user for a city and a seat, then book the trip', + inputSchema: z.object({}), +}).server(async (_args, ctx) => { + const city = await ctx.context.requestInput({ message: 'Which city?' }) + const seat = await ctx.context.requestInput({ message: 'Which seat?' }) + return `Booked ${String(city)}, seat ${String(seat)}` +}) + +const server = createMCPServer({ + name: 'trips-legacy', + version: '1.0.0', + tools: [bookTrip], + sessions: 'memory', +}) + +/** A spec 2026 request carries `mcp-method`. Refuse it, so the client uses 2025. */ +function handle(request: Request) { + if (request.headers.get('mcp-method') !== null) { + return new Response('This server speaks only spec 2025.', { status: 400 }) + } + return server.fetch(request) +} + +export const Route = createFileRoute('/api/mcp-legacy-input-server')({ + server: { + handlers: { + GET: ({ request }) => handle(request), + POST: ({ request }) => handle(request), + DELETE: ({ request }) => handle(request), + }, + }, +}) diff --git a/testing/e2e/tests/harness-protocol-inputs.spec.ts b/testing/e2e/tests/harness-protocol-inputs.spec.ts index 26e7c0579f..541ec3b9c4 100644 --- a/testing/e2e/tests/harness-protocol-inputs.spec.ts +++ b/testing/e2e/tests/harness-protocol-inputs.spec.ts @@ -548,6 +548,88 @@ test.describe('harness protocol inputs', () => { .toContain('The build is done.') }) + test('asks an MCP elicitation as a question and sends the answer to the server', async ({ + request, + testId, + aimockPort, + }) => { + const user = asUser(request, testId, aimockPort, 'e2e-token', { + 'x-harness-mcp': '1', + }) + const threadId = `elicit-${testId}` + expect( + await user.control(threadId, { + op: 'prompt', + message: '[harness-elicit] what is the forecast', + }), + ).toMatchObject({ status: 'accepted' }) + await expect.poll(() => user.question(threadId)).toBeTruthy() + const asked = await user.question(threadId) + expect(asked).toMatchObject({ + message: 'Which city?', + schema: { type: 'object', properties: { value: { type: 'string' } } }, + }) + + expect( + await user.control(threadId, { + op: 'answer', + questionId: asked?.questionId, + value: { value: 'Paris' }, + }), + ).toMatchObject({ status: 'accepted' }) + await expect + .poll(() => user.answers(threadId)) + .toContain('The forecast for Paris is sunny.') + + // The server got the answer: the tool result has the city. + const result = (await user.transcript(threadId)).find( + (message) => message.role === 'tool', + ) + expect(JSON.stringify(result?.content)).toContain('Forecast for Paris') + }) + + test('asks each elicitation/create request of a spec 2025 server, two in one tool call', async ({ + request, + testId, + aimockPort, + }) => { + const user = asUser(request, testId, aimockPort, 'e2e-token', { + 'x-harness-mcp': '2025', + }) + const threadId = `elicit-2025-${testId}` + expect( + await user.control(threadId, { + op: 'prompt', + message: '[harness-elicit-2025] book a trip', + }), + ).toMatchObject({ status: 'accepted' }) + for (const [message, value] of [ + ['Which city?', 'Paris'], + ['Which seat?', '12A'], + ]) { + await expect + .poll(async () => (await user.question(threadId))?.message) + .toBe(message) + const asked = await user.question(threadId) + expect( + await user.control(threadId, { + op: 'answer', + questionId: asked?.questionId, + value: { value }, + }), + ).toMatchObject({ status: 'accepted' }) + } + await expect + .poll(() => user.answers(threadId)) + .toContain('Your trip to Paris is booked, seat 12A.') + + // The server got both answers in the same tool call. + const result = (await user.transcript(threadId)).find( + (message) => message.role === 'tool', + ) + expect(JSON.stringify(result?.content)).toContain('Booked Paris, seat 12A') + }) + test('answers the stored user message on continue, with no new message', async ({ request, testId,