Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/harness-join-any-instance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
'@tanstack/ai-harness': patch
---

On a durable host (`stores.log` and `stores.runs`), any server instance can serve a run.

- `POST run` sends the log cursor of each event as its SSE `id`. A client that is cut off sends the same request with `Last-Event-ID`, and any instance continues the stream from there. The turn does not run again.
- `GET run?runId=` finds a turn that another instance runs. It streams the turn from the shared log, from its start or from `Last-Event-ID`, until the turn ends.
- `GET run?threadId=` shows a turn that another instance runs as `activeRun`, so a reloaded `useChat` joins it.
9 changes: 9 additions & 0 deletions .changeset/harness-secure-defaults.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
'@tanstack/ai-harness': patch
---

Make the transport defaults safe for a server with many instances.

- `createHarnessHandler` and `handleHarnessSocket` need `canAccess`. To let every principal use every thread, pass `canAccess: () => true`.
- On a durable host (`stores.log`), `createHarnessHandler` throws without `mediaSecret`, so a media URL that one instance signs works on every instance.
- `oauthConnector` needs `login`. Use `'device'` on a server, and `'loopback'` only for a harness on the user's own machine, such as a CLI.
5 changes: 5 additions & 0 deletions .changeset/harness-view-experimental.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tanstack/ai-harness': patch
---

Mark `createSessionView`, `SessionViewState`, and `SessionViewEvents` as `@experimental`. Document that event cursors are opaque strings: log positions on a durable host, and process-local counters without a log store.
6 changes: 4 additions & 2 deletions docs/harness/auth.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ import { openaiText } from '@tanstack/ai-openai'
const github = oauthConnector({
id: 'github',
label: 'GitHub',
// The harness runs on the user's machine (a CLI), so open a browser here.
login: 'loopback',
oauth: {
authorizationUrl: 'https://github.com/login/oauth/authorize',
tokenUrl: 'https://github.com/login/oauth/access_token',
Expand Down Expand Up @@ -54,10 +56,10 @@ export const assistant = defineHarness({
## 2. Sign in

- The user runs `/connect github`.
- The CLI opens the browser. The harness listens on `127.0.0.1` on a random port for one callback, with PKCE and a random `state`.
- With `login: 'loopback'`, the CLI opens the browser. The harness listens on `127.0.0.1` on a random port for one callback, with PKCE and a random `state`.
- The token goes into the credential store. `/disconnect github` deletes it.

Set `login: 'device'` for SSH sessions, containers, and CI. The CLI then shows a code to enter on the service's page.
`login` is required. Use `login: 'device'` for a harness on a server, and for SSH sessions, containers, and CI. The user then gets a code to enter on the service's page. Device login needs `oauth.deviceUrl`.

If a tool runs before sign-in, it stops with a `harness.auth_required` event. The CLI shows which service to connect.

Expand Down
26 changes: 23 additions & 3 deletions docs/harness/connect.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ export const handler = createHarnessHandler({
})
```

`canAccess` is required. It decides which threads each principal can use. To let every principal use every thread, pass `canAccess: () => true`.

On a durable host (`stores.log`), also set `mediaSecret`. It signs media URLs, so every instance must use the same value. Without a log store, the default is a random key, and signed URLs stop working after a restart.

The handler answers these paths under your route:

- `GET capabilities`: the AG-UI capabilities, with the agents in `expose.agents`.
Expand All @@ -51,6 +55,15 @@ The handler answers these paths under your route:
- `GET snapshot?threadId=`: the status, running operations, and waiting approvals.
- `GET agent-transcript?threadId=&subagentRunId=`: the saved messages of one child agent of the thread. See [Show what a child wrote after a reload](./subagents#show-what-a-child-wrote-after-a-reload).

## Cursors

Each event id of `GET events` and `GET run`, and the `cursor` of each `harness.event` frame, is an opaque string. Send it back as `Last-Event-ID` or `from`, as it is. Do not parse it.

- On a durable host (`stores.log`), a cursor is a position in the log of the thread. It works on every instance and after a restart.
- Without a log store, a cursor is a counter in the memory of one process. Another instance or a restart does not know it.

To run more than one instance behind one URL, use `stores.log`.

## Choose what clients can change

A client can send a prompt and answer questions. Other inputs can change how the session works, so a client can use only what the harness exposes. Nothing is exposed by default.
Expand Down Expand Up @@ -271,7 +284,7 @@ The approval reaches the harness, and the tool runs on the server. What else `PO
- The `forwardedProps` of the request are the input's [context](./inputs#send-context-with-a-message).
- With `persistence: true`, `useChat` loads the thread from `GET run?threadId=`.
- If a turn still runs after a reload, `useChat` joins it with `GET run?runId=`. The turn shows from its first event, and its answer continues to stream.
- With more than one server, send `GET run?runId=` to the server that runs the turn. Only that server has the live events of the turn.
- On a durable host (`stores.log` and `stores.runs`), each event of `POST run` has its log cursor as its id. If the connection drops, `useChat` sends the request again with `Last-Event-ID`. Any server continues the stream from there, and the turn does not run again.

To keep the thread after a reload, set `persistence: true` in `useChat`:

Expand All @@ -290,7 +303,8 @@ export function ReloadableChat() {
The join has these rules:

- `canAccess` decides who can join, the same as for every other path.
- The turn must run on the host that answers the request. On another host, the join gets `404`.
- On a durable host, any server can answer the join. It reads the turn from the shared log.
- Without a log store, the turn must run on the host that answers the request. On another host, the join gets `404`.
- A turn that ended gives the events that the session still keeps, and then the stream ends.

## Use a WebSocket
Expand All @@ -302,7 +316,13 @@ import { handleHarnessSocket } from '@tanstack/ai-harness'
import type { WebSocketLike } from '@tanstack/ai'

export function onUpgrade(socket: WebSocketLike) {
handleHarnessSocket({ host, harness: assistant, socket, principal: { id: 'user-1' } })
handleHarnessSocket({
host,
harness: assistant,
socket,
principal: { id: 'user-1' },
canAccess: (principal, threadId) => threadId.startsWith(`${principal.id}-`),
})
}
```

Expand Down
14 changes: 12 additions & 2 deletions packages/ai-harness/src/connectors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,17 @@ export interface OAuthConnectorOptions {
id: string
label: string
oauth: OAuthConfig
/** `'loopback'` (default) opens a browser here. `'device'` shows a code to enter elsewhere. */
login?: 'loopback' | 'device'
/**
* How the user signs in. Required, because only you know where the harness
* runs:
*
* - `'device'`: show a code to enter on another device. It works on a
* server, and needs `oauth.deviceUrl`.
* - `'loopback'`: open a browser on this machine and listen on
* `127.0.0.1` for the redirect. Only for a harness that runs on the
* user's own machine, such as a CLI.
*/
login: 'loopback' | 'device'
/**
* The tools of this service. Call `token()` inside a tool: it returns a
* fresh access token, or fails with `auth_required` before sign-in. With
Expand All @@ -39,6 +48,7 @@ export interface OAuthConnectorOptions {
* const github = oauthConnector({
* id: 'github',
* label: 'GitHub',
* login: 'device',
* oauth: { authorizationUrl, tokenUrl, deviceUrl, clientId, scopes: ['repo'] },
* tools: (token) => [listIssues(token)],
* })
Expand Down
15 changes: 15 additions & 0 deletions packages/ai-harness/src/host.ts
Original file line number Diff line number Diff line change
Expand Up @@ -518,6 +518,17 @@ export function sessionOfTurn(
)
}

/** The log and run stores of each durable host. Not public API. */
const durableStores = new WeakMap<object, { log: LogStore; runs?: RunStore }>()

/**
* @internal The log and run stores of `host` when it is durable
* (`stores.log`), else `undefined`.
*/
export function durableStoresOf(host: HarnessHost) {
return durableStores.get(host)
}

/** Throw a clear error for a durable store set that cannot work. */
function checkDurableStores(persistence: HarnessPersistence) {
const { stores } = persistence
Expand Down Expand Up @@ -944,6 +955,10 @@ export function createHarnessHost<TLogState = undefined>(
}
},
}
if (logStore) {
const { runs } = persistence.stores
durableStores.set(host, { log: logStore, ...(runs ? { runs } : {}) })
}
turnFinders.set(host, async (name, operationId) => {
for (const [key, opening] of sessions) {
if (!key.startsWith(`${name}\u0000`)) continue
Expand Down
Loading
Loading