Repository navigation
feat(ai-harness, ai-dashboard): a self-hosted dashboard for harness sessions - #1766
Closed
AlemTuzlak wants to merge 27 commits into
Closed
AlemTuzlak wants to merge 27 commits into
AlemTuzlak wants to merge 27 commits into
Conversation
…d client and OAuth connectors
…uth cases of auth.test.ts
…essions New package @tanstack/ai-dashboard: startDashboard and the tanstack-ai-dashboard bin serve a web app for hosts and sessions; connectDashboard (./connect) dials out from a harness host, pairs with a one-time code, and relays events and inputs. Split out of #1555.
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🦋 Changeset detectedLatest commit: 9960c0c The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
5 of 7 tasks
|
View your CI Pipeline Execution ↗ for commit 9960c0c
☁️ Nx Cloud last updated this comment at |
Contributor
Coverage✅ Coverage held across 69 compared package(s). Each package is measured twice in this job — on this PR and on its merge-base with
|
…t/harness-dashboard
…t/harness-dashboard
@tanstack/ai
@tanstack/ai-acp
@tanstack/ai-angular
@tanstack/ai-anthropic
@tanstack/ai-bedrock
@tanstack/ai-byteplus
@tanstack/ai-claude-code
@tanstack/ai-client
@tanstack/ai-cloudflare
@tanstack/ai-code-mode
@tanstack/ai-code-mode-snippets
@tanstack/ai-codex
@tanstack/ai-cohere
@tanstack/ai-compaction
@tanstack/ai-dashboard
@tanstack/ai-devtools-core
@tanstack/ai-durable-stream
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-grok-build
@tanstack/ai-groq
@tanstack/ai-harness
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-daytona
@tanstack/ai-isolate-e2b
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-isolate-quickjs-bun
@tanstack/ai-llmgateway
@tanstack/ai-lovable
@tanstack/ai-mcp
@tanstack/ai-memory
@tanstack/ai-mistral
@tanstack/ai-octane
@tanstack/ai-ollama
@tanstack/ai-ollaya
@tanstack/ai-openai
@tanstack/ai-opencode
@tanstack/ai-openrouter
@tanstack/ai-perplexity
@tanstack/ai-persistence
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-reactor
@tanstack/ai-remix
@tanstack/ai-sandbox
@tanstack/ai-sandbox-blaxel
@tanstack/ai-sandbox-boxd
@tanstack/ai-sandbox-cloudflare
@tanstack/ai-sandbox-daytona
@tanstack/ai-sandbox-docker
@tanstack/ai-sandbox-e2b
@tanstack/ai-sandbox-local-process
@tanstack/ai-sandbox-railway
@tanstack/ai-sandbox-sprites
@tanstack/ai-sandbox-upstash-box
@tanstack/ai-sandbox-vercel
@tanstack/ai-skills
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-typesafe
@tanstack/ai-utils
@tanstack/ai-vercel-gateway
@tanstack/ai-vertex
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/ai-worldlabs
@tanstack/openai-base
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
@tanstack/svelte-ai-devtools
commit: |
github-actions
Bot
added
the
waiting-on: maintainer
…an MCP server (#1763) * feat(ai-harness, ai-mcp): MCP servers in a harness, and a harness as an MCP server Adds @tanstack/ai-mcp/harness (createHarnessMcpServer and the mcp() plugin) and @tanstack/ai-mcp/connector (mcpConnector with OAuth sign-in and RFC 9207 iss). mcp() and mcpConnector ask spec 2025 elicitation/create requests and spec 2026 input rounds as session questions. Split out of #1555. * docs(ai-harness, ai-mcp): import definePlugin in the discoverTools sample
github-actions
Bot
added
merge-conflicts
A client can read the saved messages of a child agent run, as a local session does with session.agentTranscript. The route opens the session for the principal, so a child of another thread or user stays hidden.
…t/harness-transport
…any UI (#1761) * feat(ai-harness): createSessionView, one live store for a session in any UI * fix(ai-harness): import ../session/index in the view * feat(ai-harness): close the gaps between createSessionView and the harness - Actions: followUp, continue, upload, configure, cancel(operationId), cancelInput, setDelivery, reset, revert, unrevert, and ViewAgent.cancel. - Interrupts: generic interrupts get their own list with resolve and cancel. Approvals can send edited arguments or a payload, and find their tool in a child agent. - Subagents: AgentPart.result, and after a reload each child shows its saved messages and how it ended. - A view that opens while a turn runs shows the turn from its start without what the transcript has. The snapshot is read before the transcript. - A retry drops the failed attempt, turnEnd skips a turn that stops for an interrupt, usage, models, and settings are in the state, and media that no message shows stops getting new URLs. * docs(ai-harness): every view action, other interrupts, and subagents after a reload
…ness-transport # Conflicts: # docs/config.json # packages/ai-harness/tests/auth.test.ts
The client, the session view, and its source take one object for
setConfig, answer, setDelivery, sendToAgent, renameSession,
updateSession, and forkSession, as the session does. A view agent's
send takes { mode }, as AgentRun.send does. The protocol, the HTTP
handler, the connectors, and the MCP connector call the new store and
session shapes.
5 of 7 tasks
…t/harness-dashboard
github-actions
Bot
added
waiting-on: maintainer
github-actions
Bot
added
merge-conflicts
github-actions
Bot
added
waiting-on: maintainer
Contributor
Author
|
@jherr i'm closing this PR as discussed, will keep the original open until we get back to it |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With this PR, you can watch and steer harness sessions from a browser or a phone.
npx @tanstack/ai-dashboardstarts a self-hosted dashboard. A harness host callsconnectDashboard()from@tanstack/ai-dashboard/connect, dials out to it, and pairs with a one-time code. Then you approve tools, answer questions, and send prompts from the dashboard.This PR targets the transport PR #1757 (
feat/harness-transport), which stacks on the core harness PR #1755 (feat/harness-core). It is split out of #1555.🎯 Changes
New package
packages/ai-dashboard, ported from #1555:src/server.ts:startDashboard(). The HTTP server for pairing, host tokens, the host relay (hello, frames, an input stream), the owner API, and the SSE feed of each session. Inputs for an offline host wait for 10 minutes.src/ui.ts: the web app as one HTML string, with a manifest and a service worker, so a phone can install it.src/connect.ts(./connectexport):connectDashboard(). It pairs, sends session events in batches, and applies the inputs from the dashboard withparseHarnessInputandapplyInput. It pairs again after a dashboard restart whenonPairingCodeis set. WithallowRemoteStart, the dashboard can open threads.bin/dashboard.mjs: thetanstack-ai-dashboardbin (--port,--host,DASHBOARD_OWNER_TOKEN).package.json,tsconfig.json,vite.config.ts, and a README with the TanStack AI banner.@tanstack/aiand@tanstack/ai-harnessare peer dependencies.Fixes from the code review:
connectDashboardsends the event batches of a thread one at a time. Before, two batches could arrive out of order, and the text deltas of an answer joined in the wrong order.onError. Before, it was an unhandled rejection, which can stop the host process.Not changed (decisions, known limits). These are the same as in #1555. Each one is a separate follow-up.
/api/pair/starthas no rate limit. Run the dashboard behind TLS, on a network you trust.draw()rebuilds the page on each event, so text in an input box is lost while an answer streams.E2E.
testing/e2e/tests/dashboard.spec.ts: it starts a dashboard and pairs a host in the test process, then drives the web app in the browser.@tanstack/ai-dashboardis added to the e2epackage.json.Unit tests.
tests/relay.test.ts(pairing, the relay, re-pairing, remote start, and the UI script in a VM) andtests/child-agents.test.ts.Docs. New
docs/harness/dashboard.md(addedAt2026-10-13) with adocs/config.jsonentry. Readers: agent builders who want to approve and steer sessions away from the terminal. Tone: matches the existing pages. The CLI flag--dashboardandHARNESS_DASHBOARD_TOKENcome with the CLI PR, so this page and the README show onlyconnectDashboard.Update: merged transport with #1779. The merge had no conflicts and needed no code change. The dashboard speaks only the wire inputs (
{ op: 'answer', questionId, value }and the others). It does not call the session, client, or store methods that #1779 changed to one object.Changeset.
@tanstack/ai-dashboardminor (new package).✅ Checklist
pnpm run test:pr, or these tests do not apply to this pull request.docs/for this change, or this change is not user-facing.pnpm changeset), or this PR does not change a published package.🚀 Release Impact
Testing
Commands run (after the merge, Node 24.8.0):
nx run-many -t test:types test:oxlint test:lib -p @tanstack/ai-dashboard: pass (2 files, 12 tests).nx run-many -t test:types -p "examples/**" "testing/**": pass (27 projects).pnpm test:docs,kiira check docs/harness,pnpm test:knip,pnpm test:sherif: pass.pnpm test:pr.Manual test.
pnpm --filter @tanstack/ai-dashboard build, thennode packages/ai-dashboard/bin/dashboard.mjs.connectDashboard({ host, harness, url: 'http://127.0.0.1:8790', threads: ['main'], onPairingCode: console.log }).mainfrom the page.How this PR makes testing easy. Unit tests in
packages/ai-dashboard/tests, and the E2E specdashboard.spec.ts.Risk / rollback
Low. It is a new package. The only other changes are a docs page, a docs nav entry, and an e2e dependency. Revert the PR to undo.
Public API change
Before
// No dashboard package.After