diff --git a/.claude/skills/verify-berthd/features/README.md b/.claude/skills/verify-berthd/features/README.md index 4db31eba..076c4960 100644 --- a/.claude/skills/verify-berthd/features/README.md +++ b/.claude/skills/verify-berthd/features/README.md @@ -19,6 +19,7 @@ The maintained list of what a person can do in the app, and how to prove each on | Home's widget grid | [home-widgets.md](home-widgets.md) | none yet; `app/e2e/home-widgets.spec.ts` asserts it | | Questions in chat: an agent's form of questions as a card, answered on a real box; kept with Retry and Copy when it can't be | [chat-questions.md](chat-questions.md) | `chat-questions-live` (a real box, `--agent`); `app/e2e/chat.spec.ts` on the mock | | Chat and board: no question card or answers but the options the agent's screen shows; the mode menu while turn check waits | [chat-ask-card.md](chat-ask-card.md) | `chat-ask-card` (a real box, `--agent`); `app/e2e/ask-card.spec.ts` on a stand-in box | +| The board's cards for agents in chat: what they ask with its answers, their reply's last lines, the prompt they started with; never the host's frames or "stdio" | [chat-board-cards.md](chat-board-cards.md) | `chat-board-cards`; `app/e2e/dashboard-chat-cards.spec.ts` asserts it | | Turn check's "working": an hourglass with why in the sidebar, the tab and the rail's card, not a live turn's spinner; a handoff reads as done | [turn-check-indicator.md](turn-check-indicator.md) | `turn-check-indicator`; `app/e2e/sidebar-turn-check.spec.ts` asserts it, `TestTurnCheckLive` asks the real provider | ## Proof and skip reporting diff --git a/.claude/skills/verify-berthd/features/chat-board-cards.md b/.claude/skills/verify-berthd/features/chat-board-cards.md new file mode 100644 index 00000000..830070bf --- /dev/null +++ b/.claude/skills/verify-berthd/features/chat-board-cards.md @@ -0,0 +1,25 @@ +# The board's cards for agents in chat + +An agent that runs in chat (structured: `Session.driver` is `claude`, `codex` or `acp`) has no screen of its own: its pane holds the host that runs it (`berthd agent-host`), which logs one line per frame (`out stream_event`, `out control_request`), and its command line is the host's, ending in `'--permission-prompt-tool' 'stdio'`. So its card on the Agent Dashboard reads the agent instead (`views/dashboard/agent-card.tsx`, `use-chat-tail.ts`, `lib/chat-card.ts`): + +- **Waiting on you**: what it asks, from `Session.ask`. A permission shows its reason and the command (`$ pnpm …`) or the tool and file (`Edit src/x.ts`), with the answers it offers (`ask.choices`) as chips. A question (AskUserQuestion, ExitPlanMode) shows the question or its message, with chips only where it offers choices. +- **Otherwise**: the last lines of its newest reply, without Markdown's marks, or the step it took since (`Read search.ts`, `Edit src/x.ts +9 −2`), or a notice (a usage limit). It reads the conversation as a chat does (`GET sessions/{name}/transcript?since=N&gen=…`): the recent window once, then only what was added, which the box reads on from where it was (about 8 ms on a 30 MB record), keeping the newest 12 items. It reads when a terminal card would read its screen (on a state change, an event about it, and every 20s while it works), one read at a time, and not while the card shows an ask. +- **Hidden** when there's nothing yet (no ask, no reply, or a prompt it hasn't answered), and in Ready as before. Never a host frame. +- **The quote** under the title is its conversation's first prompt, or the command typed as one (`/ship ACME-12`), hidden when it repeats the title. Never "stdio". It is read once per session from the page at the record's start (`transcript?before=262144&limit=50`, paging back to the start when that holds more): the first prompt comes after the agent's opening frame, tens of KB in. A session's conversation list entry (`GET conversations`) has no title when its prompt was sent after it started, which is how orchestrators start agents, so it isn't used. + +A terminal agent's card is unchanged: the last 3 meaningful lines of its screen, its menu's options as chips (see [chat-ask-card.md](chat-ask-card.md) `board-answers`). + +## How to get to it (user POV) + +- ⌘J (or Home with Labs off) opens the Agent Dashboard. Cards sit in Needs you, Working, Idle and Done. + +## Driving it with drive.mjs + +- On the mock: `scenarios/chat-board-cards.mjs` opens `?mock=1&structured=1`. There, devl's search-perf Claude (Done) and gpu's judge-v2 (Working), shop (Needs you: AskUserQuestion) and ci-flake (Needs you: a Bash permission with Yes / Yes, and don't ask again / No) run in chat, with the host's command line and pane log as a real box has them. checkout-fix's Claude stays in its terminal. +- It asserts each chat card's box and quote, the chips on the permission and none on the question, the terminal card's screen and chips, and that nowhere on the board shows `out stream_event`, `out control_request`, `out system`, `in user` or `stdio`. Screenshots: `1-waiting-permission` … `5-terminal`, `6-board`. +- `app/e2e/dashboard-chat-cards.spec.ts` asserts the same on the mock. + +## Gotchas + +- The mock's Review fixtures (`lib/mock-review.ts`) answer `screen` for search-perf and checkout-fix before the mock's own screens do, so search-perf's pane reads as a Claude screen there, not a host log. Its card doesn't read the screen, so this doesn't show on the board. +- Review's summary (`views/review/summary.ts`) still reads the pane for agents in chat. diff --git a/.claude/skills/verify-berthd/scripts/scenarios/chat-board-cards.mjs b/.claude/skills/verify-berthd/scripts/scenarios/chat-board-cards.mjs new file mode 100644 index 00000000..3eea78a9 --- /dev/null +++ b/.claude/skills/verify-berthd/scripts/scenarios/chat-board-cards.mjs @@ -0,0 +1,73 @@ +// The Agent Dashboard's cards for agents that run in chat +// (features/chat-board-cards.md): their pane holds only the host that runs +// them, so a card shows what the agent asks, or the last lines of its +// reply, and the prompt it started with. Never the host's frames +// ("out stream_event"), and never "stdio". A terminal agent's card still +// shows the last lines of its screen. +// +// On the mock (?structured: search-perf's and three of gpu's Claudes run in +// chat; the rest in their terminals): +// +// node drive.mjs chat-board-cards --port $PORT + +export const params = { structured: "1" }; + +const HOST = /out stream_event|out control_request|out system|in user|stdio/; + +export async function run({ page, shot, step, expect }) { + const board = page.getByTestId("dashboard-board"); + const card = (title) => board.locator("article", { hasText: title }); + const box = (c) => c.locator(".font-mono.text-tag"); + + step("open the Agent Dashboard"); + await page.keyboard.press("Meta+j"); + await expect(page.getByRole("heading", { name: "Agent Dashboard" })).toBeVisible(); + await expect(board).toBeVisible(); + + step("in chat, waiting on a command: the command and why, its answers as chips, the prompt it started with"); + const permission = card("Move acme billing to the new ledger"); + await expect(permission).toBeVisible(); + await expect(box(permission)).toContainText("Backfill the new ledger from this year's invoices"); + await expect(box(permission)).toContainText("$ pnpm ledger:backfill --since 2025-01-01"); + for (const label of ["Yes", "No"]) await expect(permission.getByRole("button", { name: new RegExp(`^\\d\\s*${label}$`) })).toBeVisible(); + // Its conversation's first prompt, not the host's last word. + await expect(permission.getByText(/^“Move acme billing onto the new ledger\. Send helpers to map every ledger write/)).toBeVisible(); + await expect(permission).not.toContainText(HOST); + await permission.scrollIntoViewIfNeeded(); + await shot("1-waiting-permission"); + + step("in chat, asking a question: the question, no made-up answers"); + const question = card("Plan the checkout release"); + await expect(box(question)).toContainText("Three questions about the release window"); + await expect(question.getByRole("button", { name: /^\d\s/ })).toHaveCount(0); + await expect(question).not.toContainText(HOST); + await question.scrollIntoViewIfNeeded(); + await shot("2-waiting-question"); + + step("in chat, working: the last lines of what it said"); + const working = card("Tune the judge prompt"); + await expect(box(working)).toContainText("0.82 agreement with the labels, was 0.74"); + await expect(box(working)).toContainText("Next: rerun the hard set with a length penalty."); + await expect(working).not.toContainText(HOST); + await working.scrollIntoViewIfNeeded(); + await shot("3-working"); + + step("in chat, done: its last reply"); + const done = card("Speed up product search"); + await expect(box(done)).toContainText("Within budget: /search p95 is 410 ms against 500 ms."); + await expect(done).not.toContainText(HOST); + await done.scrollIntoViewIfNeeded(); + await shot("4-done"); + + step("in its terminal, waiting: unchanged, the last lines of its screen and its menu's chips"); + const terminal = card("Fix checkout webhook retries"); + await expect(box(terminal)).toContainText("the order lock for the whole backoff."); + await expect(terminal.getByRole("button", { name: /^1\s*Yes, cap it at 30s$/ })).toBeVisible(); + await terminal.scrollIntoViewIfNeeded(); + await shot("5-terminal"); + + step("nowhere on the board: a host frame or stdio"); + await expect(board).not.toContainText(HOST); + await page.evaluate(() => document.querySelector("[data-testid=dashboard-board]")?.scrollTo(0, 0)); + await shot("6-board"); +} diff --git a/app/e2e/dashboard-chat-cards.spec.ts b/app/e2e/dashboard-chat-cards.spec.ts new file mode 100644 index 00000000..d947a347 --- /dev/null +++ b/app/e2e/dashboard-chat-cards.spec.ts @@ -0,0 +1,52 @@ +import { expect, mockOnly, test } from "./fixtures"; + +// An agent that runs in chat (structured) has only the host that runs it in +// its pane, one line per frame ("out stream_event"), and the host's command +// line ends in '--permission-prompt-tool' 'stdio'. Its board card shows what +// it asks, or the last lines of its reply, and the prompt it started with: +// never those frames, and never "stdio" (lib/chat-card.ts). A terminal +// agent's card still shows its screen. + +const HOST = /out stream_event|out control_request|out assistant|stdio/; + +async function openBoard(app: import("./fixtures").App) { + mockOnly("the agents in chat are mock fixtures (?structured)"); + await app.open({ prefs: { labs: false, labsChosen: true, version: 3 }, params: { structured: "1" } }); + await app.page.getByTestId("nav-home").click(); + return { + needsYou: app.page.getByRole("region", { name: "Needs you" }), + working: app.page.getByRole("region", { name: "Working" }), + }; +} + +test("a chat card waiting on you shows what it asks, its answers, and no host frames", async ({ app }) => { + const { needsYou } = await openBoard(app); + const permission = needsYou.locator("article", { hasText: "Move acme billing to the new ledger" }); + await expect(permission).toContainText("Backfill the new ledger from this year's invoices"); + await expect(permission).toContainText("$ pnpm ledger:backfill --since 2025-01-01"); + await expect(permission.getByRole("button", { name: /^1\s*Yes$/ })).toBeVisible(); + await expect(permission.getByRole("button", { name: /^3\s*No$/ })).toBeVisible(); + // Its conversation's first prompt, not the host's last word. + await expect(permission.getByText(/^“Move acme billing onto the new ledger\. Send helpers to map every ledger write/)).toBeVisible(); + // A form of questions is answered in its chat: the question, no chips. + const question = needsYou.locator("article", { hasText: "Plan the checkout release" }); + await expect(question).toContainText("Three questions about the release window"); + await expect(question.getByRole("button", { name: /^\d\s/ })).toHaveCount(0); + for (const card of [permission, question]) await expect(card).not.toContainText(HOST); +}); + +test("a chat card at work shows the last lines of its reply", async ({ app }) => { + const { working } = await openBoard(app); + const card = working.locator("article", { hasText: "Tune the judge prompt" }); + await expect(card).toContainText("0.82 agreement with the labels, was 0.74"); + await expect(card).toContainText("Next: rerun the hard set with a length penalty."); + await expect(card).not.toContainText(HOST); +}); + +test("a terminal card beside them still shows its screen and its menu", async ({ app }) => { + const { needsYou } = await openBoard(app); + const card = needsYou.locator("article", { hasText: "Fix checkout webhook retries" }); + await expect(card).toContainText("the order lock for the whole backoff."); + await expect(card.getByRole("button", { name: /^1\s*Yes, cap it at 30s$/ })).toBeVisible(); + await expect(card.getByRole("button", { name: /^2\s*No, leave it$/ })).toBeVisible(); +}); diff --git a/app/src/lib/chat-card.test.ts b/app/src/lib/chat-card.test.ts new file mode 100644 index 00000000..2f4dee48 --- /dev/null +++ b/app/src/lib/chat-card.test.ts @@ -0,0 +1,112 @@ +// node --experimental-strip-types --test src/lib/chat-card.test.ts (pnpm test) +import assert from "node:assert/strict"; +import { test } from "node:test"; + +import { chatCard, mergeTail, openingPrompt, shownPrompt } from "./chat-card.ts"; +import type { TranscriptItem } from "./transcript.ts"; + +const YES_NO = [ + { key: "1", label: "Yes" }, + { key: "2", label: "Yes, and don't ask again this session" }, + { key: "3", label: "No" }, +]; + +const reply: TranscriptItem[] = [ + { kind: "user", id: "u1", text: "Speed up product search" }, + { kind: "tools", id: "t1", verb: "Read", done: true, items: [{ verb: "Read", target: "search.ts", file: true }] }, + { kind: "text", id: "x1", text: "## Result\n\nSearch is faster now:\n\n- p95 is **410 ms**, was 660 ms\n- all 214 tests pass\n\n```ts\nconst cache = new Map();\n```\n\nThe branch is ready for review." }, +]; + +test("a permission it waits on shows the tool, its input and why, with its answers", () => { + const card = chatCard({ tool: "Bash", input: "pnpm db:migrate --name add-key", why: "Create the migration", choices: YES_NO }, true, reply); + assert.deepEqual(card, { tail: ["Create the migration", "$ pnpm db:migrate --name add-key"], choices: YES_NO }); + assert.deepEqual(chatCard({ tool: "Edit", input: "src/search.ts", choices: YES_NO }, true, reply)?.tail, ["Edit src/search.ts"]); +}); + +test("a question it waits on shows the question, or the message it showed", () => { + assert.deepEqual(chatCard({ tool: "AskUserQuestion", input: "Which release window?" }, true, reply), { tail: ["Which release window?"], choices: [] }); + const plan = chatCard({ tool: "ExitPlanMode", message: "Claude has a plan and is ready to start", choices: YES_NO }, true, reply); + assert.deepEqual(plan, { tail: ["Claude has a plan and is ready to start"], choices: YES_NO }); + assert.deepEqual(chatCard({ tool: "AskUserQuestion" }, true, [])?.tail, ["Asks you a question"]); +}); + +test("not waiting, it shows the last lines of its newest reply, without markdown's marks", () => { + assert.deepEqual(chatCard(undefined, false, reply), { tail: ["- all 214 tests pass", "const cache = new Map();", "The branch is ready for review."], choices: [] }); + assert.deepEqual(chatCard(undefined, false, reply, 2)?.tail, ["const cache = new Map();", "The branch is ready for review."]); + // An ask left over from a wait that ended is not shown. + assert.deepEqual(chatCard({ tool: "Bash", input: "ls" }, false, reply)?.tail.at(-1), "The branch is ready for review."); +}); + +test("waiting with no ask (a question in its last message) shows that message", () => { + const asked: TranscriptItem[] = [...reply.slice(0, 2), { kind: "text", id: "x2", text: "Two ways to do this.\n\nShould I start with the cache?" }]; + assert.deepEqual(chatCard(undefined, true, asked)?.tail, ["Two ways to do this.", "Should I start with the cache?"]); +}); + +test("a step newer than its last words is what it shows", () => { + const working: TranscriptItem[] = [ + ...reply, + { kind: "tools", id: "t2", verb: "Run", items: [{ verb: "Read", target: "search.test.ts", file: true }, { verb: "Run", target: "pnpm test search" }] }, + { kind: "thinking", id: "th", since: 0 }, + ]; + assert.deepEqual(chatCard(undefined, false, working)?.tail, ["Read search.test.ts", "Run pnpm test search"]); + const edited: TranscriptItem[] = [...reply, { kind: "edit", id: "e1", file: "src/search.ts", added: 9, removed: 2 }]; + assert.deepEqual(chatCard(undefined, false, edited)?.tail, ["Edit src/search.ts +9 −2"]); + assert.deepEqual(chatCard(undefined, false, [{ kind: "tools", id: "t3", verb: "Thinking" }])?.tail, ["Thinking"]); +}); + +test("nothing to show hides the box: no reply yet, or a prompt it hasn't answered", () => { + assert.equal(chatCard(undefined, false, undefined), undefined); + assert.equal(chatCard(undefined, false, []), undefined); + assert.equal(chatCard(undefined, false, [{ kind: "user", id: "u1", text: "Fix the cart badge" }]), undefined); + // A new prompt after a reply: the reply is the last turn's, not this one's. + assert.equal(chatCard(undefined, false, [...reply, { kind: "user", id: "u2", text: "Now the tests" }, { kind: "thinking", id: "th", since: 0 }]), undefined); + assert.equal(chatCard(undefined, false, [{ kind: "text", id: "x", text: "\n\n```\n```\n" }]), undefined); +}); + +test("a notice newer than its reply (a usage limit, an API error) is what it shows", () => { + const limited: TranscriptItem[] = [...reply, { kind: "notice", id: "n", notice: "limit", level: "warning", text: "Usage limit reached. It resets at 3pm." }]; + assert.deepEqual(chatCard(undefined, false, limited)?.tail, ["Usage limit reached. It resets at 3pm."]); +}); + +test("the prompt shows unless it repeats the title, and only when there is one", () => { + assert.equal(shownPrompt("/ship ACME-12", "Fix checkout webhook retries"), "/ship ACME-12"); + assert.equal(shownPrompt("/ship ACME-12", undefined), "/ship ACME-12"); + assert.equal(shownPrompt("Fix checkout webhook retries", "Fix checkout webhook retries"), undefined); + // Titles are cut short with an ellipsis; the prompt it began is no news. + assert.equal(shownPrompt("Pick up search-perf: read the issue and get it to green.", "Pick up search-perf: read the issue and get it…"), undefined); + assert.equal(shownPrompt(undefined, "Fix checkout"), undefined); + assert.equal(shownPrompt(" ", "Fix checkout"), undefined); +}); + +test("the prompt it started with is its first prompt, or the command typed as one", () => { + assert.equal(openingPrompt(reply), "Speed up product search"); + assert.equal(openingPrompt([{ kind: "command", id: "c", command: "/ship", args: "ACME-12" }, ...reply]), "/ship ACME-12"); + assert.equal(openingPrompt([{ kind: "command", id: "c", command: "/clear" }]), "/clear"); + // Read from the middle of a long conversation, its start isn't there. + assert.equal(openingPrompt(reply, true), undefined); + assert.equal(openingPrompt([]), undefined); + assert.equal(openingPrompt(undefined), undefined); +}); + +test("a question over several lines shows its first lines, one each", () => { + const ask = { tool: "AskUserQuestion", input: "Yes, both, as two issues:\n1. **Shipyard**: shortcuts for your layout.\n2. Lookout: shorter descriptions.\n3. A third." }; + assert.deepEqual(chatCard(ask, true, [])?.tail, ["Yes, both, as two issues:", "1. Shipyard: shortcuts for your layout.", "2. Lookout: shorter descriptions."]); + assert.deepEqual(chatCard({ tool: "Bash", input: "make test", why: "Run the tests\nall of them" }, true, [])?.tail, ["Run the tests", "$ make test"]); +}); + +test("a long or pasted opening prompt is its first line, cut short", () => { + const pasted = `[scrawl 0:57 · Shipyard] ${"so right now here ".repeat(30)}\n#2 the second part`; + const p = openingPrompt([{ kind: "user", id: "u", text: pasted }]); + assert.ok(p && p.length <= 200 && p.endsWith("…") && !p.includes("#2"), String(p)); + assert.equal(openingPrompt([{ kind: "user", id: "u", text: "\n Fix the cart badge\nand its test" }]), "Fix the cart badge"); +}); + +test("a card's newest items: read again replace their old selves, new go after, the oldest go", () => { + const open: TranscriptItem = { kind: "tools", id: "t", verb: "Run", items: [{ verb: "Run", target: "pnpm test" }] }; + const done: TranscriptItem = { ...open, done: true, items: [{ verb: "Run", target: "pnpm test" }, { verb: "Read", target: "a.ts" }] }; + const said: TranscriptItem = { kind: "text", id: "x", text: "Done." }; + assert.deepEqual(mergeTail([reply[0], open], [done, said], 3), [reply[0], done, said]); + assert.deepEqual(mergeTail([reply[0], open], [done, said], 2), [done, said]); + assert.deepEqual(mergeTail([], reply, 12), reply); + assert.deepEqual(mergeTail(reply, [], 12), reply); +}); diff --git a/app/src/lib/chat-card.ts b/app/src/lib/chat-card.ts new file mode 100644 index 00000000..b15b08ce --- /dev/null +++ b/app/src/lib/chat-card.ts @@ -0,0 +1,112 @@ +import type { Ask } from "@/lib/api"; +import type { Choice } from "@/lib/screen"; +import type { TranscriptItem } from "@/lib/transcript"; + +// What a board card shows of an agent that runs in chat (structured): its +// pane holds only the host that runs it, never a screen of the agent's own, +// so the card reads the agent's ask and its conversation instead. Pure, so +// the card and its tests share it. + +// A question's tools: answered in the agent's own form, not with Allow or +// Deny, so a card shows what they ask rather than a command. +export const QUESTION_TOOLS = /^(AskUserQuestion|request_user_input|ExitPlanMode)$/; + +// chatCard is the card's box: what it asks while it waits, with the answers +// it offers; otherwise the last lines of its newest reply, or the step it +// took since. Undefined hides the box: nothing asked, nothing said yet. +export function chatCard(ask: Ask | undefined, waiting: boolean, items: TranscriptItem[] | undefined, count = 3): { tail: string[]; choices: Choice[] } | undefined { + if (asksIn(ask, waiting)) return { tail: askLines(ask, count), choices: ask.choices ?? [] }; + const tail = latestLines(items ?? [], count); + return tail.length ? { tail, choices: [] } : undefined; +} + +// asksIn says the card shows the ask, and so needs nothing of its +// conversation. +export function asksIn(ask: Ask | undefined, waiting: boolean): ask is Ask { + return waiting && !!ask && !!(ask.tool || ask.input || ask.message); +} + +// openingPrompt is what a conversation was started with: its first prompt, +// or the command typed as one ("/ship ACME-12"). items are a page from +// its start; more says the start is before them, so it isn't there. +export function openingPrompt(items: TranscriptItem[] | undefined, more?: boolean): string | undefined { + if (more) return undefined; + const first = items?.find((it) => it.kind === "user" || it.kind === "command"); + const text = first?.kind === "user" ? first.text : first?.kind === "command" ? [first.command, first.args].filter(Boolean).join(" ") : ""; + const line = text.split("\n").map((l) => l.trim()).find(Boolean); + if (!line) return undefined; + return line.length > PROMPT_MAX ? `${line.slice(0, PROMPT_MAX - 1)}…` : line; +} + +// The most of an opening prompt a card quotes: one line it truncates, and +// a pasted page's whole text is no use there. +const PROMPT_MAX = 200; + +// mergeTail is a card's newest items after a read of what was added since +// the last: items read again (a tool group still open) replace their old +// selves, new ones go after, and only the last max are kept. +export function mergeTail(held: TranscriptItem[], read: TranscriptItem[], max: number): TranscriptItem[] { + const again = new Map(read.map((it) => [it.id, it])); + const have = new Set(held.map((it) => it.id)); + return [...held.map((it) => again.get(it.id) ?? it), ...read.filter((it) => !have.has(it.id))].slice(-max); +} + +// askLines is a wait in words: a permission's reason and the command or +// file it is for; a question's first lines, or the message it showed. +function askLines({ tool, input, why, message }: Ask, count: number): string[] { + if (!tool || QUESTION_TOOLS.test(tool)) return plainLines(input || message || (tool ? "Asks you a question" : "Waiting for you")).slice(0, count); + const what = !input ? tool : tool === "Bash" ? `$ ${input}` : `${tool} ${input}`; + const reason = why && plainLines(why)[0]; + return reason ? [reason, what] : [what]; +} + +// latestLines reads back from the newest item to the last thing the agent +// said or did in this turn. A prompt after it starts another turn, which +// has nothing to show yet. +function latestLines(items: TranscriptItem[], count: number): string[] { + for (let i = items.length - 1; i >= 0; i--) { + const it = items[i]; + switch (it.kind) { + case "user": + case "report": + case "agent-message": + return []; + case "text": { + const lines = plainLines(it.text); + if (lines.length) return lines.slice(-count); + break; + } + case "tools": + return it.items?.length ? it.items.slice(-count).map((c) => `${c.verb} ${c.target}`.trim()) : [it.verb]; + case "edit": + return [`Edit ${it.file} +${it.added} −${it.removed}`]; + case "notice": + return plainLines(it.text).slice(0, count); + } + } + return []; +} + +// plainLines is Markdown as a card's few lines: no blank lines, fences, +// table rules or emphasis marks. +function plainLines(text: string): string[] { + return text + .split("\n") + .map((l) => + l + .trim() + .replace(/^#{1,6}\s+/, "") + .replace(/^>\s?/, "") + .replace(/\*\*|__/g, "") + .replace(/`([^`]+)`/g, "$1"), + ) + .filter((l) => l && !/^(```|~~~)/.test(l) && !(l.includes("|") && /^[\s:|-]+$/.test(l)) && !/^([-*_])(\s*\1){2,}$/.test(l)); +} + +// shownPrompt is the prompt a card quotes, unless it only repeats the +// title (titles are a prompt's first words, cut with an ellipsis). +export function shownPrompt(prompt: string | undefined, title: string | undefined): string | undefined { + const p = prompt?.replace(/\s+/g, " ").trim(); + if (!p || (title && p.startsWith(title.replace(/…$/, "")))) return undefined; + return p; +} diff --git a/app/src/lib/mock-conversation.ts b/app/src/lib/mock-conversation.ts index 5aff7b16..2211bbb5 100644 --- a/app/src/lib/mock-conversation.ts +++ b/app/src/lib/mock-conversation.ts @@ -264,6 +264,9 @@ export function seedTranscript(box: string, session: string, state: string, work items.push({ kind: "ask", id: id(), tool: "Bash", detail: "pnpm db:migrate --name add-idempotency-key", why: "Create the migration for the new idempotency_key column", structured: true, choices: PERMISSION }); } if (state === "running") { + // ?structured: the judge's Claude, in chat, has said what it found. + if (session === "judge-v2-claude" && new URLSearchParams(location.search).has("structured")) + items.push({ kind: "text", id: id(), text: "The judge over-scores short answers on the hard set.\n\n- **0.82** agreement with the labels, was 0.74\n- Next: rerun the hard set with a length penalty." }); items.push({ kind: "crew", id: id(), names: ["Explore: failing tests", "Explore: fixtures"] }); items.push({ kind: "thinking", id: id(), since: Date.now() - 12_000 }); const t = Date.now(); diff --git a/app/src/lib/mock-history.ts b/app/src/lib/mock-history.ts index e6408eb0..5d334676 100644 --- a/app/src/lib/mock-history.ts +++ b/app/src/lib/mock-history.ts @@ -3,6 +3,8 @@ import { keyOf, useConversations } from "@/lib/conversation-store"; import type { Helper, TranscriptPage } from "@/lib/history"; import type { ToolDetail, TranscriptItem } from "@/lib/transcript"; import { benchOlder } from "@/lib/mock-bench"; +import { seedTranscript } from "@/lib/mock-conversation"; +import { sessionState } from "@/lib/derive"; // The demo's stand-in for the box's history endpoints (?mock=1): older // turns of a long chat (?long=5000 seeds one, for measuring), the helpers' @@ -71,6 +73,20 @@ function olderPage(key: string, before: number, limit: number): TranscriptPage { return { source: "claude", items, next: n, more: start > 0 }; } +// demoConversation is a session's conversation in the demo: what its chat +// holds, or made up as its chat would when no one has opened it. +function demoConversation(box: string, s: Session): TranscriptItem[] { + return useConversations.getState().items[keyOf(box, s.name)] ?? seedTranscript(box, s.name, sessionState(s), s.location?.split("/").pop() ?? s.name); +} + +// demoPage is a page of a session's conversation as the box reads one +// (before=0 is its end), its items placed 1,000 bytes apart. +function demoPage(box: string, s: Session, before: number, limit: number): TranscriptPage { + const all = demoConversation(box, s); + const items = all.map((it, i) => ({ ...it, off: (i + 1) * 1000 })).filter((it) => before <= 0 || it.off < before); + return { source: "claude", items: items.slice(-limit), next: all.length, more: items.length > limit }; +} + // ---- Helpers --------------------------------------------------------------- function helpersOf(key: string): Helper[] { @@ -128,7 +144,23 @@ export function mockHistoryCall( const key = keyOf(box, session); if (method === "GET" && rest.startsWith("transcript?") && rest.includes("before=")) { const q = new URLSearchParams(rest.split("?")[1]); - return wait(260).then(() => olderPage(key, Number(q.get("before")), Number(q.get("limit") || 200))); + const before = Number(q.get("before")); + const limit = Number(q.get("limit") || 200); + if (!long.has(key) && !benchOlder(key, before, limit)) { + const s = st.sessions[box]?.find((x) => x.name === session); + if (!s) return Promise.reject(new Error("That session isn't in the demo.")); + return wait(120).then(() => demoPage(box, s, before, limit)); + } + return wait(260).then(() => olderPage(key, before <= 0 ? Infinity : before, limit)); + } + // A board card's read of a chat-mode agent (views/dashboard/use-chat-tail): + // the whole demo conversation as its window, then what was added since. + if (method === "GET" && rest.startsWith("transcript?") && rest.includes("since=")) { + const s = st.sessions[box]?.find((x) => x.name === session); + if (!s) return Promise.reject(new Error("That session isn't in the demo.")); + const all = demoConversation(box, s); + const since = Number(new URLSearchParams(rest.split("?")[1]).get("since")) || 0; + return wait(120).then(() => ({ source: "claude", items: all.slice(since), next: all.length, crew: [], gen: "demo", start: 0 })); } if (method === "GET" && rest === "subagents") return wait(150).then(() => ({ helpers: helpersOf(key) })); const sub = /^subagents\/([^/]+)\/(transcript|tool\/(.+))/.exec(rest); diff --git a/app/src/lib/mock.ts b/app/src/lib/mock.ts index a30d5ced..6e92c74b 100644 --- a/app/src/lib/mock.ts +++ b/app/src/lib/mock.ts @@ -223,11 +223,37 @@ const stats: Record = { // ?mock=1&crowd=1: many more agents, on more boxes (lib/mock-crowd.ts). if (new URLSearchParams(location.search).has("crowd")) crowd({ status, locations, sessions, stats }); // ?structured: search-perf's Claude runs structured (Labs), its chat's -// /resume the app's own picker of Claude's past conversations there. +// /resume the app's own picker of Claude's past conversations there. On +// gpu, the working Claude, the one asking questions and ci-flake's (waiting +// on a command) run in chat too, for the board's cards. const structuredDemo = new URLSearchParams(location.search).has("structured"); if (structuredDemo) { + // Its command line is the host's, which runs Claude Code over its stream + // (internal/box/claudedriver.go): no prompt in it. + const inChat = (x: Session) => + Object.assign(x, { + driver: "claude", + command: `'/home/me/.local/bin/berthd' 'agent-host' --dir '/home/me/.berth/conversations/${x.name}' --socket '/home/me/.berth/conversations/${x.name}/host.sock' -- claude '-p' '--input-format' 'stream-json' '--output-format' 'stream-json' '--verbose' '--include-partial-messages' '--replay-user-messages' '--permission-mode' 'default' '--permission-prompt-tool' 'stdio'`, + }); const s = sessions.devl?.find((x) => x.name === "search-perf-claude"); - if (s) s.driver = "claude"; + if (s) inChat(s); + for (const x of sessions.gpu ?? []) if (["judge-v2-claude", "shop-claude", "ci-flake-claude"].includes(x.name)) inChat(x); + const ci = sessions.gpu?.find((x) => x.name === "ci-flake-claude"); + if (ci) + Object.assign(ci, { + agent_state: "waiting", + state_since: ago(3), + ask: { + tool: "Bash", + input: "pnpm ledger:backfill --since 2025-01-01", + why: "Backfill the new ledger from this year's invoices", + choices: [ + { key: "1", label: "Yes" }, + { key: "2", label: "Yes, and don't ask again this session" }, + { key: "3", label: "No" }, + ], + }, + } satisfies Partial); } // ?mock=1&bench=…: far more than the fixtures, for measuring (lib/mock-bench.ts). if (BENCH === "fleet") benchFleet({ status, locations, sessions, stats }); @@ -783,8 +809,11 @@ function boxCall(box: string, method: string, path: string, body?: unknown): Pro if (method === "GET" && /^sessions\/[^/]+\/screen/.test(path)) { const name = decodeURIComponent(path.split("/")[1]); const s = sessions[box]?.find((x) => x.name === name); - const screen = - s?.agent_state === "waiting" + // In chat, the pane holds the host that runs the agent: its log of + // the frames it passes, one line each (internal/agenthost). + const screen = s?.driver + ? ["— started claude", "in user", "out system init", "out stream_event", "out stream_event", "out assistant", "out stream_event", s.agent_state === "waiting" ? "out control_request" : "out result success"].join("\n") + : s?.agent_state === "waiting" ? "● The fix needs a migration for the new idempotency_key column.\n\n Do you want me to create it?\n ❯ 1. Yes\n 2. No, and tell Claude what to do differently" : s?.agent_state === "finished" ? "● Search results render 38% faster on the slow-network profile.\n All 214 tests pass.\n\n✻ Baked for 6m 41s · done" diff --git a/app/src/views/dashboard/agent-card.tsx b/app/src/views/dashboard/agent-card.tsx index 099edb35..7794bfb1 100644 --- a/app/src/views/dashboard/agent-card.tsx +++ b/app/src/views/dashboard/agent-card.tsx @@ -9,13 +9,16 @@ import { Checkbox } from "@/components/ui/checkbox"; import { toastManager } from "@/components/ui/toast"; import type { SessionEntry } from "@/hooks/use-agent-counts"; import { boxApi } from "@/lib/api"; +import { asksIn, chatCard, shownPrompt } from "@/lib/chat-card"; import { agentOf, atQuestion, ON_ITS_WORK } from "@/lib/derive"; import { errorMessage } from "@/lib/format"; import { useStore } from "@/lib/store"; +import { isStructured } from "@/lib/structured"; import { cn } from "@/lib/utils"; import { focusSession } from "@/lib/workspaces"; import { describeAgent, startedAt } from "@/views/dashboard/names"; import { confirmStop, StopMenuItems } from "@/views/dashboard/stop"; +import { useChatTail, useOpeningPrompt } from "@/views/dashboard/use-chat-tail"; import { type Choice, useScreenTail } from "@/views/dashboard/use-screen-tail"; // One clock for every card, so a board of them re-renders once a second at @@ -59,8 +62,17 @@ export function AgentCard({ entry, selecting, selected, onSelect }: { entry: Ses // A titled agent leads with its work, its place below; otherwise its place. const title = work ?? place; const now = useNow(); + // In chat, its pane shows only the host that runs it: the card reads its + // ask and conversation instead of that screen (lib/chat-card). + const chat = isStructured(session.driver); + const waiting = atQuestion(session, state); // A ready agent has said nothing yet: its screen is only a banner. - const { tail, choices } = useScreenTail(box, session, 3, state === "running", state !== "ready"); + const screen = useScreenTail(box, session, 3, state === "running", state !== "ready" && !chat); + // What it asks needs nothing of its conversation. + const items = useChatTail(box, session, state === "running", state !== "ready" && chat && !asksIn(session.ask, waiting)); + const opening = useOpeningPrompt(box, session.name, session.state_since, chat); + const { tail, choices } = chat ? (chatCard(session.ask, waiting, items) ?? { choices: [] }) : screen; + const quote = shownPrompt(opening ?? prompt, work); const since = session.state_since ?? session.created; const [shown, setShown] = useState(false); useEffect(() => { @@ -69,9 +81,9 @@ export function AgentCard({ entry, selecting, selected, onSelect }: { entry: Ses }, []); const open = () => void focusSession(box, session.name); - const asking = atQuestion(session, state) && choices.length > 0; + const asking = waiting && choices.length > 0; // The question, without the options the chips below already show. - const lines = asking ? (tail ?? []).filter((l) => !/^\s*(?:[❯›>]\s*)?\d[.)]\s/.test(l)) : tail; + const lines = asking && !chat ? (tail ?? []).filter((l) => !/^\s*(?:[❯›>]\s*)?\d[.)]\s/.test(l)) : tail; return (
)} - {prompt && (!work || !prompt.startsWith(work.replace(/…$/, ""))) && ( -
- “{prompt}” + {quote && ( +
+ “{quote}”
)} {lines && lines.length > 0 && state !== "ready" && ( diff --git a/app/src/views/dashboard/names.ts b/app/src/views/dashboard/names.ts index 24b67a48..ae412f7f 100644 --- a/app/src/views/dashboard/names.ts +++ b/app/src/views/dashboard/names.ts @@ -1,5 +1,6 @@ import type { Location, Session, Worktree } from "@/lib/api"; import { agentLabel, agentOf, sessionName, worktreeOf } from "@/lib/derive"; +import { isStructured } from "@/lib/structured"; export interface AgentNames { where?: { location: Location; worktree: Worktree }; @@ -11,7 +12,8 @@ export interface AgentNames { // More than one agent works in the same worktree, so the place alone // doesn't say which card is which. crowded: boolean; - // The prompt it was started with, when its command carries one. + // The prompt it was started with, when its command carries one: never + // an agent's in chat, whose command line is its host's. prompt?: string; // What its work is called (Session.title), and then its agent, as // secondary text: "Fix checkout webhook" · "Claude Code". @@ -26,7 +28,7 @@ export function describeAgent(session: Session, sessions: Session[] | undefined, const place = where?.worktree.main ? where.location.name : (where?.worktree.name ?? session.location ?? session.name); const crowded = (sessions ?? []).filter((o) => o.dir === session.dir && !o.exited && agentOf(o)).length > 1; const agent = agentOf(session); - return { where, place, name: sessionName(session, { sessions }), crowded, prompt: promptOf(session.command), title: session.title?.trim() || undefined, agent: agent ? agentLabel(agent) : "Shell" }; + return { where, place, name: sessionName(session, { sessions }), crowded, prompt: isStructured(session.driver) ? undefined : promptOf(session.command), title: session.title?.trim() || undefined, agent: agent ? agentLabel(agent) : "Shell" }; } // promptOf is the prompt in an agent's command line: `claude --model x diff --git a/app/src/views/dashboard/use-chat-tail.ts b/app/src/views/dashboard/use-chat-tail.ts new file mode 100644 index 00000000..e1296e62 --- /dev/null +++ b/app/src/views/dashboard/use-chat-tail.ts @@ -0,0 +1,106 @@ +import { useEffect, useRef, useState } from "react"; + +import { isMock } from "@/hooks/use-berth-connection"; +import type { Session } from "@/lib/api"; +import { mergeTail, openingPrompt } from "@/lib/chat-card"; +import { offOf } from "@/lib/conversation-store"; +import { historyApi, useHasHistory } from "@/lib/history"; +import { useStore } from "@/lib/store"; +import type { TranscriptItem } from "@/lib/transcript"; +import type { TranscriptResult } from "@/lib/transcript-feed"; +import { useCardRead } from "@/views/dashboard/use-screen-tail"; + +// How many of its newest items a card keeps: enough to reach back past +// the steps and markers to its last words. +const TAIL = 12; + +// useChatTail is the newest few items of a chat-mode (structured) agent's +// conversation, for its board card. It reads the box's transcript the way +// a chat does (transcript?since=N, lib/transcript-feed): the recent window +// once, then only what was added since, which the box reads on from where +// it was rather than parsing the record again. It is read when a terminal +// card would read its screen, in its place (useCardRead). +export function useChatTail(box: string, session: Session, working: boolean, enabled: boolean): TranscriptItem[] | undefined { + const supported = useStore((s) => isMock() || !!s.boxes[box]?.info?.capabilities?.includes("transcript")); + const [items, setItems] = useState(); + // Where the next read starts, in the reading gen counts in. + const at = useRef<{ next: number; gen?: string }>({ next: 0 }); + useEffect(() => { + at.current = { next: 0 }; + setItems(undefined); + }, [box, session.name]); + useCardRead(box, session, working, enabled && supported, async (client, alive) => { + const { next, gen } = at.current; + const g = next && gen ? `&gen=${encodeURIComponent(gen)}` : ""; + const r = await client.box(box, "GET", `sessions/${encodeURIComponent(session.name)}/transcript?since=${next}${g}`); + if (!alive()) return; + if (r.source === "none") return setItems([]); + // An older box reading another record now: start it afresh. + if (r.gen === undefined && typeof r.next === "number" && r.next < next) { + at.current = { next: 0 }; + return setItems([]); + } + // A whole window (the first read, or one the box read afresh) stands + // alone; anything else is what was added since. + const fresh = next === 0 || !!r.reset; + if (r.file) recordOf.set(`${box}|${session.name}`, r.file); + setItems((held) => mergeTail(fresh ? [] : (held ?? []), r.items ?? [], TAIL)); + at.current = { next: r.next ?? next, gen: r.gen }; + }); + return items; +} + +// Where in its record a conversation's first prompt is looked for: before +// HEAD_BYTES, after the agent's opening frame (its tools and servers, tens +// of KB), HEAD_ITEMS at a time, back to the start in at most HEAD_PAGES. +const HEAD_BYTES = 256 << 10; +const HEAD_ITEMS = 50; +const HEAD_PAGES = 4; + +// The record each session's card last read (newer boxes say), by box and +// session: a session that carries on another conversation reads another. +const recordOf = new Map(); + +// The prompts conversations started with, by box and session, and the +// record each came from: one never changes for its record, so it is read +// once while the app runs, and again for another record. +const opening = new Map(); + +// useOpeningPrompt is the prompt a chat-mode session's conversation +// started with, or the command typed as one ("/ship ACME-12"): one read of +// the page at its record's start. Until there is one, it is read again as +// the session's state changes (since). +export function useOpeningPrompt(box: string, session: string, since: string | undefined, enabled: boolean): string | undefined { + const client = useStore((s) => s.client); + const supported = useHasHistory(box); + const key = `${box}|${session}`; + const [prompt, setPrompt] = useState(() => opening.get(key)?.prompt); + + useEffect(() => { + if (!client || !enabled || !supported) return; + const file = recordOf.get(key); + const have = opening.get(key); + // Read before its record was known, it is that record's. + if (have && !have.file) have.file = file; + if (have && have.file === file) return setPrompt(have.prompt); + let alive = true; + const head = async () => { + let page = await historyApi.older(client, box, session, HEAD_BYTES, HEAD_ITEMS); + // More items than one page before HEAD_BYTES: read back to the start. + for (let i = 1; i < HEAD_PAGES && page.more && page.items?.length && offOf(page.items[0]) > 0 && alive; i++) page = await historyApi.older(client, box, session, offOf(page.items[0]), HEAD_ITEMS); + return page; + }; + head() + .then((r) => { + const p = openingPrompt(r.items, r.more) ?? ""; + if (p) opening.set(key, { file, prompt: p }); + if (alive) setPrompt(p); + }) + .catch(() => {}); + return () => { + alive = false; + }; + }, [client, box, session, key, since, enabled, supported]); + + return prompt || undefined; +} diff --git a/app/src/views/dashboard/use-screen-tail.ts b/app/src/views/dashboard/use-screen-tail.ts index 7781be82..3ebe8c06 100644 --- a/app/src/views/dashboard/use-screen-tail.ts +++ b/app/src/views/dashboard/use-screen-tail.ts @@ -1,46 +1,84 @@ -import { useEffect, useState } from "react"; +import { useEffect, useRef, useState } from "react"; -import { boxApi, type Session } from "@/lib/api"; +import { boxApi, type Client, type Session } from "@/lib/api"; import { useEventLog } from "@/lib/events"; import { askIn, type Choice, meaningfulTail } from "@/lib/screen"; import { useStore } from "@/lib/store"; export { type Choice, meaningfulTail }; -// useScreenTail is the last few meaningful lines of a session's screen. It -// refetches when the session's state changes or an event about it arrives -// (debounced), and every 20s while the agent works, since output changes -// between events; it never polls harder than that. -export function useScreenTail(box: string, session: Session, count = 3, working = false, enabled = true): { tail?: string[]; choices: Choice[] } { +// useCardRead runs a card's read of a session: when the session's state +// changes or an event about it arrives (at most every 250ms), and every +// 20s while the agent works, since its output changes between events; it +// never polls harder than that. One read at a time, in one loop that +// events only prompt: one asked for meanwhile runs once it is in, so a +// busy agent's events never throw a read away. Disabled, it reads nothing. +export function useCardRead(box: string, session: Session, working: boolean, enabled: boolean, read: (client: Client, alive: () => boolean) => Promise) { const client = useStore((s) => s.client); - const [tail, setTail] = useState(); - const [choices, setChoices] = useState([]); + const reader = useRef(read); + reader.current = read; + const kick = useRef(() => {}); const latest = useEventLog((s) => { + if (!enabled) return undefined; const e = s.events.find((x) => x.box === box && (x.data?.path === session.dir || x.data?.name === session.name || x.data?.session === session.name)); return e?.time; }); useEffect(() => { if (!client || !enabled) return; - let cancelled = false; - const fetchTail = () => - boxApi - .screen(client, box, session.name) - .then((r) => { - if (cancelled) return; - setTail(meaningfulTail(r.screen ?? "", count)); - // Options its own menu shows, never a list in its last message. - setChoices(askIn(r.screen ?? "").choices); - }) - .catch(() => {}); - const t = window.setTimeout(fetchTail, 250); - const every = working ? window.setInterval(fetchTail, 20_000) : 0; + let alive = true; + let busy = false; + let again = false; + let soon = 0; + const run = () => { + if (!alive) return; + if (busy) { + again = true; + return; + } + busy = true; + void reader + .current(client, () => alive) + .catch(() => {}) + .finally(() => { + busy = false; + if (again) { + again = false; + run(); + } + }); + }; + kick.current = () => { + if (!soon) + soon = window.setTimeout(() => { + soon = 0; + run(); + }, 250); + }; + const every = working ? window.setInterval(run, 20_000) : 0; return () => { - cancelled = true; - window.clearTimeout(t); + alive = false; + kick.current = () => {}; + window.clearTimeout(soon); window.clearInterval(every); }; - }, [client, box, session.name, session.agent_state, session.state_since, latest, count, working, enabled]); + }, [client, box, session.name, working, enabled]); + // A change to the session, or an event about it: read, in the loop. + useEffect(() => kick.current(), [client, box, session.name, session.agent_state, session.state_since, latest, working, enabled]); +} + +// useScreenTail is the last few meaningful lines of a session's screen. +export function useScreenTail(box: string, session: Session, count = 3, working = false, enabled = true): { tail?: string[]; choices: Choice[] } { + const [tail, setTail] = useState(); + const [choices, setChoices] = useState([]); + useCardRead(box, session, working, enabled, (client, alive) => + boxApi.screen(client, box, session.name).then((r) => { + if (!alive()) return; + setTail(meaningfulTail(r.screen ?? "", count)); + // Options its own menu shows, never a list in its last message. + setChoices(askIn(r.screen ?? "").choices); + }), + ); return { tail, choices }; } diff --git a/app/src/views/home/widgets/agents.tsx b/app/src/views/home/widgets/agents.tsx index 0a9d8387..9de91199 100644 --- a/app/src/views/home/widgets/agents.tsx +++ b/app/src/views/home/widgets/agents.tsx @@ -7,6 +7,7 @@ import { Button } from "@/components/ui/button"; import { type SessionEntry, useAllSessions } from "@/hooks/use-agent-counts"; import { lastOf } from "@/lib/last-of"; import { boxApi, type Session } from "@/lib/api"; +import { QUESTION_TOOLS } from "@/lib/chat-card"; import { agentLabel, agentOf, worktreeOf } from "@/lib/derive"; import { parseScreen } from "@/lib/screen-status"; import { permissionChoices } from "@/lib/screen"; @@ -97,9 +98,6 @@ function OfflineNote({ boxes }: { boxes: string[] }) { // ---------- Needs you ---------- -// A question's tools: these are answered in the agent's own form, not with -// Allow or Deny. -const QUESTION_TOOLS = /^(AskUserQuestion|request_user_input|ExitPlanMode)$/; export function NeedsYouWidget() { const { lines, height } = useHomeWidget(); diff --git a/docs/changelog.mdx b/docs/changelog.mdx index 1402e0c6..b68921d1 100644 --- a/docs/changelog.mdx +++ b/docs/changelog.mdx @@ -32,6 +32,13 @@ again. it stops anything or runs the archive script, instead of running the script first and only then finding that git won't remove it. Remove anyway still forces it. +- **Cards for agents in chat say what the agent is doing.** On the Agent + Dashboard, an agent that runs in chat showed the frames of the host + that runs it ("out stream_event") and "stdio" as its prompt. Its card + now shows what it asks while it waits (the command and why, or the + question) with its answers, otherwise the last lines of its reply, and + the prompt it started with. Cards for agents in a terminal are as + before. ## v0.3.30