Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/verify-berthd/features/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ The maintained list of what a person can do in the app, and how to prove each on
| Plugins: built-in and your own, what they add (screens, sidebar items, widgets, panels) | [plugins.md](plugins.md) | `plugin-nav-badges` (sidebar items) |
| Home's widget grid | [home-widgets.md](home-widgets.md) | none yet; `app/e2e/home-widgets.spec.ts` asserts it |
| Questions in chat: an agent's form of questions as a card, answered on a real box; kept with Retry and Copy when it can't be | [chat-questions.md](chat-questions.md) | `chat-questions-live` (a real box, `--agent`); `app/e2e/chat.spec.ts` on the mock |
| Chat and board: no question card or answers but the options the agent's screen shows; the mode menu while turn check waits | [chat-ask-card.md](chat-ask-card.md) | `chat-ask-card` (a real box, `--agent`); `app/e2e/ask-card.spec.ts` on a stand-in box |
| Chat and board: no question card or answers but the options the agent's screen shows; the mode menu while turn check waits; a structured agent's form while its helpers report back, and a reply box that answers a card with nothing to pick | [chat-ask-card.md](chat-ask-card.md) | `chat-ask-card`, `chat-structured-ask-live` (a real box, `--agent`); `app/e2e/ask-card.spec.ts` on a stand-in box |
| Turn check's "working": an hourglass with why in the sidebar, the tab and the rail's card, not a live turn's spinner; a handoff reads as done | [turn-check-indicator.md](turn-check-indicator.md) | `turn-check-indicator`; `app/e2e/sidebar-turn-check.spec.ts` asserts it, `TestTurnCheckLive` asks the real provider |

## Proof and skip reporting
Expand Down
22 changes: 22 additions & 0 deletions .claude/skills/verify-berthd/features/chat-ask-card.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,15 @@ When an agent waits for you, its chat shows what to answer. A permission prompt
- `ask-none`: no card for a wait with no options on screen: turn check's wait, a screen at the agent's prompt (whatever list its last message has), Remote Control's menu, the footer's `/rc`.
- `mode-at-soft-wait`: the mode menu switches modes during turn check's wait, and stays blocked at a permission prompt.
- `board-answers`: the Agent Dashboard's card offers a menu's options as answer chips, and never a numbered list from the agent's last message (`views/dashboard/agent-card.tsx`, `use-screen-tail.ts`).
- `structured-form`: a structured agent's (`driver: "claude"`, the chat's header says "stdio") AskUserQuestion draws as its form even when helpers it left in the background report back after it, and even when their lines put the question megabytes back in its log and the chat is read afresh (a box restart, a chat nobody had open). The box's first read of a log reaches further back when its last 4 MB hold next to none of the conversation (`internal/transcript/transcript.go` `read`); a helper's word never answers the question (`peer.go` `addMessage`, `lib/questions.ts` `openQuestion`).
- `structured-reply`: a structured agent's question or permission card with nothing to pick says "Reply below to answer it.", and the reply box takes words: the box says no to the ask and gives Claude the words (`claudedriver.go` `Send`). A structured agent has no screen of its own to answer at, so only something to pick holds the reply box (`lib/questions.ts` `atMenu`). An agent in its terminal keeps the old hold: its own screen opens instead.

## How to get to it (user POV)

- Open a worktree's agent and switch the pane to **Conversation**.
- Type `/rc` in the reply box; pick a mode from the mode chip under it (**Permission mode: …**).
- Ask the agent something that makes it end its turn on a question in words; ask it to run a command in **Ask before edits**; ask it to use AskUserQuestion.
- A structured agent (an agent the app started, with structured agents on): ask it to leave helpers working in the background and ask you something while they do.

## Driving it with drive.mjs

Expand All @@ -32,6 +35,23 @@ node .claude/skills/verify-berthd/scripts/drive.mjs chat-ask-card --port $PORT -
```

It drives `/rc`, a switch to bypass, a turn that ends asking which of two numbered items to fix (it waits for `soft_wait: true` on the session), then `/rc` and a switch to **Ask before edits** during that wait, asserting no "Needs your answer" and no Yes button each time. Then a `touch` that asks permission (the card, then Allow) and AskUserQuestion (the form, then Esc). Second view: the box's own API through the agent (`call`): `sessions` (`agent_state`, `soft_wait`, `ask.tool`), `sessions/ask/screen` and `/controls`.
- `structured-form` and `structured-reply` on a real box: `scenarios/chat-structured-ask-live.mjs`, on a dev box whose agent runs structured. A phase asks (four helpers read ~6 MB in the background while the question waits, about two minutes), the box starts again so the chat reads the log afresh, and a phase looks, expecting `form`, `reply` or (for a before on an older build) `locked`. In bash, for its arrays:

```bash
VERIFY_DRIVER=claude .claude/skills/verify-berthd/scripts/dev-box.sh up
eval "$(.claude/skills/verify-berthd/scripts/dev-box.sh env)"
S=chat-structured-ask-live; A=(--port $PORT --agent http://127.0.0.1:9378)
VERIFY_PHASE=ask node .claude/skills/verify-berthd/scripts/drive.mjs $S "${A[@]}"
.claude/skills/verify-berthd/scripts/dev-box.sh restart # this checkout's berthd: the form
VERIFY_PHASE=look VERIFY_EXPECT=form node .claude/skills/verify-berthd/scripts/drive.mjs $S "${A[@]}"
# Nothing to pick: an older berthd (a main checkout's bin/berthd) reads no form; this app takes a reply.
VERIFY_PHASE=ask node .claude/skills/verify-berthd/scripts/drive.mjs $S "${A[@]}"
.claude/skills/verify-berthd/scripts/dev-box.sh restart ../main/bin/berthd
VERIFY_PHASE=look VERIFY_EXPECT=reply node .claude/skills/verify-berthd/scripts/drive.mjs $S "${A[@]}"
.claude/skills/verify-berthd/scripts/dev-box.sh down
```

Second view: the session's own log (`events.jsonl` under the dev box's `box-home`), read by the scenario: the question's `control_request`, the helpers' `task_notification`s after it, and the `control_response` that carried the answers or the reply.

## Gotchas

Expand All @@ -40,3 +60,5 @@ node .claude/skills/verify-berthd/scripts/drive.mjs chat-ask-card --port $PORT -
- Esc on AskUserQuestion declines it, but Claude Code sends no hook for that, so the box keeps the wait (and the reply box stays blocked) until the next prompt. Keep that check last.
- Turn check scores the turn: a scripted "reply word for word" prompt scored under its 0.65 bar. Ask for a real decision.
- Claude Code 2.1's footer calls the default mode "manual mode".
- Read keeps about 600 lines of a file, ~63 KB of log per call, and one helper told to read 40 files stopped after 24: the scenario splits 100 files over four helpers.
- `dev-box.sh restart` restarts berthd only; the agents in its tmux keep running, so a question stays waiting across it.
2 changes: 1 addition & 1 deletion .claude/skills/verify-berthd/features/chat-questions.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,5 +36,5 @@ The preview must be opened as `http://localhost:<1420-1439>`: the agent's API on
## Gotchas

- The scenario sends its prompt from the chat, so the session must be at its prompt: one left at a form from a failed run needs Esc first.
- A box started from the app starts agents structured (Claude through its SDK, which answers through a hook, not keys); `dev-box.sh` sets the box's defaults so they start in their terminals, the path this covers.
- A box started from the app starts agents structured (Claude through its SDK, which answers through a hook, not keys); `dev-box.sh` sets the box's defaults so they start in their terminals, the path this covers. A structured agent's questions are covered by `chat-structured-ask-live` (`VERIFY_DRIVER=claude .claude/skills/verify-berthd/scripts/dev-box.sh up`, [chat-ask-card.md](chat-ask-card.md)).
- `dev-box.sh up` refuses if the dev box rewrote the agent hooks in your home (it does that only when they are older than its own); put them back from your backup before going on.
53 changes: 43 additions & 10 deletions .claude/skills/verify-berthd/scripts/dev-box.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,12 @@
# their own state, tmux server and ports. `up` builds bin/, starts berthd
# on 127.0.0.1:7544 and the agent on 9377/9378 (never 1377-1379), pairs
# them as box "devq", adds a synthetic repo "acme-shop" and starts Claude
# Code in its terminal there, session "ask". `env` prints what drive.mjs
# and tmux need; `down` stops it all and deletes the state.
# Code in its terminal there, session "ask" (VERIFY_DRIVER=claude: Claude
# structured instead, as the app starts agents). `env` prints what
# drive.mjs and tmux need; `restart [berthd]` starts the box again, every
# conversation then read afresh, with another build if given (a main
# checkout's bin/berthd, for a before); `down` stops it all and deletes
# the state.
#
# HOME stays real: Claude Code needs its sign-in. The box refreshes
# Claude's hooks in ~ only when they are out of date; `up` stops if it
Expand All @@ -19,6 +23,18 @@ lap_env=(BERTH_HOME="$dir/lap-home" BERTH_USER_DIR="$dir/lap-user" BERTH_UI_ADDR
box() { env "${box_env[@]}" "$root/bin/berthd" "$@"; }
lap() { env "${lap_env[@]}" "$root/bin/berth" "$@"; }
tm() { env TMUX_TMPDIR="$dir/tmux" tmux -L berth "$@"; }
serve() {
env "${box_env[@]}" nohup "$1" serve --listen 127.0.0.1:7544 >"$dir/berthd.log" 2>&1 &
local pid=$!
echo $pid >"$dir/berthd.pid"
for _ in $(seq 120); do
grep -q "berthd serving" "$dir/berthd.log" && return
kill -0 $pid 2>/dev/null || { echo "berthd exited before it served: $dir/berthd.log" >&2; exit 1; }
sleep 0.5
done
echo "berthd isn't serving after a minute: $dir/berthd.log" >&2
exit 1
}

case ${1:-} in
up)
Expand All @@ -27,9 +43,7 @@ up)
done
mkdir -p "$dir"/{box-home,box-user,lap-home,lap-user,tmux} && chmod 700 "$dir/tmux"
make -C "$root" build >/dev/null
env "${box_env[@]}" nohup "$root/bin/berthd" serve --listen 127.0.0.1:7544 >"$dir/berthd.log" 2>&1 &
echo $! >"$dir/berthd.pid"
until grep -q "berthd serving" "$dir/berthd.log"; do sleep 0.5; done
serve "$root/bin/berthd"
if grep -q "updated berth's hooks" "$dir/berthd.log"; then
echo "the dev box rewrote agent hooks in $HOME (berthd.log): put them back before going on" >&2
exit 1
Expand All @@ -45,12 +59,31 @@ up)
git -C "$repo" add . && git -C "$repo" -c user.email=dev@acme.test -c user.name=acme commit -qm init
fi
box location add acme-shop "$repo" >/dev/null
box session new acme-shop --name ask -- claude --dangerously-skip-permissions >/dev/null
# A new folder asks to be trusted first.
sleep 8
tm send-keys -t ask Up && sleep 0.5 && tm send-keys -t ask Enter
if [[ ${VERIFY_DRIVER:-} == claude ]]; then
# Its chat is its stream: no screen of its own, no folder to trust.
curl -sf -X POST -H "Authorization: Bearer $token" -H 'content-type: application/json' \
http://127.0.0.1:9378/v1/boxes/devq/api/sessions -d '{"name":"ask","location":"acme-shop","agent":"claude","driver":"claude","mode":"bypassPermissions"}' >/dev/null
else
box session new acme-shop --name ask -- claude --dangerously-skip-permissions >/dev/null
# A new folder asks to be trusted first.
sleep 8
tm send-keys -t ask Up && sleep 0.5 && tm send-keys -t ask Enter
fi
echo "up: box devq (127.0.0.1:7544), agent http://127.0.0.1:9378, worktree devq/acme-shop, session ask"
;;
restart)
# berthd only: tmux, and the agents in it, keep running.
bin=${2:-$root/bin/berthd}
[[ -x $bin ]] || { echo "no berthd at $bin" >&2; exit 1; }
# Until it has exited, not just let go of the port: its cleanup on the
# way out would take the new one's socket.
pid=$(cat "$dir/berthd.pid")
kill "$pid"
for _ in $(seq 150); do kill -0 "$pid" 2>/dev/null || break; sleep 0.2; done
if kill -0 "$pid" 2>/dev/null; then echo "berthd ($pid) is still running after 30 s" >&2; exit 1; fi
serve "$bin"
echo "restarted: $bin"
;;
env)
# VERIFY_BERTH_TOKEN is a secret: export it, don't print it anywhere else.
echo "export VERIFY_BERTH_TOKEN=$(lap ui-token | jq -r .token) VERIFY_WORKTREE=devq/acme-shop TMUX_TMPDIR=$dir/tmux"
Expand All @@ -66,7 +99,7 @@ down)
echo "down: $dir removed"
;;
*)
echo "usage: dev-box.sh up|env|down" >&2
echo "usage: dev-box.sh up|env|restart [berthd]|down" >&2
exit 2
;;
esac
Loading