Skip to content

v1.1.0 — complete the Orca delegation substrate - #27

Merged
choiyounggi merged 9 commits into
mainfrom
feat/orca-completeness
Aug 5, 2026
Merged

v1.1.0 — complete the Orca delegation substrate#27
choiyounggi merged 9 commits into
mainfrom
feat/orca-completeness

Conversation

@choiyounggi

Copy link
Copy Markdown
Owner

Closes the gaps that kept the Orca substrate behind the tmux one. Planned as a four-task orchestration; the run itself then surfaced more defects than the plan did, so those are here too — each with the command and error that proved it.

Wave 1

orca-wait.sh — three defects (134 → 153 tests)

  1. Escalation was checked once, before the blocking window. guardrails writes its record from inside the worker whether or not the worker survives to send the matching Orca message, so a worker that died while blocked cost the entire timeout. The wait is now sliced into ORCA_WAIT_RECHECK_MS windows (default 15000, matching watch-status.sh) with the dir re-read between them. The slices sum to exactly the caller's <timeout-ms> — the deadline is neither extended nor truncated.
  2. One call consumed one Delivery, so a Wave of N workers could cost N coordinator turns. --until-all <ids> consumes successful batches until every listed id is completed. 3/5/6 still return immediately without acking.
  3. exit 0 meant "a batch arrived", not "a completion arrived". --types asks for worker_done/escalation/question but heartbeats are delivered anyway, and so are completions from earlier Waves or unrelated Tasks — all fell through to "everything in this batch was a successful completion", acked, and returned 0. Measured during this run: a batch of one previous-Wave plan completion + one probe completion + a heartbeat returned exit 0 while all three current tasks were still running; only the completed=0/3 line revealed it. Exit 0 now requires a successful worker_done for a task id the caller passed, using the same exact-match index semantics as the progress count.

orca-worker-start.sh — no second agent on one checkout (153 → 166)

Re-entry restarts a "dead" worker with --worktree/--agent, which for a dev-loop worker always lands in worker mode and used to run terminal create unconditionally. If the liveness read was wrong — and this run proved a worker can be ALIVE yet wedged — that puts two agents on one working tree with the Dispatch bound to one of them. Worker mode now joins dispatch-show against terminal list and rebinds the live assignee. Unreadable probe → exit 6, creates nothing; only ok:true + dispatch:null counts as a first start. Field names (orphaned, connected, assignee_handle) verified against the live runtime.

session-prompt.md — §O1–§O4

The file was four tmux one-liners, so the Orca protocol was spliced into every --spec by hand. Two of the four also ended with "wait for an approval message", correct only for a live pane. Adds the Orca set plus a shared protocol block; the four tmux prompts are byte-identical.

Wave 2

orca-worker-stalled.sh — the third worker state (166 → 183)

ALIVE/DEAD cannot express what stopped this orchestration: three workers wedged on an interactive usage-limit prompt held live PTYs for 75 minutes with byte-identical diffs while orca-worktree-alive.sh said ALIVE, Tasks read dispatched, worker-read said worker=ready terminal=running, and orca-wait.sh checkpointed forever. Only reading a worker's raw screen found it.

The signal was chosen by measurement and one candidate was rejected: lastOutputAt tracks terminal writes, and a TUI agent repaints its spinner continuously, so a wedged worker reads as output 0 seconds ago — against the three real worktrees it gave −1s, −1s, 340s. agents[].updatedAt discriminated cleanly (12s / 64s / 340s). A test pins the rejected case so the trap cannot come back.

SKILL.md — five instructions this run proved wrong

Was Actually
--deps "chain with --deps" --deps task_x errors (wants a JSON array); even the array form gave task_not_startable with the dep already completed — isolated by dispatching an identical no-deps Task to the same terminal, which started fine
failed start "do not auto-retry" the Task is spent (status=failed); a retry needs a new Task
send --type unspecified note/guidance/message/info are all invalid_argument; omit --type
--to dispatch: "attempt-specific coordinator guidance" a Claude worker never polls its mailbox and a new Dispatch cannot land while one runs — there is no mid-flight steering; the rework round is the path
Wave task ids progress reporting correctness — it is what separates your completions from heartbeats and other Waves

Verification

  • 183/183 pass, 0 failures
  • All shell scripts pass sh -n; new tests invoke the scripts with sh, not bash, so the #!/bin/sh shebang is actually exercised (a bash "$script" suite cannot detect the bashisms that shebang exists to prevent)
  • Version bumped in both .claude-plugin/plugin.json and .claude-plugin/marketplace.json

Tag v1.1.0 after merge.

🤖 Generated with Claude Code

dch0202 and others added 9 commits August 5, 2026 10:05
session-prompt.md held four tmux one-liners and nothing else, so an
orchestrator on the Orca substrate had to splice the worker protocol into
every `--spec` by hand — which is how the two substrates drift apart. Two of
the four also ended with "and wait for an approval message", correct only
when the orchestrator will later send-keys into the same live pane; on Orca
each phase is its own Task and the worker must report and end its turn.

Adds §O1–§O4 plus a shared Orca worker-protocol block, and turns the header
into a substrate selector so the wrong set cannot be picked by accident. The
single-line send-keys constraint now sits inside the tmux banner, and the
`--spec` multi-line exemption inside the Orca one.

The four tmux prompts are byte-identical; only the file header changed.

Verified: §1–§4 bodies show no diff; 0 occurrences of "wait" in the §O
region; all 8 tokens used are listed in the token table; every `orca` flag
written in the file resolves inside SKILL.md's worker-protocol block;
bats 134/134.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SKILL.md's re-entry path restarts a worker the coordinator believes is dead
by calling orca-worker-start.sh again with --worktree/--agent. For a dev-loop
worker GROUNDWORK_ESCALATION_DIR is always set, so that lands in worker mode,
which ran `orca terminal create` unconditionally. If the liveness read was
wrong — and this run proved a worker can be ALIVE yet wedged — that puts a
second agent on the same checkout with the Dispatch bound to only one of
them. The tmux path has no such hazard: launch-session.sh detects an existing
session and reuses it, loudly.

Worker mode now probes first, automatically, with no flag to pass: it joins
`orchestration dispatch-show --task` against `terminal list --worktree` and
binds the recorded assignee when that terminal is still live on this
worktree, reporting the reuse on stderr.

Fails toward the safe side. Creating the second agent is the damaging
direction, so a probe that cannot answer is UNKNOWN, never "no agent": an
unreadable dispatch or terminal list exits 6 and creates nothing. Only
`ok:true` with `dispatch:null` counts as a definite first start. Both probe
calls take `</dev/null` so a headless coordinator cannot block on a prompt.
Reuse also skips the tui-idle wait — that exists to pass a new CLI's trust
screen, and a mid-task agent may never report idle.

Field names verified against the live runtime, not assumed: `terminal list`
exposes `orphaned` and `connected`; `dispatch-show` exposes `assignee_handle`.

Every pre-existing fail-closed guard is untouched. Tests 134 -> 147: reuse,
first start, orphaned, disconnected, empty list, assignee on another
worktree, both probes returning ok:false, both returning malformed JSON, a
blank payload, and proof the probe never runs in --terminal or composed mode.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…all, honest exit 0

Three defects in the Orca wait loop, all of which cost real time in the
orchestration run that produced this change.

1. An escalation was only checked ONCE, before the blocking window. guardrails
   writes its record from inside the worker whether or not the worker survives
   to send the matching Orca message, so a worker that died while blocked cost
   the entire timeout. The wait is now sliced into ORCA_WAIT_RECHECK_MS windows
   (default 15000, matching watch-status.sh's poll) with the dir re-read between
   them, so detection is bounded by one interval. The slices sum to exactly the
   caller's <timeout-ms> — the deadline is neither extended nor truncated.

2. One call consumed one Delivery, so a Wave of N workers could cost N
   coordinator turns. `--until-all <ids>` keeps consuming successful batches
   until every requested id is completed, then exits 0 once. Codes 3/5/6 still
   return immediately without acking, because those need a decision and the
   batch must replay. Without ids there is no set to be "all" of, so that is a
   usage error.

3. exit 0 meant "a batch arrived", not "a completion arrived". `--types` asks
   for worker_done/escalation/question but heartbeats are delivered anyway, and
   a completion from an earlier Wave or an unrelated task lands here too. Both
   fell through to "everything in this batch was a successful completion",
   acked, and returned 0 — sending the coordinator after artifacts nobody
   produced. Measured live during this run: a batch of one previous-Wave plan
   completion + one probe completion + a heartbeat returned exit 0 while all
   three of the current Wave's tasks were still running; only the completed=0/3
   line revealed it. Exit 0 now requires a SUCCESSFUL worker_done for a task id
   the caller asked about, judged with the same exact-match `index` semantics as
   the progress count. Such batches are still acked — liveness needs no
   decision, and replaying it forever would livelock the wait — but they no
   longer report progress. With no ids passed, relevance cannot be judged, so
   any completion counts, as before.

That last point makes the Wave-scoped id list a correctness requirement, not
the progress-reporting nicety O4 currently describes.

Tests 134 -> 153. The `orca` CLI is replaced by a test double (via the existing
ORCA_BIN hook) so multi-window and multi-batch behavior is observable; the
script, jq and the ack path stay real, and assertions are on what the script
did — exit code, acked deliveries, window count and sizes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The re-entry design splits workers into ALIVE and DEAD. This run produced a
third state neither can express: three workers wedged on an interactive Claude
usage-limit prompt held live PTYs for 75 minutes with byte-identical diffs
while orca-worktree-alive.sh said ALIVE, their Orca Tasks read `dispatched`,
worker-read reported `worker=ready terminal=running`, no guardrails escalation
was written, and orca-wait.sh simply checkpointed forever. The only thing that
found it was reading a worker's raw screen by hand.

Signal chosen by measurement, not assumption. The obvious candidate —
`lastOutputAt` — is wrong: a TUI agent repaints its spinner continuously, so a
wedged worker reports output "0 seconds ago" (in fact a fraction of a second
in the future, since the field is ms-precision and `date +%s` is not). Against
the three real worktrees it read -1s, -1s, 340s and could not separate work
from a redraw. `worktree ps` also exposes an `agents[]` array with each agent's
own `state` and `updatedAt`; that discriminated cleanly (12s / 64s / 340s) and
is what the script uses. A test pins the rejected case so the trap cannot be
reintroduced.

Fails toward not-stalled: an untracked worktree, a missing agents array, a
malformed payload, a far-future timestamp and an unusable threshold are all
exit 2, and a worktree with no live terminal is exit 0 — that is the alive/dead
question, already answered elsewhere, and reporting it twice would double-count
one worker. Killing a working agent is the damaging direction.

Verified live: flags the restarted worker idle 423s with `agent state: done`
while the two still-ticking workers pass. 17 tests, suite 166 -> 183. Invoked
with `sh`, not `bash`, so the `#!/bin/sh` shebang is actually exercised.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ions this run proved wrong

SKILL.md is executed by a model, so a wrong instruction there is not a typo —
it is a failure the coordinator walks into. Orchestrating the Wave-1 changes
with SKILL.md v1.0.0 walked into five.

- O2 said "chain with `--deps`". `--deps task_x` errors (it wants a JSON
  array), and even `--deps '["task_x"]'` produced `task_not_startable` with
  the dependency already `completed` — isolated by dispatching an identical
  no-deps Task to the same terminal, which started fine. Phase ordering is now
  documented as coordinator-enforced.
- A failed `worker-start` spends the Task: it goes to `status=failed` and
  every later attempt returns `task_not_startable`. "Do not auto-retry" was
  there; "the Task is now unusable, make a new one" was not.
- `send --to dispatch:<id>` was offered as coordinator guidance without saying
  that `--type note`/`guidance`/`message`/`info` are all rejected, or — the
  part that matters — that a Claude worker never polls its mailbox and a new
  Dispatch cannot land while one is running. There is no mid-flight steering;
  the rework round is the path. Said plainly now.
- O4 presented the Wave task-id list as progress cosmetics. It is correctness:
  it is what separates your completions from heartbeats and from other Waves'.
- O5 offered alive/dead as the whole liveness story. It is not — hence
  orca-worker-stalled.sh, now documented beside it, with the instruction to
  read the worker's screen before acting on a stall.

Also documents what Wave 1 actually merged: `orca-wait.sh --until-all` and
`ORCA_WAIT_RECHECK_MS`, orca-worker-start.sh's automatic reuse-instead-of-
duplicate on re-entry, and session-prompt.md's §O1–§O4 as the `--spec` source
(the tmux §1–§4 stay pointed at send-keys).

Every flag and exit code here was checked against the merged script's usage
header or a live `--help`, not restated from memory. Suite 183/183.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Orca substrate completeness: orca-wait --until-all + bounded escalation
detection + honest exit 0, re-entry reuse instead of a duplicate agent,
orca-worker-stalled.sh, §O1-§O4 prompt templates, and the SKILL.md
corrections this run proved necessary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@choiyounggi
choiyounggi merged commit a585239 into main Aug 5, 2026
2 checks passed
@choiyounggi
choiyounggi deleted the feat/orca-completeness branch August 5, 2026 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant