Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/contract-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,6 @@ jobs:
run: cargo test --locked
- name: Run shell CLI and lifecycle contracts
run: tests/run.sh
- name: Run malicious orchestrator boundary contracts
if: runner.os == 'Linux'
run: sudo -E tests/malicious-orchestrator.sh
225 changes: 69 additions & 156 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,199 +1,112 @@
# Multiagent

Multiagent is the reference implementation of an orchestration layer for
coding agents. It is not another coding agent: it composes existing Codex and
Claude CLIs into parallel roles, records their work, independently verifies the
result, and gates acceptance on evidence bound to the exact Git diff.

The project prioritizes orchestration, evaluation, and runtime rigor over a
custom UI or model implementation.
Multiagent is a Rust control plane for coordinating existing coding agents. It
does not implement another coding agent or model loop. It runs Codex, Claude
Code, and Qwen Code in explicit roles, records durable workflow state, and
accepts work only when reviewer evidence matches the exact final Git diff.

## Requirements

Building from source requires Rust 1.75 or newer, Cargo, Bash, and Git. Rust owns
the production control plane. Python 3.8 or newer is required only for evaluation
and evidence-analysis commands; those modules have no third-party Python package
dependency. Live agent sessions also require `tmux` plus the configured Codex or
Claude CLI.
From a source checkout you need Rust 1.75+, Cargo, Bash, Git, and tmux. Install
and authenticate at least one supported coding-agent CLI. Python 3.8+ is used
only by evaluation and evidence-analysis tools, not the production control
plane.

## Try It Locally
## Quick Start

Run the deterministic local demo from the repository root:
Run:

```bash
./scripts/demo.sh
```

It needs Rust/Cargo, Bash, and Git. It does not launch an agent, use an
API key, or spend model tokens. In under five minutes it exercises the real
repository control plane:

1. a deterministic verifier records a blocking behavior finding;
2. `gate-check` rejects the open todo;
3. a worker repair and validation result are recorded;
4. verifier acceptance is bound to the exact final-diff SHA-256;
5. the gate accepts, rejects a later stale diff, and accepts the restored
verified diff.

See [the three-minute walkthrough](docs/demo.md) for the expected output and
the artifacts behind each transition.

## System Flow

```mermaid
flowchart TD
User["User task"] --> Pre["Pre-implementation"]
Pre --> Authority["Independent authority review"]
Authority --> Choice{"User-owned decision?"}
Choice -- "yes" --> UserDecision["Ask user and record choice"]
Choice -- "no" --> Context["Approved implementation context"]
UserDecision --> Context
Context --> DAG["Assignments and dependency DAG"]
DAG --> WorkerA["Worker A"]
DAG --> WorkerB["Worker B"]
WorkerA --> Repo["Target Git repository"]
WorkerB --> Repo
Repo --> Snapshot["Exact diff snapshot"]
Snapshot --> Reviews["Post-implementation reviews"]
Reviews --> Findings["Findings, todos, and recheck evidence"]
Findings --> Todo{"Active TODO?"}
Todo -- "yes" --> Pre
Todo -- "no" --> Gate{"Lifecycle and technical gates"}
Gate -- "hash-bound evidence passes" --> Result["Accepted patch"]
./launch.sh --session multiagent --root /absolute/path/to/target-repo
```

`multiagent` is the unified CLI. Its Rust core owns exact Git snapshots,
decisions, DAGs, lifecycle transitions, assignments, findings, repair todos,
validation leases, validation subprocesses, tmux process orchestration, status,
watching, and recovery. `launch.sh` is the source-checkout bootstrap: it locates
or builds the Rust executable and immediately runs `multiagent launch`. tmux—not
shell or Rust—continues to own the PTY. Python under `evaluation/` is limited to
benchmark execution, status reading, and provenance. The SWE Bench Pro adapter
drives the production Rust path and transports its workspace diff to the
official scorer; it does not implement a second solver or acceptance gate. See
[the control-plane boundary](docs/control-plane-boundary.md).

## Run With Agents

Live orchestration additionally requires `tmux` and at least one configured
Codex or Claude CLI:
`launch.sh` is only a compatibility bootstrap. It locates or builds the Rust
binary and immediately executes:

```bash
./launch.sh --session multiagent --root /absolute/path/to/target-repo
multiagent launch --session multiagent --root /absolute/path/to/target-repo
```

Launches are clean by default. Explicit crash recovery is opt-in:
Launches are clean by default. Resume durable state after an interrupted run
with:

```bash
./launch.sh --resume --session multiagent --root /absolute/path/to/target-repo
```

## Implementation Lifecycle

`multiagent launch` bundles the orchestrator role with the mandatory lifecycle
prompt, records prompt hashes, and initializes durable lifecycle state under:

```text
$MULTIAGENT_STATE_DIR/workflows/$MULTIAGENT_WORKFLOW_ID/lifecycle/
```

`multiagent workflow` is the Rust lifecycle state machine in `src/workflow.rs`.
Existing v1 state files remain readable.

The enforced normal path is `pre-implementation -> implementation ->
post-implementation`. An independent authority review identifies consequential
choices and whether the user or orchestrator owns each one. Writable workers
receive the complete approved implementation context, not only a partial
assignment summary. Any accepted review finding creates a TODO and returns through
pre-implementation before another edit iteration.
The implementation permit also verifies that `multiagent decision` contains a
committed decision whose selected plan matches the context and assignment.

Inspect and advance the state with:
The default role backends are Codex for orchestration and verification and
Claude Code for workers. To use one backend for every role:

```bash
multiagent workflow status "$MULTIAGENT_WORKFLOW_ID"
multiagent workflow prepare-implementation "$MULTIAGENT_WORKFLOW_ID" \
--decision-id DECISION_ID --plan-id PLAN_ID --decision-revision REVISION \
--implementation-context CONTEXT_PATH --authority-review REVIEW_ID
multiagent workflow transition "$MULTIAGENT_WORKFLOW_ID" implementation
multiagent workflow completion-check "$MULTIAGENT_WORKFLOW_ID"
ORCHESTRATOR_CLI=codex \
WORKER_CLI=codex \
SUBAGENT_CLI=codex \
VERIFIER_CLI=codex \
./launch.sh --root /absolute/path/to/target-repo
```

`MULTIAGENT_LIFECYCLE_ENFORCEMENT=1` is the default. Existing structured
technical findings and repair TODOs remain authoritative. Running
`multiagent orchestrator complete` requires both the lifecycle completion gate and
`multiagent subagent gate-check`.

The default roles use Codex for orchestration and verification and Claude for
workers. `WORKER_CLI`: worker CLI for manual worker windows, default `claude`.
`VERIFIER_CLI`: verifier CLI, default `codex`. CLI choices, recovery, ownership
policy, role prompts, DAG workflows, and all control-plane commands are in the
[getting-started and operations guide](docs/getting-started.md).
Supported backend names are `codex`, `claude`, and `qwen`.

## Operations Reference
## What Runs

The operations guide preserves the full reference for these framework
contracts and workflows:
```mermaid
flowchart LR
U["Task"] --> O["Read-only orchestrator"]
O --> D["Decision + contract"]
D --> W["Path-scoped writer"]
W --> S["Canonical Git snapshot"]
S --> V["Read-only reviewers"]
V --> G{"Supervisor gates pass?"}
G -- no --> D
G -- yes --> C["Atomic completion"]
```

- **Parallel DAG Discipline** and the **Structured Repair Loop**, including
`finding-todo-loop.md`, `todo-close`, and a bounded repair worker;
- **Prompt Modules**, **Contract Scout Workflow**, `acceptance-scout.md`,
`hidden-contract-ledger`, and hidden-contract edge cases;
- **Scope Guard Workflow**, **Validation Coordinator Workflow**, the validation lease table,
`validation-run`, and `validation-lease-acquire`;
- **Verifier Workflow**, its compact contract ledger, and the
`MULTIAGENT_VERIFIER_MAX_ITERATIONS=3` escalation threshold;
- Codex UI dashboard watching through `multiagent watch`, backed by tmux pane logs
under `.multiagent/logs`, blocked-agent state, and workflow DAG nodes;
- preflight checks that prevent a scaffold, shim, or proxy behavior from being
mistaken for the target production system.
The Rust binary owns decisions, workflow phases, assignments, snapshots,
findings, todos, reviewer evidence, process lifecycle, status, and recovery.
Tmux owns PTYs and interactive terminal lifecycle. Python is restricted to
evaluation and provenance; it does not implement a second production workflow
or acceptance gate.

## Evaluation Framework
On production Linux, separate Unix identities isolate the orchestrator, the
single active writer, read-only agents, and the authority supervisor. The
orchestrator can read worker and reviewer state but cannot write the target
repository or protected lifecycle state. Completion is a request to the
supervisor, which checks every gate under the lifecycle lock before changing
the phase to `complete`.

No-spend adapter checks are available locally:
## Common Commands

```bash
python3 -m evaluation.cli --adapter ponytail --selftest
python3 -m evaluation.cli --adapter orchestration --selftest
multiagent status
multiagent watch
multiagent decision list
multiagent workflow status "$MULTIAGENT_WORKFLOW_ID"
multiagent subagent list
multiagent subagent gate-check
multiagent orchestrator complete
```

The `orchestration` adapter covers planning behavior, dependency edges,
parallel fan-out, ownership, and final consolidation. Adapter task definitions
live under `evaluation/tasks`.

The historical production-native first-50 report records `36/50` clean
official passes. That number is a cumulative best-known aggregate from
iterative focused reruns, not a single held-out 50-row run. The exact report
snapshot, contributing run prefixes, limitations, failure analysis, and a
pinned clean-run command are in [the benchmark guide](docs/benchmark.md).
The Docker workflow uses roughly 20 GB per task container and is intentionally
an advanced path.
Normally the orchestrator issues lifecycle and subagent commands. Operators use
the status, watch, recovery, and inspection commands to supervise a run.

## Documentation

- [Three-minute local demo](docs/demo.md)
- [Getting started and operations](docs/getting-started.md)
- [Benchmark results and reproducibility](docs/benchmark.md)
- [Internal pilot request one-pager](docs/internal-pilot-request.md)
- [Evaluation framework](evaluation/README.md)
- [Decisions](docs/decisions.md) — why the control plane and backend boundary
have this shape.
- [Architecture](docs/architecture.md) — components, authority boundaries,
lifecycle, state, and evaluation boundary.
- [Getting started and operations](docs/getting-started.md) — configuration,
normal operation, decisions, agents, recovery, traces, and troubleshooting.

## Test

```bash
tests/run.sh
cargo test
bash tests/run.sh
```

## Enforcement Caveat
Linux authority-boundary coverage is exercised by:

Decision-authority review, approved-context handoff, lifecycle TODO convergence,
and completion are enforced by the orchestrator prompt plus normal-path checks
in `multiagent workflow`, `multiagent subagent`, and `multiagent orchestrator`. This makes
ordinary violations fail visibly, but it is not a security or capability
boundary: an orchestrator with direct shell and state-file access can bypass or
disable these checks.

Revisit this limitation before treating the workflow as strict enforcement.
The stronger design is a trusted supervisor that exclusively owns writable
worker launch and independently validates TODO state, decision ownership, user
approval, context revision, and assignment scope before starting a worker.
```bash
bash tests/malicious-orchestrator.sh
```
Loading
Loading