Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
f002e08
fix(adapters): kill the process group so timeoutSec actually aborts a…
datj9 Aug 23, 2026
f5bfb9d
fix(adapters): abort agent runs at the timeout, name a missing binary…
datj9 Aug 23, 2026
c811511
fix: catch the auth-header, .netrc and auth.json shapes the scanner m…
datj9 Aug 23, 2026
8776e78
fix(graph): reject adapter on command and human nodes
datj9 Aug 23, 2026
1927463
fix(codex): fail a verifier whose sandbox is broken, and validate the…
datj9 Aug 23, 2026
5d54914
fix(resume): reject --answer for unknown or non-paused node ids
datj9 Aug 23, 2026
e001823
fix: treat a bare `key` assignment as a credential
datj9 Aug 23, 2026
fb84b9d
fix(graph): reject unknown template node references at validate time
datj9 Aug 23, 2026
f65f410
fix: make the assignment rule linear and stop it firing on prose
datj9 Aug 23, 2026
f370042
fix(engine): L4 match verifier pass string on word boundaries
datj9 Aug 23, 2026
a433872
fix: rewrite the machine hostname out of a handoff bundle
datj9 Aug 23, 2026
2377004
fix(report): publish only the report, not its parent directory
datj9 Aug 23, 2026
53d6c15
fix(budget): H3 make the three ceilings exclusive
datj9 Aug 23, 2026
8b8f388
fix(engine): C2 gate the node-run ceiling before every dispatch
datj9 Aug 23, 2026
568d2dd
fix(report): validate --visibility at runtime, not just in the type
datj9 Aug 23, 2026
6b82da5
fix(engine): C2 budget-check every retry attempt
datj9 Aug 23, 2026
bbe7b3a
fix(report): declare charset and language in the report HTML
datj9 Aug 23, 2026
25b4fbf
fix(engine): M2 fail the run when a template reference cannot be reso…
datj9 Aug 23, 2026
bf27ea1
fix(engine): M5 interpolate a human node's question
datj9 Aug 23, 2026
92270fa
fix(events): reject an unknown --kind instead of printing nothing
datj9 Aug 23, 2026
b302e67
test(engine): M4 pin cumulative node runs and spend across a resume
datj9 Aug 23, 2026
87abd90
fix(handoff): make files.txt repo-relative only, and never drop silently
datj9 Aug 23, 2026
95c65b6
fix(handoff): exit 1 when push cannot find the bundle directory
datj9 Aug 23, 2026
3fa7200
fix(handoff): validate --expires as a real date, not just a shape
datj9 Aug 23, 2026
078465b
docs: match the README to the fixes the other slices landed
datj9 Aug 23, 2026
1075b90
Merge branch 's2-budget-engine' into fix/self-test-findings
datj9 Aug 23, 2026
b04018f
Merge branch 'worktree-agent-a64700a37af50f2df' into fix/self-test-fi…
datj9 Aug 23, 2026
d4ef5a7
Merge branch 's4-graph-validation' into fix/self-test-findings
datj9 Aug 23, 2026
4c7682e
Merge branch 'worktree-agent-a816f0becffdc8c37' into fix/self-test-fi…
datj9 Aug 23, 2026
80d9a67
Merge branch 'worktree-agent-a8b1b2acee9ce8e06' into fix/self-test-fi…
datj9 Aug 23, 2026
8669e3c
fix(handoff): wire the machine hostname through redaction so the rewr…
datj9 Aug 23, 2026
b28774c
fix(handoff): close three credential-scan bypasses
datj9 Aug 23, 2026
046a4e8
fix(handoff): redact the metadata block, not just the session
datj9 Aug 23, 2026
df263ea
fix(adapters): stop orphaning agent CLIs, and force-kill a stubborn g…
datj9 Aug 23, 2026
2af3e1e
fix(core): make the node-run ceiling hold across a concurrent batch
datj9 Aug 23, 2026
dc7f7d9
fix(engine): stop a budget stop from burying a genuine node failure
datj9 Aug 23, 2026
3e60c13
fix(handoff): bound the url-credentials scheme so a long hyphen run c…
datj9 Aug 23, 2026
3a5b6de
fix(handoff): let the scan check for the identity it just redacted
datj9 Aug 23, 2026
82e2e9e
test(handoff): bound the hostname assertion so a single-label host ca…
datj9 Aug 26, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 79 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ where succeeded 1 0.0000 0.0s
done succeeded 1 0.0000 0.0s

budget 0.0000/0.0100 usd 0s/60s wall clock 3/5 node runs
note: adapters that do not report a price (codex, command) record 0.0000 usd - the number is not estimated.
```

Then:
Expand All @@ -70,7 +71,7 @@ lg events <runId> # the full audit trail as JSONL
```yaml
name: fix-failing-test

budget: # enforced before every dispatch batch
budget: # enforced before every node dispatch
maxUsd: 2.00
maxWallClockSec: 1800
maxNodeRuns: 20
Expand Down Expand Up @@ -120,7 +121,11 @@ edges:

Templates resolve against run state: `{{vars.ticket}}` (or the shorthand `{{ticket}}`) and `{{nodes.<id>.output}}`. An unresolvable reference is an error, not an empty string and never a passthrough — which is why node ids are restricted to `[A-Za-z0-9_-]`, 1 to 64 characters. A dot would collide with the reference syntax itself, so `lg validate` rejects it rather than letting `{{nodes.my.node.output}}` mean nothing at run time.

Check a graph before running it — `lg validate` catches unknown node ids, cycles, missing budgets, and bad adapters:
Check a graph before running it - `lg validate` catches unknown node ids, cycles, missing
budgets, bad adapters, and `{{nodes.<id>.output}}` references to a node the graph never
declares. Unknown *variable* references are not caught: `lg run --var` can supply a variable
the `vars:` block never declares, so an undeclared `{{vars.x}}` is not statically decidable
and stays a run-time error.

```bash
lg validate examples/fix-failing-test.yaml
Expand Down Expand Up @@ -160,6 +165,22 @@ A paused human node resumes the same way:
lg resume <runId> --answer approve="ship it"
```

The node id has to be one the run is actually paused on. An unknown id, or one that is not
awaiting an answer, exits 1 and names it - a typo used to be indistinguishable from a correct
answer.

### What cannot be resumed

Only a run that stopped without reaching a verdict is resumable: killed, interrupted, or
paused on a human node. A run whose status is already `failed` is not. `lg resume` on one
prints `run "<id>" is already failed and cannot be resumed` and exits 1, and the same applies
to a run that already succeeded.

That covers a node that failed all its retries and a run stopped by a budget ceiling. There is
no way to fix the cause and continue from the last checkpoint - you start a new run, from the
top, and pay for the completed nodes again. The failed run's checkpoint and event log stay on
disk so you can still read what happened. Resume rescues an interrupted run, not a failed one.

## Node types

| Type | What it does | Fails when |
Expand All @@ -169,7 +190,23 @@ lg resume <runId> --answer approve="ship it"
| `verifier` | Runs an agent CLI, then checks the output for a literal `pass` string | The `pass` string is absent |
| `human` | Pauses the whole run and exits cleanly, holding zero context | Never — it waits |

Every node accepts `retries` (default 0), `timeoutSec` (default 900), and `cwd`.
Every node accepts `retries` (default 0), `timeoutSec` (default 900), and `cwd`. `timeoutSec`
kills the process *group*, not just the direct child, so a command that backgrounds a
grandchild still aborts at the deadline instead of holding the run open until the command ends
on its own.

A `human` node's field is `question`, not `prompt`. It is the one node type that dispatches
nothing, so there is no prompt to send; writing `prompt:` on it fails validation with
`question: Invalid input: expected string, received undefined`.

```yaml
approve:
type: human
question: "Ship the fix? Repro notes: {{nodes.reproduce.output}}"
```

The question is template-interpolated exactly like an agent prompt, so the reviewer reads the
resolved text rather than a raw `{{nodes.reproduce.output}}`.

A `command` node also accepts two optional assertions, because a shell command that exits 0
having done nothing is not a passing check: `expectNonEmpty: true` fails the node when the
Expand All @@ -179,14 +216,22 @@ exist for.

## Budgets

Three ceilings, all enforced *before* each dispatch batch **and once more before a run is
allowed to finish successfully**, all recorded in the checkpoint:
Three ceilings, all enforced *before every single node is dispatched* - each member of a
fan-out batch and each retry attempt separately - **and once more before a run is allowed to
finish successfully**, all recorded in the checkpoint:

- `maxUsd` — summed from what the adapters actually report.
- `maxWallClockSec` — measured from the run's creation, so it survives a resume.
- `maxNodeRuns` — counts every attempt, retries included.

Hitting a ceiling stops the run with status `failed`, a `budget_exceeded` event naming the ceiling, and exit code 3. Nothing further is dispatched.
The ceilings are exclusive: the limit itself is allowed, and only going over it stops the run.
`maxNodeRuns: 20` permits 20 node runs, and `maxUsd: 2.00` permits a run that spends exactly
2.00.

Crossing a ceiling stops the run with status `failed`, a `budget_exceeded` event naming the
ceiling, and exit code 3. Nothing further is dispatched, and that is literal: a fan-out stops
part-way through its batch, the nodes that were never admitted leave no side effect behind, and
a retry that would cross the ceiling is not attempted.

A ceiling breached by the final batch fails the run too. A node that already finished keeps
its result — the run fails, the work does not unwind — so `lg status` still shows what was
Expand Down Expand Up @@ -230,20 +275,25 @@ review:
pass: "PASS"
```

Omit it and the CLI's own resolution decides, which is not always what the config says: with no `-m`, opencode ignored a configured `model` and fell through to a provider with no credentials. `OPENCODE_MODEL` is ignored — the flag is the only way. A `command` or `human` node that declares a model is a validation error rather than a silently ignored key.
Omit it and the CLI's own resolution decides, which is not always what the config says: with no `-m`, opencode ignored a configured `model` and fell through to a provider with no credentials. `OPENCODE_MODEL` is ignored — the flag is the only way. A `command` or `human` node that declares a `model` - or an `adapter` - is a validation error
rather than a silently ignored key.

### Environment

| Variable | Effect |
| --- | --- |
| `CLAUDE_CONFIG_DIR` | Which Claude Code credential directory to use. Set it when your default `~/.claude` session is expired or you keep several logins side by side. |
| `LOOMGRAPH_CODEX_SANDBOX` | Codex sandbox policy: `read-only` (default), `workspace-write`, or `bypass`. |
| `LOOMGRAPH_CODEX_SANDBOX` | Codex sandbox policy: `read-only` (default), `workspace-write`, or `bypass`. Any other value is a hard error naming those three, rather than a silently wider or narrower sandbox. |

### Two failure modes worth knowing

**An expired login does not look like an error.** Claude Code returns `subtype: "success"` *and* `is_error: true` when its OAuth session has lapsed, with the authentication message sitting in the `result` field. An adapter that trusts `subtype` alone records a node that spent nothing, changed nothing, and reported success. loomgraph checks both fields and fails the node with the message the CLI actually returned.

**A verifier that cannot read the tree must fail, not pass.** Codex sandboxes the commands it runs, and some containers cannot start that sandbox at all — every read fails with `bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted`. A review under those conditions is worthless, so the verifier node fails and says why. Set `LOOMGRAPH_CODEX_SANDBOX=bypass` only when the host is already isolated.
**A verifier that cannot read the tree must fail, not pass.** Codex sandboxes the commands it runs, and some containers cannot start that sandbox at all — every read fails with `bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted`. A review under those conditions is worthless, so any line
beginning `bwrap:` on either stderr or stdout fails the node and is quoted back in the error.
The exit code is not consulted for this: bubblewrap fails per tool call rather than at startup,
so the realistic shape is codex exiting 0 with a confident verdict from a run that read
nothing. Set `LOOMGRAPH_CODEX_SANDBOX=bypass` only when the host is already isolated.

Both agent adapters close stdin before spawning. Codex otherwise prints `Reading additional input from stdin...` and waits until the node's timeout fires, which is indistinguishable from a slow model.

Expand All @@ -259,6 +309,10 @@ Both agent adapters close stdin before spawning. Codex otherwise prints `Reading
| `lg report <runId> [--out path] [--publish] [--title t] [--visibility private\|org]` | Render the run to a self-contained html file; `--publish` hosts it with the `enclave` cli |
| `lg events <runId> [--kind K]` | The JSONL audit trail, filterable |

`lg report --visibility` accepts only `private` or `org`; anything else exits 1 and nothing is
published. `lg events --kind` accepts only the nine kinds listed above; an unknown kind exits 1
rather than printing nothing, which used to be indistinguishable from "no such events".

Exit codes: `0` success, `1` validation or usage error, `2` run failed, `3` budget exceeded, `4` paused awaiting a human.

## Handoff
Expand Down Expand Up @@ -382,16 +436,27 @@ file-history snapshots, codex `base_instructions`, MCP config and permission mod
dropped by the readers and never reach the page at all.

Then the scanner runs, and `push` refuses on any hit. It knows URL-embedded credentials
(`scheme://user:pass@host`), `Authorization: Bearer` / `Basic` headers, Anthropic, OpenAI,
Stripe (`sk_` and `rk_`), GitHub, GitLab, Slack, AWS access key ids, GCP, HuggingFace
(`hf_`), Google OAuth (`GOCSPX-`), npm and SendGrid key shapes, JWTs, PEM private keys,
and token/secret/password assignments. The git remote is special-cased: it is published
(`scheme://user:pass@host`), `Authorization: Bearer` / `Basic` headers in both their plain-text
and JSON-encoded (`{"Authorization":"Bearer ..."}`) forms, `.netrc` rows
(`machine <host> login <user> password <secret>`), the OAuth material an opencode `auth.json`
stores under `refresh` / `access` / `credential`, Anthropic, OpenAI, Stripe (`sk_` and `rk_`),
GitHub, GitLab, Slack, AWS access key ids, GCP, HuggingFace (`hf_`), Google OAuth (`GOCSPX-`),
npm and SendGrid key shapes, JWTs, PEM private keys, and token / secret / password / api-key /
bare `key` assignments. The git remote is special-cased: it is published
verbatim and never passes through path rewriting, so a `user:password@` in it is stripped
at the source.

**This is an allowlist of shapes, not a proof.** A credential in a shape it has never seen
goes straight through. Known gaps include AWS secret access keys, PEM bodies without a
header, hex client secrets, and non-home absolute paths. Read the brief before you send
header, hex client secrets, and non-home absolute paths.

One gap is a deliberate trade: an *unquoted* assignment value shorter than eight characters is
not treated as a credential, so a YAML line like `password: abc123` is missed. The alternative
was a scanner that fires on ordinary prose - the line `Standalone token: user` blocked a real
pack. A quoted value is caught at any length, and a genuine credential in a `KEY=value` line is
longer than eight characters, so the `.env` shapes still fire.

Read the brief before you send
the link - it is one screen, and you are the last check. If the scanner cannot read a file
it was asked to scan, it reports that as a finding rather than staying quiet, so "clean"
always means "looked at and found nothing".
Expand Down
51 changes: 49 additions & 2 deletions src/adapters/claude.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
import { describe, it, expect } from "vitest";
import { buildClaudeArgs, parseClaudeJson } from "./claude.js";
import { describe, it, expect, beforeAll, afterAll } from "vitest";
import { mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { buildClaudeArgs, parseClaudeJson, ClaudeAdapter } from "./claude.js";

const SUCCESS = `{"type":"result","subtype":"success","result":"done","session_id":"abc","num_turns":3,"total_cost_usd":0.0787}`;
const MAX_TURNS = `{"type":"result","subtype":"error_max_turns","result":"","total_cost_usd":0.5}`;
Expand Down Expand Up @@ -147,4 +150,48 @@ describe("parseClaudeJson", () => {
expect(Array.isArray(out.raw)).toBe(true);
expect((out.raw as unknown[]).length).toBe(4);
});

it("clamps a negative total_cost_usd to 0", () => {
const out = parseClaudeJson(`{"subtype":"success","result":"ok","total_cost_usd":-1.5}`);
expect(out.ok).toBe(true);
expect(out.costUsd).toBe(0);
});

it("clamps a non-finite total_cost_usd to 0", () => {
const out = parseClaudeJson(`{"subtype":"success","result":"ok","total_cost_usd":1e999}`);
expect(out.costUsd).toBe(0);
});
});

describe("ClaudeAdapter process handling", () => {
// Every test here points the adapter at a throwaway stub script by absolute
// path. No real agent CLI is ever spawned and PATH is never touched.
let dir: string;

beforeAll(async () => {
dir = await mkdtemp(join(tmpdir(), "lg-claude-stub-"));
});

afterAll(async () => {
await rm(dir, { recursive: true, force: true });
});

it("names the binary when it is missing instead of blaming the json parser", async () => {
const bin = join(dir, "definitely-not-installed");
const out = await new ClaudeAdapter(bin).run({ prompt: "hi", cwd: dir, timeoutSec: 10 });
expect(out.ok).toBe(false);
expect(out.error).toContain(bin);
expect(out.error).toMatch(/not found on PATH/);
expect(out.error).not.toMatch(/could not parse/);
});

it("times out when a grandchild outlives the CLI", async () => {
const bin = join(dir, "slow-stub.sh");
await writeFile(bin, `#!/bin/sh\nsleep 30 &\necho '{"subtype":"success","result":"ok"}'\n`, { mode: 0o755 });
const started = Date.now();
const out = await new ClaudeAdapter(bin).run({ prompt: "hi", cwd: dir, timeoutSec: 2 });
expect(out.ok).toBe(false);
expect(out.error).toMatch(/timeout after 2s/);
expect(Date.now() - started).toBeLessThan(3500);
}, 20000);
});
29 changes: 17 additions & 12 deletions src/adapters/claude.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { execa } from "execa";
import { clampCostUsd, runProcess } from "./types.js";
import type { Adapter, AdapterInput, AdapterOutput } from "./types.js";

/**
Expand Down Expand Up @@ -59,7 +59,7 @@ export function parseClaudeJson(stdout: string): AdapterOutput {
}

// Cost is harvested even on failure - budget accounting depends on it.
const costUsd = typeof result.total_cost_usd === "number" ? result.total_cost_usd : 0;
const costUsd = clampCostUsd(result.total_cost_usd);
const text = typeof result.result === "string" ? result.result : "";

// Claude Code can report `subtype: "success"` while `is_error` is true - an
Expand Down Expand Up @@ -91,17 +91,22 @@ export class ClaudeAdapter implements Adapter {

async run(input: AdapterInput): Promise<AdapterOutput> {
const args = buildClaudeArgs(input.prompt, input.maxTurns, input.model);
const result = await execa(this.bin, args, {
cwd: input.cwd,
timeout: input.timeoutSec * 1000,
reject: false,
// Keep the run non-interactive: an open stdin can stall the CLI until the
// node's timeout fires, which is indistinguishable from a hung model call.
input: "",
});
const result = await runProcess(this.bin, args, { cwd: input.cwd, timeoutSec: input.timeoutSec });

const stdout = typeof result.stdout === "string" ? result.stdout : "";
const stderr = typeof result.stderr === "string" ? result.stderr : "";
const { stdout, stderr } = result;

// Without this the spawn error is swallowed, the adapter parses an empty
// stdout, and the run fails with "could not parse claude json output:" -
// which never names the binary that is missing.
if (result.spawnErrorCode === "ENOENT") {
return {
ok: false,
text: "",
costUsd: 0,
raw: { stdout, stderr },
error: `${this.bin} not found on PATH`,
};
}

if (result.timedOut) {
return { ok: false, text: stdout, costUsd: 0, raw: { stdout, stderr }, error: `timeout after ${input.timeoutSec}s` };
Expand Down
Loading
Loading