feat: DAG-style dependency ordering for exec_batch write mode

[Hazzng]

Problem

exec_batch has two modes today:

• Write mode (readOnly omitted/false): strictly sequential under a single exclusive lock (runSequential, batch-exec.ts:33–100). No concurrency ever, even between fully independent scripts.
• Read-only mode (readOnly: true): up to 16 parallel workers under a shared reader lock (runParallel, batch-exec.ts:102–207).

There is no middle ground. A write batch with independent subsets (e.g. "install package A" and "install package B" followed by "run combined tests") cannot parallelise at all — even the independent installs run one after the other.

A DAG mode lets callers declare dependency edges between script entries. The executor topologically-sorts the DAG and runs all ready nodes concurrently (bounded by MAX_BATCH_PARALLELISM = 16) — within a single write-lock acquisition, so the filesystem remains protected.

Proposed API Change

Add an optional deps: string[] field per script entry:

{
  "scripts": [
    { "id": "install-a", "script": "pip install numpy",  "deps": [] },
    { "id": "install-b", "script": "pip install pandas", "deps": [] },
    { "id": "run-tests", "script": "pytest /app/tests",  "deps": ["install-a", "install-b"] }
  ],
  "timeoutMs": 60000
}

install-a and install-b start immediately in parallel. run-tests waits until both finish.

Activation: if any entry carries a non-empty deps field, the batch automatically runs in DAG mode — no separate flag needed.

Python SDK:

results = sb.exec_batch([
    {"id": "a", "script": "echo a"},
    {"id": "b", "script": "echo b"},
    {"id": "c", "script": "echo c", "deps": ["a", "b"]},
])

Implementation

BatchScriptEntry — src/api/lib/batch-exec.ts:5–8

Add readonly deps?: readonly string[].

New runDag() function — src/api/lib/batch-exec.ts (after line 207)

Algorithm: Kahn's BFS topological sort + bounded worker pool (reuses MAX_BATCH_PARALLELISM = 16 and the cursor++ worker pattern from runParallel).

Key points:

• Build inDegree[] and dependents[][] from the deps declarations.
• Seed a ready[] queue with all zero-in-degree nodes.
• Workers advance a shared cursor into ready[]; on each script completion, decrement inDegree of its dependents and push newly-ready nodes, waking sleeping workers via a replaced Promise.
• results[idx] slot assignment (not .push()) preserves input-order output — same contract as runParallel.
• perScriptTimeoutMs wires in identically to runParallel (lines 149–164).
• Shared sharedController / deadlineTimer pattern for the outer timeoutMs ceiling.

executeBatch dispatch — src/api/lib/batch-exec.ts:209–221

const hasDeps = scripts.some((s) => s.deps && s.deps.length > 0);
if (inReadOnlyScope) return runParallel(...);
if (hasDeps)         return runDag(...);
return runSequential(...);

batchExecBodySchema — src/api/routes/exec.ts:37–50

z.object({
  id: z.string(),
  script: z.string(),
  deps: z.array(z.string()).optional(),  // ← add
})

DAG validation — src/api/routes/exec.ts handler (~line 350, before executeBatch call)

After schema parse, validate:

1. Missing dep IDs — dep references an id not present in the batch → HTTP 400
2. Self-reference — deps includes the entry's own id → HTTP 400
3. Cycle detection — run Kahn's BFS; if visited < scripts.length → HTTP 400, report the cycle members
4. readOnly + deps — forbidden → HTTP 400 (readOnly already runs everything in parallel unconditionally)

OpenAPI spec — src/api/openapi-spec.ts:714–726

Add deps to the script entry items schema with description and example.

Edge Cases

Scenario	Behaviour
Dependency exits non-zero	Dependent still runs (matches runSequential pass-through policy)
Missing dep ID	HTTP 400 at request time, before any execution
Self-referential dep	HTTP 400 at request time
Cycle	HTTP 400 at request time; report cycle members
readOnly: true + deps	HTTP 400 — forbidden combination
Empty deps: [] vs omitted	Both mean "no deps, start immediately"
Budget pre-exhausted	Un-started nodes get exitCode: -1, error: "timeout"
perScriptTimeoutMs	Fully compatible, no additional changes
Result ordering	Always input-index order (slot-based assignment, not push)

Files to Change

File	Location	Change
src/api/lib/batch-exec.ts	Lines 5–8	Add deps?: readonly string[] to BatchScriptEntry
src/api/lib/batch-exec.ts	After line 207	New runDag(...) function (~80 lines)
src/api/lib/batch-exec.ts	Lines 217–221	Add hasDeps branch to executeBatch
src/api/routes/exec.ts	Lines 37–50	Add deps to batchExecBodySchema script item
src/api/routes/exec.ts	~Line 350	Add DAG validation block (4 checks)
src/api/openapi-spec.ts	Lines 714–726	Add deps to script item schema

Tests to Add (src/api/tests/unit/exec-batch-dag.test.ts)

1. Independent scripts run in parallel (assert maxConcurrent > 1)
2. Dependency ordering is respected (script B must not start until A completes)
3. Multi-wave fan-out (a,b → c(deps:a), d(deps:b) → e(deps:c,d))
4. Result order matches input order regardless of execution order
5. MAX_BATCH_PARALLELISM = 16 cap respected in a single wave of 50 scripts
6. Deadline timeout aborts in-flight and queued nodes
7. perScriptTimeoutMs kills a slow dep; its dependent still runs
8. Failed dep (exit 1) does not block dependent
9. HTTP: valid DAG → 200
10. HTTP: missing dep ID → 400 INVALID_INPUT
11. HTTP: self-referential dep → 400 INVALID_INPUT
12. HTTP: cycle → 400 INVALID_INPUT
13. HTTP: deps + readOnly: true → 400 INVALID_INPUT
14. HTTP: no deps field → sequential behaviour unchanged (regression guard)