diff --git a/docs/README.md b/docs/README.md index 2c23f257..bc915ad0 100644 --- a/docs/README.md +++ b/docs/README.md @@ -47,6 +47,7 @@ These are internal working documents: design theses, research narrative, and roa | Doc | Role | Purpose | |---|---|---| +| [design/prime-agent-harness-integration.md](./design/prime-agent-harness-integration.md) | integration contract | Prime Agent as a sandbox-materialized harness: the boundary, the anti-reinvention map, the substrate wish-list, and the first gated experiment. | | [simplification-plan.md](./research/simplification-plan.md) | historical tracker | Earlier simplification analysis. The active execution and API convergence plan is [agent-managed-compute/roadmap.md](./agent-managed-compute/roadmap.md). | | [research/README.md](./research/README.md) | research index | Forward-looking design threads + decision log. Not the canonical spine. | | [archive/](./archive/) | retired notes | Superseded/niche docs kept for history (delivery manifest, conversation economics, artifact-lifecycle, go-live, results, benchmark-matrix consolidation). | diff --git a/docs/api/primitive-catalog.md b/docs/api/primitive-catalog.md index d5df35d7..247dfe54 100644 --- a/docs/api/primitive-catalog.md +++ b/docs/api/primitive-catalog.md @@ -524,7 +524,7 @@ Import from `@tangle-network/agent-runtime/intelligence` — 166 exports. ### Execution kernel — recursive atom, supervision, executors, round-synchronous loop -Import from `@tangle-network/agent-runtime/kernel` — 745 exports. +Import from `@tangle-network/agent-runtime/kernel` — 755 exports. | Symbol | Kind | Summary | |---|---|---| @@ -544,6 +544,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 745 exports. | `auditIntent` | function | The route-rigor analyst: compare declared vs revealed vs user intent over a trajectory and return aligned / drifting / diverged with evidence and one recommended intervention. | | `authorStrategy` | function | Author + load a strategy from losses. Throws when the author emits no loadable module; | | `bestSoFar` | function | The best-so-far fold — the ONE definition of "how good was the run after k results", shared by | +| `boxSurfaceReader` | function | A {@link SurfaceReader} over a sandbox box's filesystem — the same `box.fs.read` seam | | `breadthStrategy` | function | BREADTH: K independent rollouts (each own artifact), verifier picks the best. | | `buildSteerContext` | function | Build the `SteerContext` a combinator reads to steer (its `loopUntil.until`, `widen` gate, any | | `canDisplace` | function | The repair keep-best guard: a challenger displaces the incumbent only when it is | @@ -614,9 +615,11 @@ Import from `@tangle-network/agent-runtime/kernel` — 745 exports. | `flatWidenGate` | function | The flat default `ScopeWidenGate` — never widens, keeping the R2 selector≠judge collision | | `formatPromptHandle` | function | The string form of a handle: `/v`. | | `freeSlots` | function | Free worker slots under a simultaneity cap: `cap - live`, floored at 0, or `null` when there is | +| `fsSurfaceReader` | function | A {@link SurfaceReader} over the local filesystem, for worktree/local workers. Every path — | | `gateOnDeliverable` | function | Wrap an `Executor` so its settlement `valid` reflects the deliverable check, not the | | `gitWorkspace` | function | A `Workspace` over a git checkout: materialize an isolated worktree at `ref`, commit produced changes (conflict-aware), and read `head` — hooks disabled, identity pinned. | | `harvestCorpus` | function | Batch the firewalled `observe()` analyst over completed runs and accrete the trace-derived facts into the durable corpus — the production-traces→corpus write side of the flywheel. | +| `harvestSurfaceDiffs` | function | Re-read every mounted (and watched) surface and report the ones whose settled state differs from | | `inlineSandboxClient` | function | Adapt an `ExecutorFactory` into a `SandboxClient` for `runAgentRounds`. The factory is | | `inProcessSandboxClient` | function | Adapt a single `onPrompt(prompt, ctx)` callback into a `SandboxClient` for | | `isWaitOutcome` | function | Narrow a settlement's `out` to a wait outcome — a wait settles on the SAME cursor as workers, | @@ -804,6 +807,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 745 exports. | `AuthorizedSpawnContext` | interface | Exact trusted context after a manager-authored spawn has passed product authorization. | | `BenchmarkCell` | interface | One strategy's outcome on one task — the per-task cell an optimizer consumes. | | `BenchmarkReport` | interface | Benchmark output: per-strategy means plus the full per-task × per-strategy losses table an optimizer mines. | +| `BoxSurfaceReaderOptions` | interface | Retry and cancellation controls for {@link boxSurfaceReader}. | | `BridgeModelCredential` | interface | A live, request-scoped model credential reference for a local cli-bridge. | | `BridgeSeam` | interface | cli-bridge seam. A local OpenAI-compatible bridge that fronts harness CLIs | | `Budget` | interface | A budget envelope on a spawn or the root. All ceilings; the pool reserves against them. | @@ -872,6 +876,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 745 exports. | `ForkCapableBox` | interface | Loop-side widening of the box's optional fork method. | | `GraphNode` | interface | A graph node: an id and a canonical `AgentProfile`. The profile is the ONLY way a node is | | `Handle` | interface | A live child handle. `abort()` is defined over the ACQUIRE lifecycle: it chains into | +| `HarvestSurfaceDiffsOptions` | interface | Inputs to {@link harvestSurfaceDiffs}: the run's mount manifest, the read seam, and optional | | `InboxMessage` | interface | The worker-side receive end of the down-leg: a per-worker inbox an executor exposes as | | `InMemoryRunContext` | interface | The bundle of stores a supervised run needs, shaped to spread into `SupervisorOpts`. | | `InMemoryRunContextOptions` | interface | Options for a supervised run context. | @@ -999,6 +1004,8 @@ Import from `@tangle-network/agent-runtime/kernel` — 745 exports. | `SupervisorSpanOutcome` | interface | How the supervised run ended, as `finish()` records it on the root span. | | `SupervisorToolDescriptor` | interface | One product-owned tool. It reuses the canonical MCP descriptor fields while Runtime supplies | | `SupervisorToolInvocationContext` | interface | Trusted context for one product-tool invocation. The node identity remains the same detached, | +| `SurfaceDiff` | interface | One watched surface whose settled state differs from what was mounted (or from absence). | +| `SurfaceReadBox` | interface | The minimal box surface the box-backed reader needs — structurally typed so the real | | `SurfaceWorkerConfig` | interface | How a worker runs the surface task (its router substrate + per-attempt bounds). | | `SurfaceWorkerOut` | interface | What a surface worker settles with — the surface verdict the driver + deliverable read. `resolved` is | | `ToolLoopCompaction` | interface | Self-compaction — bound the loop's OWN context window the way a fresh-respawn (dumb-Ralph) loop | @@ -1013,6 +1020,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 745 exports. | `WaitOpts` | interface | Options for `Scope.wait`. `label` is the wait's identity within its parent scope — it is what | | `WaitOutcome` | interface | The `out` a settled wait node delivers through `Scope.next()`. `settled` is the outcome the | | `WaitProbeRegistry` | interface | Resolves a `poll` spec's `probe` name to its predicate. Threaded through `SupervisorOpts` so | +| `WatchedSurface` | interface | A path to check at settle that was NOT necessarily mounted — where a harness is known to write | | `WidenGate` | interface | The progressive-widening gate (MCTS-PW). Decides whether a settled child is | | `WidenLineage` | interface | A lineage the gate may widen toward — the settled child that looked promising + the findings | | `WidenSpec` | interface | `widen({ gate })` (G5) — the STREAMING spawn-on-completion driver. Unlike the static-fanout | @@ -1107,6 +1115,8 @@ Import from `@tangle-network/agent-runtime/kernel` — 745 exports. | `SupervisorNodeContextSeed` | type | Context known before `Agent.act`; Runtime adds the concrete node, profile, and task. | | `SupervisorProfile` | type | A supervisor is an exact canonical AgentProfile; no looser model/prompt shape exists. | | `SupervisorSpanAttributes` | type | OTLP span attribute values. Exported because `SupervisorSpanOptions.attributes` is public and | +| `SurfaceReader` | type | The read seam: fetch the current bytes at a mounted path. Implemented by a sandbox box's | +| `SurfaceReadOutcome` | type | Outcome of reading one surface back at settle. `missing: true` means the path no longer exists | | `ToolLoopCompactionOptions` | type | Public supervisor-facing compaction config: same knobs as the primitive, but `distill` is optional | | `ToolLoopMessageRecord` | type | Provider-neutral conversation record accepted by a tool-loop brain. | | `TrajectoryReportFn` | type | `trajectoryReport(...)` — the tree+cost reconstructor. Async (reads journal + optionally blobs). | diff --git a/docs/api/runtime.md b/docs/api/runtime.md index 08d8225d..340f8da4 100644 --- a/docs/api/runtime.md +++ b/docs/api/runtime.md @@ -16889,6 +16889,173 @@ returning an incomplete reproducibility receipt. *** +### SurfaceDiff + +One watched surface whose settled state differs from what was mounted (or from absence). + +- `modified` — the surface exists with different bytes (`settledSha256`/`settledBytes` present). +- `removed` — the surface no longer exists at its mounted path. +- `created` — a watched path that was never mounted now exists (`settledSha256`/`settledBytes` + present, no `mountedSha256`) — the shape a harness's new memory/skill file takes. +- `unreadable` — the read seam failed for a reason other than absence; `error` carries the + diagnostic. Reported rather than dropped so a permissions or transport failure cannot + masquerade as "nothing changed". + +#### Properties + +##### path + +> **path**: `string` + +The mounted/watched path, exactly as recorded. + +##### status + +> **status**: `"modified"` \| `"removed"` \| `"created"` \| `"unreadable"` + +##### mountedSha256? + +> `optional` **mountedSha256?**: `string` + +Hex SHA-256 of the bytes that were mounted (from the manifest). Absent for `created`. + +##### source + +> **source**: `string` + +Free-form origin: the manifest entry's `source`, or the watch entry's `source`. + +##### settledSha256? + +> `optional` **settledSha256?**: `string` + +Hex SHA-256 of the settled bytes. Present for `modified` and `created`. + +##### settledBytes? + +> `optional` **settledBytes?**: `number` + +Size of the settled bytes. Present for `modified` and `created`. + +##### error? + +> `optional` **error?**: `string` + +The read seam's diagnostic. Present only for `unreadable`. + +*** + +### WatchedSurface + +A path to check at settle that was NOT necessarily mounted — where a harness is known to write + self-authored surfaces (a memory dir's files, a refinement log). A watched path that was also + mounted compares against its mount; one that wasn't reports `created` if it now exists. + `created` is an inference from the mount manifest, not a proof of authorship: a file the box + IMAGE shipped at a never-mounted path also reports `created`. Watch paths known absent at run + start (or enumerate the tree at start AND settle and watch the difference) to make the label + mean what it says. + +#### Properties + +##### path + +> **path**: `string` + +##### source? + +> `optional` **source?**: `string` + +Origin label carried onto the diff (default `'watched'`). + +*** + +### HarvestSurfaceDiffsOptions + +Inputs to [harvestSurfaceDiffs](#harvestsurfacediffs): the run's mount manifest, the read seam, and optional + watch paths for surfaces the agent may have created. + +#### Properties + +##### mounts + +> **mounts**: readonly [`MountManifestEntry`](#mountmanifestentry)[] + +The run's mount manifest (`RunProvenance.mounts`). Entries sharing a path are collapsed to the + LAST entry — the bytes the agent actually saw at start. + +##### read + +> **read**: [`SurfaceReader`](#surfacereader) + +How to read a mounted path's current bytes. + +##### watch? + +> `optional` **watch?**: readonly [`WatchedSurface`](#watchedsurface)[] + +Additional paths to check that may not have been mounted (see [WatchedSurface](#watchedsurface)). The + caller enumerates them (it knows the harness's state layout — e.g. via the box's file tree); + the harvest stays layout-agnostic. + +*** + +### SurfaceReadBox + +The minimal box surface the box-backed reader needs — structurally typed so the real + `@tangle-network/sandbox` box and a test double both satisfy it, no SDK import. + +#### Properties + +##### fs + +> **fs**: `object` + +###### read() + +> **read**(`path`): `Promise`\<`string`\> + +###### Parameters + +###### path + +`string` + +###### Returns + +`Promise`\<`string`\> + +*** + +### BoxSurfaceReaderOptions + +Retry and cancellation controls for [boxSurfaceReader](#boxsurfacereader). + +#### Properties + +##### attempts? + +> `optional` **attempts?**: `number` + +Read attempts per path before settling on a failed outcome. The data plane can transiently + 404 a just-written file (the same blip `openSandboxRun`'s deliverable read retries for), and a + first-attempt 404 taken at face value turns a fresh self-edit into a false `removed`/dropped + `created`. Default 3. + +##### retryDelayMs? + +> `optional` **retryDelayMs?**: `number` + +Linear backoff base between attempts (delay = base × attempt). Default 250. + +##### signal? + +> `optional` **signal?**: `AbortSignal` + +Cuts the retry waits short when the run is abandoned. The reader still returns a typed + outcome — the harvest reports what it managed to read rather than rejecting. + +*** + ### CreateTangleSandboxExactProcessProviderOptions #### Properties @@ -20507,6 +20674,34 @@ Terminal artifact of one worktree-CLI run — the canonical worktree-harness res *** +### SurfaceReadOutcome + +> **SurfaceReadOutcome** = \{ `succeeded`: `true`; `value`: `Uint8Array`; \} \| \{ `succeeded`: `false`; `missing`: `boolean`; `error`: `string`; \} + +Outcome of reading one surface back at settle. `missing: true` means the path no longer exists + (a deletion — a valid, reportable outcome); any other failure carries its diagnostic. + +*** + +### SurfaceReader + +> **SurfaceReader** = (`path`) => `Promise`\<[`SurfaceReadOutcome`](#surfacereadoutcome)\> + +The read seam: fetch the current bytes at a mounted path. Implemented by a sandbox box's + `fs.read`, a local worktree read ([fsSurfaceReader](#fssurfacereader)), or a test double. + +#### Parameters + +##### path + +`string` + +#### Returns + +`Promise`\<[`SurfaceReadOutcome`](#surfacereadoutcome)\> + +*** + ### SandboxControlClient > **SandboxControlClient** = `Pick`\<`Sandbox`, `"create"` \| `"get"` \| `"list"`\> @@ -26112,6 +26307,88 @@ the shared valid-only `selectValidWinner` (never a judge). *** +### harvestSurfaceDiffs() + +> **harvestSurfaceDiffs**(`options`): `Promise`\<[`SurfaceDiff`](#surfacediff)[]\> + +Re-read every mounted (and watched) surface and report the ones whose settled state differs from +the manifest — modified, removed, or created. Unchanged surfaces and still-absent watched paths +produce no entry; reads run concurrently; output preserves record order, mounts before +watch-only paths. Mounts and watches sharing a path key are each collapsed to the LAST entry, +and a watched path that was also mounted compares against its mount (never reports `created`). + +The harvest takes no `AbortSignal`: it is pure fan-out over the read seam and waits on nothing +itself, so every cancellable moment belongs to the reader. Pass a signal to the reader instead +([BoxSurfaceReaderOptions.signal](#signal-23), or close over one in a custom [SurfaceReader](#surfacereader)) — +that cuts the backoff waits, and the harvest still returns the diffs it did establish rather +than discarding settle-time evidence on a late cancellation. + +#### Parameters + +##### options + +[`HarvestSurfaceDiffsOptions`](#harvestsurfacediffsoptions) + +#### Returns + +`Promise`\<[`SurfaceDiff`](#surfacediff)[]\> + +*** + +### boxSurfaceReader() + +> **boxSurfaceReader**(`box`, `options?`): [`SurfaceReader`](#surfacereader) + +A [SurfaceReader](#surfacereader) over a sandbox box's filesystem — the same `box.fs.read` seam +`openSandboxRun` reads deliverables through, with the same transient-404 posture (bounded +retry). The box wire returns UTF-8 TEXT (the SDK's binary path is `download()`), which profile +surfaces are; hashes are computed over the UTF-8 encoding, and content the wire had to +lossy-decode (a U+FFFD replacement character) is reported `unreadable` rather than hashed as +mojibake. The SDK's not-found error is detected structurally (`err.name === 'NotFoundError'`) +and maps to `missing: true` — unless its `resourceType` names something other than a file/path +(the BOX or session being gone), which is a transport failure, not an absent surface. + +#### Parameters + +##### box + +[`SurfaceReadBox`](#surfacereadbox) + +##### options? + +[`BoxSurfaceReaderOptions`](#boxsurfacereaderoptions) = `{}` + +#### Returns + +[`SurfaceReader`](#surfacereader) + +*** + +### fsSurfaceReader() + +> **fsSurfaceReader**(`root`): [`SurfaceReader`](#surfacereader) + +A [SurfaceReader](#surfacereader) over the local filesystem, for worktree/local workers. Every path — +relative or absolute — must resolve INSIDE `root`: a path that escapes it (`../`, an absolute +path elsewhere) fails as a contained non-missing outcome rather than reading outside the +worktree, so a persisted or mistyped manifest path cannot turn the harvest into an +existence/hash oracle over the host filesystem. Containment is checked twice — once on the +lexical path, then again on the symlink-resolved path, because `readFile` follows a link and a +link planted inside the root would otherwise read host bytes through a contained-looking name. +Absence maps to `missing: true`; every other failure carries the error message. + +#### Parameters + +##### root + +`string` + +#### Returns + +[`SurfaceReader`](#surfacereader) + +*** + ### createTangleSandboxExactProcessProvider() > **createTangleSandboxExactProcessProvider**(`client`, `options?`): `AgentEnvironmentProvider` diff --git a/docs/design/prime-agent-harness-integration.md b/docs/design/prime-agent-harness-integration.md new file mode 100644 index 00000000..2cefd7d9 --- /dev/null +++ b/docs/design/prime-agent-harness-integration.md @@ -0,0 +1,204 @@ +# Prime Agent as a harness — boundary, adoption map, substrate wish-list + +> **Status: decision record + integration contract.** Prime Agent (Prime Intellect's +> local agent OS: persistent IPython kernel, daemon-backed session trees, native `rlm(…)` +> subagents, `/refine` continual-harness edits) is being added as a **sandbox-materialized +> harness** — the sandbox/adc side owns the image and the adapter; the harness id shipped +> as `prime` in `agent-interface` and the sandbox backend enum. This doc owns the Runtime side: the boundary we hold, which proposed +> integration machinery we adopt or reject (most of it already exists here), and the +> concrete asks on the substrate. On architecture conflict, +> [architecture.md](../architecture.md) wins. + +## 1. The boundary (the one non-negotiable) + +**A harness owns its native inner model/tool/subagent loop. Runtime owns the external +execution identity, budget, lifecycle contract, and cross-harness coordination. Agent +Eval owns verdicts. Intelligence owns promotion.** + +This is not new policy — it is [canonical-api.md §1.5](../canonical-api.md) applied to a +harness that happens to be unusually complete. Acceptance criteria: + +- No Prime-specific inner-loop logic enters Runtime. Prime's IPython surface, session + JSONL schema, daemon protocol, and TUI are harness features, not Runtime contracts. +- No Runtime tool loop wraps Prime's own loop. Prime is a **leaf** behind the one + `Executor` port / a sandbox backend, exactly like claude-code or codex. +- Prime's local `AgentConnection`/daemon protocol never becomes a public Runtime API. + Callers see opaque run ids, session ids, artifact refs, and receipts. +- A stronger harness is a reason to author better profiles, never to specialize Runtime: + a lever Prime exposes that the profile cannot yet express is a **materializer gap in + the substrate**, not a bespoke Prime adapter here. + +## 2. Anti-reinvention map — what the integration needs that already exists + +Most of the machinery a "host Prime properly" plan calls for is built. Reuse it. + +| Integration need | Existing primitive | Where | +|---|---|---| +| Harness identity + per-harness capability truth | `HarnessType` + the harness-capability layer (model lock, reasoning clamp, selector honoring) | `@tangle-network/agent-interface` (`harness.ts`, `harness-capabilities.ts`) | +| Environment capability negotiation (streaming, sessions, workspace, **checkpoint/fork**) | `AgentEnvironmentCapabilities` incl. `branching.checkpoint/fork`; `AgentEnvironment.checkpoint()/fork()` | `src/runtime/environment-provider.ts` | +| Portable capability with per-harness lowerings ("one skill, many bindings") | `CertifiedCapability` = `{ iface, binding, auth, provenance }` — interfaces closed, bindings open; a Prime Python skill is one binding arm (`file`/`sandbox-code`), never a new manifest format | `src/intelligence/capability.ts` | +| Execution-identity receipts (what exactly ran) | `materialized` journal event (authored/effective profile + platform-attachment digests), `RunProvenance.mounts`, `SelectionReceipt`, usage integrity guard | `src/runtime/supervise/`, `src/runtime/types.ts` | +| Frozen candidate identity + workspace bytes | `buildAgentCandidateBundle` / `captureAgentCandidateWorkspace` | `src/candidate-execution/` | +| Improvement governance (train/selection/final-test partitions, review, activation, rollback receipts) | `improve()` + `proposeAgentImprovement` → `reviewAgentImprovementProposal` → activation | `src/improvement/`, `src/intelligence/` | +| Equal-compute topology comparison + holdout gates | conserved budget pool (`src/runtime/supervise/budget.ts`), `promotionGate`, `pairedLift`, `heldOutGate` | `src/runtime/promotion-gate.ts`, `bench/src/stats.mts`, `@tangle-network/agent-eval` contract | +| Harness × model × task matrix | `defineLeaderboard` / `runProfileMatrix` (harness×model axes) | `src/runtime/define-leaderboard.ts`, `@tangle-network/agent-eval` campaign | +| Harness-agnostic trace normalization | `TraceSource` + the per-harness `toolPartDecoders` registry (add a harness = one validated decoder entry) | `src/runtime/supervise/trace-source.ts` | +| Uncertain-effect posture after restart | uncertain reservations charged at full declared budget with telemetry explicitly unknown; instruction receipts retained as evidence, never auto-redelivered | `src/runtime/supervise/budget.ts`, `supervise/coordination-mcp.ts` | +| Prime Intellect ecosystem bridge (verifiers env packaging, trace import) | `writePrimeIntellectPackage`, `importPrimeIntellectTraces` → `RunRecord` | `src/primeintellect/` | + +## 3. What Runtime adopts + +### 3.1 Surface-diff harvest (built — `harvestSurfaceDiffs`, `/kernel`) + +The harness-agnostic generalization of Prime's `/refine` observation, and the read-back +dual of the mount manifest: `RunProvenance.mounts` records what the substrate placed into +a run (instructions, skills, memory); `harvestSurfaceDiffs({ mounts, read })` re-hashes +those surfaces at settle and reports what the agent itself changed +(`src/runtime/surface-diff.ts`). Every harness self-mutates this way — claude-code edits +CLAUDE.md/memory files, opencode edits AGENTS.md, Prime edits its continual harness — and +today that evidence evaporates at box teardown. + +The harvest is **caller-invoked at settle by the same caller that recorded the mounts** +(the kernel never reads workspace contents itself — same law as `recordMount`). Two +shipped readers join it to real substrates: `boxSurfaceReader` rides the same +`box.fs.read` seam `openSandboxRun` reads deliverables through; `fsSurfaceReader` +covers worktree/local workers. Surfaces the agent *created* (a new memory file — the +common `/refine` outcome) are covered by `watch` entries: the caller enumerates the +harness-state paths it cares about (e.g. via the box file tree) and never-mounted +paths that now exist report as `created`. + +The scope law it feeds (Prime's `/refine` scopes, generalized to any harness): + +- **Session-scoped self-edits are observations.** They applied during the run, they are + recorded (path, mounted/settled hashes, source), they promote nothing. +- **Reuse-scoped edits become candidates.** A diff worth keeping is materialized through + the existing proposal pipeline (`proposeAgentImprovement` → review → gated activation) + and must clear the same held-out, equal-compute gate as any other candidate. An LLM + rationale ("expectedOutcome") is never promotion evidence — the trajectory that + motivated an edit is not an independent measurement of it. + +### 3.2 The `prime` id — shipped and wired + +The id shipped as **`prime`** in `agent-interface` (`HarnessType`; capability rows for +its reasoning ladder and system-prompt semantics) and in the sandbox backend enum, and +`'prime'` is in `harnessBackends` (`src/runtime/sandbox-backend.ts`) — the +double-`satisfies` pin keeps the three enums aligned at compile time. Two follow-ups +remain, each gated on evidence rather than releases: + +1. Register a `toolPartDecoders['prime']` entry — ONLY once validated against Prime's + real session output (the registry law). If the substrate normalizes into the + canonical `ToolPart` (wish-list item 3), the existing decoder covers it and the + entry is a one-line alias; until then unregistered harnesses fall through to the + try-all composite, which is correct. +2. Fill the **measured** `prime` column in + [research/harness-compat.md](../research/harness-compat.md). Prime is Pi-lineage but + not Pi (the fork's daemon rejects pi-line clients): do not inherit `pi`'s capability + row by assumption. + +## 4. What we deliberately do NOT build + +- **A `TopologyAuthority` type.** The fear it answers — Runtime spawns 5 Prime roots, + each spawns 5 native children, unbounded — is structurally impossible here: a worker's + native subagents spend **inside its conserved-budget reservation** (`supervise/budget.ts`), + so no arm can buy more compute regardless of what the harness does internally. The + layering doctrine already assigns native orchestration to the harness (layer one) and + cross-harness composition to Runtime. The three "modes" are expressible today as + authoring decisions: harness-owned = spawn one worker and let it expand; runtime-owned + = author a profile with native subagents disabled (where the harness supports it — see + the compat matrix); hybrid = the default. What is genuinely missing is *visibility* + into native children (wish-list 4) and *fan-out caps as a materializable profile lever* + (wish-list 8) — observability and configuration, not a new authority enum. +- **A `HarnessExecutionReceipt` envelope.** The fields exist across the `materialized` + journal event, `RunProvenance`, the budget ledger, and candidate bundles. Missing + fields (environment/image digest, harness version, harness-state digest, snapshot + refs) are substrate facts — the fix is the substrate *reporting* them (wish-list 5) so + existing receipts can carry them, not a parallel envelope Runtime must keep consistent + with the ones it already emits. +- **An `AdvancedHarnessBackend` optional-method interface.** Capability truth lives in + agent-interface (`harness-capabilities`, `AgentEnvironmentCapabilities`) and behavior + behind the existing ports (`Executor.progress`/`traceSource`, provider + `checkpoint`/`fork`). Growing the backend port into a 10-optional-method surface + re-creates the closed adapter zoo the one `Executor` port replaced. +- **A persistent-kernel / programmable-computer abstraction in Runtime.** Prime's IPython + surface is a strong cognitive substrate and a real reason to route long-horizon + data-heavy tasks to Prime — via the leaderboard matrix and profile authoring, not via a + Runtime kernel API. If the pattern proves out empirically, the portable expression is a + capability binding (a code-execution MCP) any harness can mount, measured like any + other candidate. +- **Prime's daemon/durability layer inside Runtime.** Reconnect, leases, adoption, + replay cursors belong to the sandbox/orchestrator boundary (see + [agent-managed-compute/](../agent-managed-compute/)). Prime's daemon patterns + (generation-aware cursors, claim-before-delivery, explicit uncertain mutations) are + good prior art for that work where Runtime doesn't already have the equivalent. + +## 5. Substrate wish-list (the asks on `@tangle-network/sandbox` / adc) + +Numbered so the sandbox-side work can check them off. Items 1–5 are required for +instrumented runs; 6–9 unlock the experiment tier. + +1. **Harness identity — DONE.** Shipped as `'prime'` in `HarnessType` (agent-interface, + with reasoning-ladder and prompt-channel capability rows) and as a sandbox + `backend.type` — a distinct id, not `pi`. The remaining capability facts (model lock, + selector honoring per the real CLI) still land measured, not inherited. +2. **Profile materialization.** Render an `AgentProfile` into Prime's native surfaces — + system-prompt addendum, skills (as Prime executable skills or SKILL.md), MCP config, + subagent specs, hooks where expressible — with the standard materialization receipt + (exact mounted bytes + digests). A profile lever Prime cannot express must fail loud + in the materializer, not silently drop. +3. **Canonical tool parts.** Normalize Prime session output into agent-interface's + published `ToolPart`/`ToolState` (terminal-state semantics + `callId` dedup), the + shape every adc sdk-provider already targets. Then Runtime's existing decoder handles + Prime with zero new decode logic. If normalization is impossible for some part kinds, + publish the raw wire shape so a decoder entry can be *validated* (the registry rule: + a decoder lands only checked against the harness's real output). +4. **Native session tree exposure.** Prime's root session and `rlm` children as + session-tree metadata (session id + parent id + status + per-child usage) readable + off the box. Runtime projects these as child spans/runs; without it a Prime worker is + one opaque blob and per-child cost attribution is lost. +5. **Environment identity reporting.** Image digest, Prime version, Python-environment + digest, and a continual-harness state digest surfaced on the box/session so run + receipts can bind results to the *effective* behavioral identity (profile digest alone + under-identifies a harness with mutable local state). +6. **Kernel-cell and effect visibility.** One IPython cell can read 100 files, hit the + network, and spawn children. Surface cell start/end, shell subprocesses, and file + effects as parts or a metadata channel so trace analysts see causal structure, not + one giant tool call. (Order/count first; byte-accurate effect receipts can come + later.) +7. **Refinement events.** Prime `/refine` plans and applied edits (scope, surface, + before/after hashes, rationale) surfaced on the session stream, and scope policy + enforceable at materialization: session-local edits apply freely; project/global + edits are emitted as events for the host to lift into proposals, never self-applied + to shared state from inside a run. During gated evaluation: auto-refinement off, + candidate and baseline harness state frozen and separated. +8. **Fan-out caps as config.** Prime's native child limit (and equivalents on other + harnesses) as a materializable knob so an authored profile can bound native expansion. +9. **Checkpoint/branch coverage for the Prime process tree.** The mechanism already + exists — `box.snapshot()` / `box.branch(count)` in the sandbox SDK, the kernel's + CRIU capability probe (`SandboxClient.criuStatus`) and fork lineage + (`LoopLineageOptions.forkFanout`), and the `AgentEnvironment.checkpoint()/fork()` + contract. The ask reduces to: **verify** a CRIU/box snapshot actually covers the + Prime daemon + IPython kernel process tree (a whole-box checkpoint should capture + kernel heap by construction — verify, don't assume), and report the answer through + the environment capability row. A transcript-tree branch is not a world branch; + paired baseline/candidate arms need same-state forks. Credentials stay short-lived + and injected — never serialized into snapshots, session JSONL, or harness state. + +**Adapter shape note.** Prime is daemon-backed with persistent sessions, which maps more +naturally onto the `AgentEnvironmentProvider` contract (sessions, capability +negotiation, `checkpoint`/`fork` — `src/runtime/environment-provider.ts`) than onto a +plain one-shot `backend.type`. Both integration shapes reach Runtime through existing +ports (`providerAsExecutor` / `providerAsSandboxClient` on one side, `buildBackendOptions` +on the other); the sandbox side should pick per tier — box backend for basic runs, +provider for instrumented/experiment tiers — rather than forcing one. + +## 6. The first gated experiment (when 1–5, 7, 9 land) + +One question, run through the existing machinery (no new rig): **does a +`/refine`-derived edit improve held-out tasks, or only continuity on the task that +produced it?** Source tasks generate candidate edits; `improve()`'s partitions select +and hold out; both arms run frozen harness state at equal compute under a deployable +verifier; `proposeAgentImprovement` carries the winner to review. Ablate edit kinds +(memory / skill / subagent spec / bundle) against a same-size generic-edit control. +Powered paired n per the eval substrate's rules — not anecdotes. Separately (never in +the same causal claim): Prime vs other harnesses on the same profile via the +leaderboard matrix. diff --git a/docs/research/harness-compat.md b/docs/research/harness-compat.md index 7f11eb0d..31df835e 100644 --- a/docs/research/harness-compat.md +++ b/docs/research/harness-compat.md @@ -28,5 +28,19 @@ Harnesses wired via cli-bridge: **claude-code** (2.1.177), **codex** (0.139.0), 5. **Three different MCP wiring mechanisms** (file-flag / synthetic-HOME-TOML / env-config-file) — no uniform `--mcp-config`. opencode http = `"type":"remote"`+`url` (the MEMORY `transport:'http'` note is the *claude/kimi* `--mcp-config` layer, a different file). 6. **Resume identity differs per harness** (claude uuid / codex thread_id / opencode session id) — a driver resuming across a sandbox boundary must keep the external→internal id map per-harness; ids are not cross-harness valid. +## Pending columns + +- **prime** — Prime Intellect's agent OS (persistent IPython kernel, native `rlm(…)` + subagents, `/refine` continual-harness edits, daemon-backed session trees). The id + shipped in `agent-interface` `HarnessType` + the sandbox backend enum; see + [design/prime-agent-harness-integration.md](../design/prime-agent-harness-integration.md). + **Do not inherit `pi`'s row** — Prime is Pi-lineage but the fork's wire protocol has + diverged (its daemon rejects pi-line clients); every cell must be measured against the + real CLI before this column fills in. + Expected steering-relevant deltas to verify: native subagent fan-out cap and whether it + is configurable (runaway surface #3), whether the daemon protocol allows mid-step + interrupt (black-box harnesses today degrade `steer_agent({interrupt:true})` to next + spawn), and whether tool activity surfaces as structured parts or only via the kernel. + ## Files `~/code/cli-bridge/src/backends/{claude,codex,opencode}.ts` (invocation), `.../profile-support.ts` (MCP materializers), `.../modes.ts` (byob/hosted-safe gating); `~/code/agent-dev-container` sidecar registry (12 bindings). diff --git a/src/runtime/box-read-retry.test.ts b/src/runtime/box-read-retry.test.ts new file mode 100644 index 00000000..5eed0ed7 --- /dev/null +++ b/src/runtime/box-read-retry.test.ts @@ -0,0 +1,103 @@ +import { describe, expect, it } from 'vitest' +import { boxReadErrorMessage, readBoxPathWithRetry } from './box-read-retry' + +describe('readBoxPathWithRetry', () => { + it('returns the text as soon as an attempt succeeds, without spending the rest', async () => { + let calls = 0 + const read = () => { + calls += 1 + return calls < 3 ? Promise.reject(new Error('404')) : Promise.resolve('flushed') + } + const result = await readBoxPathWithRetry(read, 'a.md', { attempts: 4, delayMs: 0 }) + expect(result).toEqual({ succeeded: true, text: 'flushed' }) + expect(calls).toBe(3) + }) + + it('carries the LAST error out after every attempt fails', async () => { + let calls = 0 + const read = () => { + calls += 1 + return Promise.reject(new Error(`attempt ${calls}`)) + } + const result = await readBoxPathWithRetry(read, 'a.md', { attempts: 3, delayMs: 0 }) + expect(calls).toBe(3) + expect(result.succeeded).toBe(false) + if (!result.succeeded) expect(boxReadErrorMessage(result.error)).toBe('attempt 3') + }) + + it('lets beforeAttempt abandon the read with the caller’s own error', async () => { + let calls = 0 + const read = () => { + calls += 1 + return Promise.reject(new Error('transient')) + } + await expect( + readBoxPathWithRetry(read, 'a.md', { + attempts: 5, + delayMs: 0, + beforeAttempt: (lastError) => { + if (lastError !== undefined) + throw new Error(`gave up after: ${boxReadErrorMessage(lastError)}`) + }, + }), + ).rejects.toThrow('gave up after: transient') + expect(calls).toBe(1) + }) + + it('spends no read at all when the signal is already aborted', async () => { + let calls = 0 + const controller = new AbortController() + controller.abort() + const result = await readBoxPathWithRetry( + () => { + calls += 1 + return Promise.resolve('never reached') + }, + 'a.md', + { attempts: 3, delayMs: 0, signal: controller.signal }, + ) + expect(calls).toBe(0) + expect(result.succeeded).toBe(false) + if (!result.succeeded) + expect(boxReadErrorMessage(result.error)).toContain('aborted before reading') + }) + + it('runs beforeAttempt exactly once per attempt, including the aborted one', async () => { + const controller = new AbortController() + const seen: (string | undefined)[] = [] + await readBoxPathWithRetry( + () => { + controller.abort() + return Promise.reject(new Error('transient')) + }, + 'a.md', + { + attempts: 4, + delayMs: 0, + signal: controller.signal, + beforeAttempt: (lastError) => { + seen.push(boxReadErrorMessage(lastError)) + }, + }, + ) + // One call before the read that failed, one before the attempt the abort cancels. + expect(seen).toEqual([undefined, 'transient']) + }) + + it('treats a sub-1 attempt count as a single attempt rather than skipping the read', async () => { + let calls = 0 + const read = () => { + calls += 1 + return Promise.resolve('once') + } + const result = await readBoxPathWithRetry(read, 'a.md', { attempts: 0, delayMs: 0 }) + expect(result).toEqual({ succeeded: true, text: 'once' }) + expect(calls).toBe(1) + }) + + it('passes undefined through boxReadErrorMessage so "no failure yet" stays distinguishable', () => { + expect(boxReadErrorMessage(undefined)).toBeUndefined() + expect(boxReadErrorMessage(new Error('boom'))).toBe('boom') + expect(boxReadErrorMessage('plain string')).toBe('plain string') + }) +}) diff --git a/src/runtime/box-read-retry.ts b/src/runtime/box-read-retry.ts new file mode 100644 index 00000000..30bab3d0 --- /dev/null +++ b/src/runtime/box-read-retry.ts @@ -0,0 +1,70 @@ +/** + * One bounded-retry read over a sandbox box's filesystem, shared by every caller that reads a + * path the agent may have just written. + * + * The box data plane can transiently 404 a file that exists: the write is not yet flushed, or an + * edge read lands on a stale view. A first-attempt failure is therefore not evidence of absence, + * and a caller that treats it as such records a lie — "the agent produced nothing" for a + * deliverable, or "the agent deleted this surface" for a mounted profile file. + * + * Retries wait `delayMs × attempt` (linear), so the total wait for n attempts is + * `delayMs × n(n-1)/2`. Set `delayMs` to 0 to retry without waiting (tests). + */ + +import { sleep } from './util' + +/** Text read from the box, or the last error after every attempt failed. */ +export type BoxReadAttemptResult = + | { succeeded: true; text: string } + | { succeeded: false; error: unknown } + +export interface BoxReadRetryOptions { + /** Total read attempts, including the first. Values below 1 are treated as 1. */ + attempts: number + /** Linear backoff base in ms; the i-th retry waits `delayMs × i`. 0 disables the wait. */ + delayMs: number + /** Stops the retries once aborted: the pending wait ends and no further attempt is spent on an + * abandoned run. The last error is still returned, so a caller that must raise its own abort + * error keeps that authority through `beforeAttempt`. */ + signal?: AbortSignal + /** Runs before every attempt, carrying the previous attempt's error (`undefined` on the first). + * Throw from here to abandon the read — the way an aborting caller cancels with an error only + * it can construct. */ + beforeAttempt?: (lastError: unknown) => void +} + +/** The diagnostic for a failed attempt. `undefined` in, `undefined` out — so a caller can pass the + * "no attempt has failed yet" state through without inventing a message for it. */ +export function boxReadErrorMessage(error: unknown): string | undefined { + if (error === undefined) return undefined + return error instanceof Error ? error.message : String(error) +} + +export async function readBoxPathWithRetry( + read: (path: string) => Promise, + path: string, + options: BoxReadRetryOptions, +): Promise { + const attempts = Math.max(1, options.attempts) + let lastError: unknown + for (let attempt = 1; attempt <= attempts; attempt += 1) { + options.beforeAttempt?.(lastError) + // An abandoned run earns no further reads. The guard sits AFTER `beforeAttempt` so a caller + // that raises its own abort error keeps that authority, and BEFORE the read so an already + // cancelled run spends nothing. + if (options.signal?.aborted) { + lastError ??= new Error( + `readBoxPathWithRetry: aborted before reading ${JSON.stringify(path)}`, + ) + break + } + try { + return { succeeded: true, text: await read(path) } + } catch (err) { + lastError = err + if (attempt < attempts && options.delayMs > 0) + await sleep(options.delayMs * attempt, options.signal) + } + } + return { succeeded: false, error: lastError } +} diff --git a/src/runtime/index.ts b/src/runtime/index.ts index 6732de4c..7204a078 100644 --- a/src/runtime/index.ts +++ b/src/runtime/index.ts @@ -986,6 +986,18 @@ export { type SurfaceWorkerOut, superviseSurface, } from './supervise-surface' +export { + type BoxSurfaceReaderOptions, + boxSurfaceReader, + fsSurfaceReader, + type HarvestSurfaceDiffsOptions, + harvestSurfaceDiffs, + type SurfaceDiff, + type SurfaceReadBox, + type SurfaceReader, + type SurfaceReadOutcome, + type WatchedSurface, +} from './surface-diff' export type { SandboxControlClient } from './tangle-sandbox-exact-process-provider' // Profile-owned supervisor configuration. The arbitrary model callback and raw driver constructor // live only under `/testing`; production model execution enters through `supervise(AgentProfile)`. diff --git a/src/runtime/sandbox-backend.ts b/src/runtime/sandbox-backend.ts index 48ae47f7..19ea023e 100644 --- a/src/runtime/sandbox-backend.ts +++ b/src/runtime/sandbox-backend.ts @@ -30,6 +30,7 @@ const harnessBackends = [ 'opencode', 'kimi-code', 'pi', + 'prime', 'hermes', 'openclaw', 'amp', diff --git a/src/runtime/sandbox-run.ts b/src/runtime/sandbox-run.ts index a3d8db70..4212df6a 100644 --- a/src/runtime/sandbox-run.ts +++ b/src/runtime/sandbox-run.ts @@ -34,11 +34,12 @@ import type { PromptOptions, SandboxEvent, SandboxInstance } from '@tangle-network/sandbox' import type { RuntimeHooks, RuntimeHookTarget } from '../runtime-hooks' import { notifyRuntimeHookEvent } from '../runtime-hooks' +import { boxReadErrorMessage, readBoxPathWithRetry } from './box-read-retry' import { probeSandboxCapabilities } from './sandbox-capabilities' import { notifySandboxEventObserver } from './sandbox-events' import { createSandboxLineage, type SandboxLineageHandle } from './sandbox-lineage' import type { AgentRunSpec, SandboxClient } from './types' -import { isAbortError, randomSuffix, sleep } from './util' +import { isAbortError, randomSuffix } from './util' /** * How a typed deliverable `Out` is materialized from a finished turn. @@ -269,26 +270,21 @@ export async function openSandboxRun( return { out: deliverable.fromEvents(collected), events: collected } } if (options.signal.aborted) throw new SandboxRunAbortError(collected) - let raw = '' - let readError: string | undefined // The data plane can transiently 404 a just-written artifact (write not yet // flushed, or an edge-read blip) — retry a few times with backoff before // declaring the deliverable empty, so a transient read failure is not recorded // as "the agent produced nothing". - const readAttempts = 4 - const readDelayMs = options.readRetryDelayMs ?? 1000 - for (let attempt = 0; attempt < readAttempts; attempt += 1) { - if (options.signal.aborted) throw new SandboxRunAbortError(collected, readError) - try { - raw = await box.fs.read(deliverable.path) - readError = undefined - break - } catch (err) { - readError = err instanceof Error ? err.message : String(err) - if (attempt < readAttempts - 1 && readDelayMs > 0) - await sleep(readDelayMs * (attempt + 1), options.signal) - } - } + const attempted = await readBoxPathWithRetry(box.fs.read.bind(box.fs), deliverable.path, { + attempts: 4, + delayMs: options.readRetryDelayMs ?? 1000, + signal: options.signal, + beforeAttempt: (lastError) => { + if (options.signal.aborted) + throw new SandboxRunAbortError(collected, boxReadErrorMessage(lastError)) + }, + }) + const raw = attempted.succeeded ? attempted.text : '' + const readError = attempted.succeeded ? undefined : boxReadErrorMessage(attempted.error) return { out: deliverable.fromArtifact(raw, collected), events: collected, diff --git a/src/runtime/surface-diff.test.ts b/src/runtime/surface-diff.test.ts new file mode 100644 index 00000000..81462a93 --- /dev/null +++ b/src/runtime/surface-diff.test.ts @@ -0,0 +1,395 @@ +import { createHash } from 'node:crypto' +import { mkdtempSync, rmSync, symlinkSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterAll, describe, expect, it } from 'vitest' +import { + boxSurfaceReader, + fsSurfaceReader, + harvestSurfaceDiffs, + type SurfaceReader, +} from './surface-diff' +import type { MountManifestEntry } from './types' + +const sha = (s: string) => createHash('sha256').update(s).digest('hex') + +const mount = (path: string, content: string, source = 'test'): MountManifestEntry => ({ + path, + sha256: sha(content), + bytes: Buffer.byteLength(content), + source, +}) + +const readerOf = (files: Record): SurfaceReader => { + return (path) => { + const content = files[path] + if (content === undefined) + return Promise.resolve({ succeeded: false, missing: true, error: 'ENOENT' }) + return Promise.resolve({ succeeded: true, value: new TextEncoder().encode(content) }) + } +} + +describe('harvestSurfaceDiffs', () => { + it('reports nothing when every surface settles with its mounted bytes', async () => { + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('CLAUDE.md', 'instructions'), mount('memory/notes.md', 'notes')], + read: readerOf({ 'CLAUDE.md': 'instructions', 'memory/notes.md': 'notes' }), + }) + expect(diffs).toEqual([]) + }) + + it('reports a modified surface with the settled hash and size, preserving record order', async () => { + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('a.md', 'aaa'), mount('b.md', 'bbb', 'corpus:42'), mount('c.md', 'ccc')], + read: readerOf({ 'a.md': 'aaa', 'b.md': 'bbb-edited-by-agent', 'c.md': 'ccc!' }), + }) + expect(diffs.map((d) => d.path)).toEqual(['b.md', 'c.md']) + expect(diffs[0]).toEqual({ + path: 'b.md', + status: 'modified', + mountedSha256: sha('bbb'), + source: 'corpus:42', + settledSha256: sha('bbb-edited-by-agent'), + settledBytes: Buffer.byteLength('bbb-edited-by-agent'), + }) + }) + + it('reports a missing surface as removed and a failed read as unreadable with its error', async () => { + const read: SurfaceReader = (path) => { + if (path === 'gone.md') + return Promise.resolve({ succeeded: false, missing: true, error: 'ENOENT' }) + return Promise.resolve({ succeeded: false, missing: false, error: 'EACCES: denied' }) + } + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('gone.md', 'x'), mount('locked.md', 'y')], + read, + }) + expect(diffs).toEqual([ + { path: 'gone.md', status: 'removed', mountedSha256: sha('x'), source: 'test' }, + { + path: 'locked.md', + status: 'unreadable', + mountedSha256: sha('y'), + source: 'test', + error: 'EACCES: denied', + }, + ]) + }) + + it('collapses duplicate paths to the last mount — the bytes the agent actually saw', async () => { + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('a.md', 'first'), mount('a.md', 'second')], + read: readerOf({ 'a.md': 'second' }), + }) + expect(diffs).toEqual([]) + }) + + it('reports a watched never-mounted path that now exists as created, and stays silent while absent', async () => { + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('CLAUDE.md', 'base')], + read: readerOf({ 'CLAUDE.md': 'base', 'memory/new-lesson.md': 'learned it' }), + watch: [ + { path: 'memory/new-lesson.md', source: 'harness-state' }, + { path: 'memory/still-absent.md' }, + ], + }) + expect(diffs).toEqual([ + { + path: 'memory/new-lesson.md', + status: 'created', + source: 'harness-state', + settledSha256: sha('learned it'), + settledBytes: Buffer.byteLength('learned it'), + }, + ]) + }) + + it('compares a watched path that was ALSO mounted against its mount, not as created', async () => { + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('a.md', 'v1')], + read: readerOf({ 'a.md': 'v2' }), + watch: [{ path: 'a.md' }], + }) + expect(diffs.map((d) => d.status)).toEqual(['modified']) + }) + + it("treats a './'-prefixed mount and a bare watched path as one surface, and dedupes duplicate watches", async () => { + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('./AGENTS.md', 'given')], + read: readerOf({ './AGENTS.md': 'given', 'memory/x.md': 'note' }), + watch: [ + { path: 'AGENTS.md' }, + { path: 'memory/x.md' }, + { path: 'memory/x.md', source: 'tree' }, + ], + }) + expect(diffs).toEqual([ + { + path: 'memory/x.md', + status: 'created', + source: 'tree', + settledSha256: sha('note'), + settledBytes: Buffer.byteLength('note'), + }, + ]) + }) + + it('contains a reader that throws: the bad path reports unreadable, every other diff survives', async () => { + const read: SurfaceReader = (path) => { + if (path === 'boom.md') throw new Error('reader contract violation') + return Promise.resolve({ succeeded: true, value: new TextEncoder().encode('v2') }) + } + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('boom.md', 'x'), mount('a.md', 'v1')], + read, + }) + expect(diffs.map((d) => [d.path, d.status])).toEqual([ + ['boom.md', 'unreadable'], + ['a.md', 'modified'], + ]) + expect(diffs[0]?.error).toBe('reader contract violation') + }) + + it('treats an uppercase manifest hash as equal to the settled lowercase hash', async () => { + const entry = { ...mount('a.md', 'same'), sha256: sha('same').toUpperCase() } + const diffs = await harvestSurfaceDiffs({ + mounts: [entry], + read: readerOf({ 'a.md': 'same' }), + }) + expect(diffs).toEqual([]) + }) +}) + +describe('boxSurfaceReader', () => { + it('reads through box.fs.read, maps the SDK NotFoundError to missing, and reports other errors', async () => { + const notFound = new Error('no such file') + notFound.name = 'NotFoundError' + const box = { + fs: { + read: (path: string) => { + if (path === 'AGENTS.md') return Promise.resolve('edited') + if (path === 'gone.md') return Promise.reject(notFound) + return Promise.reject(new Error('transport down')) + }, + }, + } + const read = boxSurfaceReader(box, { retryDelayMs: 0 }) + const hit = await read('AGENTS.md') + expect(hit.succeeded).toBe(true) + if (hit.succeeded) expect(new TextDecoder().decode(hit.value)).toBe('edited') + expect(await read('gone.md')).toEqual({ + succeeded: false, + missing: true, + error: 'no such file', + }) + expect(await read('other.md')).toEqual({ + succeeded: false, + missing: false, + error: 'transport down', + }) + }) + + it('retries a transient first-attempt 404 instead of reporting a fresh write as missing', async () => { + const blip = new Error('not flushed yet') + blip.name = 'NotFoundError' + let calls = 0 + const box = { + fs: { + read: () => { + calls += 1 + return calls < 3 ? Promise.reject(blip) : Promise.resolve('finally visible') + }, + }, + } + const outcome = await boxSurfaceReader(box, { retryDelayMs: 0 })('memory/new.md') + expect(calls).toBe(3) + expect(outcome.succeeded).toBe(true) + }) + + it('does not report a still-flushing mounted file as removed when the first read 404s', async () => { + const blip = new Error('not flushed yet') + blip.name = 'NotFoundError' + let calls = 0 + const box = { + fs: { + read: () => { + calls += 1 + return calls === 1 ? Promise.reject(blip) : Promise.resolve('edited by the agent') + }, + }, + } + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('CLAUDE.md', 'original')], + read: boxSurfaceReader(box, { retryDelayMs: 0 }), + }) + expect(diffs.map((d) => d.status)).not.toContain('removed') + expect(diffs).toEqual([ + { + path: 'CLAUDE.md', + status: 'modified', + mountedSha256: sha('original'), + source: 'test', + settledSha256: sha('edited by the agent'), + settledBytes: Buffer.byteLength('edited by the agent'), + }, + ]) + }) + + it("treats the SDK's default 'Resource' resourceType as a missing FILE, so a deletion reports removed", async () => { + // The SDK's HTTP mapper builds `new NotFoundError(data.resourceType || 'Resource', ...)`, so a + // file 404 carries 'Resource' whenever the server does not name the type. + const deleted = Object.assign(new Error('not found'), { + name: 'NotFoundError', + resourceType: 'Resource', + resourceId: 'unknown', + }) + const box = { fs: { read: () => Promise.reject(deleted) } } + expect(await boxSurfaceReader(box, { attempts: 1 })('CLAUDE.md')).toEqual({ + succeeded: false, + missing: true, + error: 'not found', + }) + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('CLAUDE.md', 'original')], + read: boxSurfaceReader(box, { attempts: 1 }), + }) + expect(diffs.map((d) => d.status)).toEqual(['removed']) + }) + + it('stops retrying once the signal aborts instead of spending the remaining attempts', async () => { + const controller = new AbortController() + let calls = 0 + const box = { + fs: { + read: () => { + calls += 1 + controller.abort() + return Promise.reject(new Error('transport down')) + }, + }, + } + const outcome = await boxSurfaceReader(box, { + attempts: 5, + retryDelayMs: 0, + signal: controller.signal, + })('CLAUDE.md') + expect(calls).toBe(1) + expect(outcome).toEqual({ succeeded: false, missing: false, error: 'transport down' }) + }) + + it('reports a box-level NotFoundError (resourceType names the sandbox) as unreadable, never missing', async () => { + const boxGone = Object.assign(new Error('sandbox sb-1 not found'), { + name: 'NotFoundError', + resourceType: 'Sandbox', + resourceId: 'sb-1', + }) + const box = { fs: { read: () => Promise.reject(boxGone) } } + expect(await boxSurfaceReader(box, { attempts: 1 })('CLAUDE.md')).toEqual({ + succeeded: false, + missing: false, + error: 'sandbox sb-1 not found', + }) + }) + + it('refuses to hash content the text wire lossy-decoded instead of reporting a false modification', async () => { + const box = { fs: { read: () => Promise.resolve('binary�garbage') } } + const outcome = await boxSurfaceReader(box, { attempts: 1 })('memory/store.db') + expect(outcome).toMatchObject({ succeeded: false, missing: false }) + if (!outcome.succeeded) expect(outcome.error).toContain('not valid UTF-8') + }) + + it('composes with the harvest over a box double: modified mount + created watch', async () => { + const box = { + fs: { + read: (path: string) => { + if (path === 'CLAUDE.md') return Promise.resolve('rewritten') + if (path === 'memory/lesson.md') return Promise.resolve('new note') + const err = new Error('missing') + err.name = 'NotFoundError' + return Promise.reject(err) + }, + }, + } + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('CLAUDE.md', 'original')], + read: boxSurfaceReader(box), + watch: [{ path: 'memory/lesson.md' }, { path: 'memory/other.md' }], + }) + expect(diffs.map((d) => [d.path, d.status])).toEqual([ + ['CLAUDE.md', 'modified'], + ['memory/lesson.md', 'created'], + ]) + }) +}) + +describe('fsSurfaceReader', () => { + const root = mkdtempSync(join(tmpdir(), 'surface-diff-')) + afterAll(() => rmSync(root, { recursive: true, force: true })) + + it('reads relative paths against the root and maps absence to missing', async () => { + writeFileSync(join(root, 'notes.md'), 'hello') + const read = fsSurfaceReader(root) + const hit = await read('notes.md') + expect(hit.succeeded).toBe(true) + if (hit.succeeded) expect(new TextDecoder().decode(hit.value)).toBe('hello') + const miss = await read('absent.md') + expect(miss).toMatchObject({ succeeded: false, missing: true }) + }) + + it('contains paths inside the root: ../ escapes and outside absolute paths fail without reading', async () => { + const read = fsSurfaceReader(root) + const escaped = await read('../outside.md') + expect(escaped).toMatchObject({ succeeded: false, missing: false }) + if (!escaped.succeeded) expect(escaped.error).toContain('outside the reader root') + const absolute = await read('/etc/hostname') + expect(absolute).toMatchObject({ succeeded: false, missing: false }) + const insideAbsolute = await read(join(root, 'notes.md')) + expect(insideAbsolute.succeeded).toBe(true) + }) + + it('refuses a symlink planted inside the root that points at a host file', async () => { + const outside = mkdtempSync(join(tmpdir(), 'surface-diff-outside-')) + const secret = join(outside, 'host-secret.txt') + writeFileSync(secret, 'host bytes') + symlinkSync(secret, join(root, 'looks-contained.md')) + const read = fsSurfaceReader(root) + const outcome = await read('looks-contained.md') + expect(outcome).toMatchObject({ succeeded: false, missing: false }) + if (!outcome.succeeded) expect(outcome.error).toContain('outside the reader root') + rmSync(outside, { recursive: true, force: true }) + }) + + it('still reads a file reached through a symlinked root', async () => { + const realRoot = mkdtempSync(join(tmpdir(), 'surface-diff-real-')) + const linkedRoot = join(mkdtempSync(join(tmpdir(), 'surface-diff-link-')), 'root-link') + writeFileSync(join(realRoot, 'inside.md'), 'contained') + symlinkSync(realRoot, linkedRoot) + const outcome = await fsSurfaceReader(linkedRoot)('inside.md') + expect(outcome.succeeded).toBe(true) + if (outcome.succeeded) expect(new TextDecoder().decode(outcome.value)).toBe('contained') + rmSync(realRoot, { recursive: true, force: true }) + }) + + it('reports a vanished worktree root as unreadable, never as every mount removed', async () => { + const doomed = mkdtempSync(join(tmpdir(), 'surface-diff-doomed-')) + writeFileSync(join(doomed, 'a.md'), 'v1') + const read = fsSurfaceReader(doomed) + rmSync(doomed, { recursive: true, force: true }) + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('a.md', 'v1'), mount('b.md', 'v1')], + read, + }) + expect(diffs.map((d) => d.status)).toEqual(['unreadable', 'unreadable']) + expect(diffs[0]?.error).toContain('reader root') + }) + + it('composes with the harvest over a real worktree edit', async () => { + writeFileSync(join(root, 'skill.md'), 'v1') + const mounts = [mount('skill.md', 'v1')] + writeFileSync(join(root, 'skill.md'), 'v2') + const diffs = await harvestSurfaceDiffs({ mounts, read: fsSurfaceReader(root) }) + expect(diffs).toMatchObject([ + { path: 'skill.md', status: 'modified', settledSha256: sha('v2') }, + ]) + }) +}) diff --git a/src/runtime/surface-diff.ts b/src/runtime/surface-diff.ts new file mode 100644 index 00000000..49a57185 --- /dev/null +++ b/src/runtime/surface-diff.ts @@ -0,0 +1,318 @@ +/** + * + * Settle-time surface-diff harvest — the read-back dual of the mount manifest. + * + * `RunProvenance.mounts` records what the caller placed INTO a run's workspace before the agent saw + * it (instructions, skills, memory files — the profile surfaces the substrate materialized). This + * module answers the reverse question at settle: **which of those mounted surfaces did the agent + * itself change while working?** Harnesses mutate their own profile-adjacent surfaces as a matter of + * course — a memory file appended to, an instructions file edited, a skill rewritten — and that + * self-mutation is improvement-relevant evidence: an OBSERVED, session-scoped edit that may later be + * lifted into the measured proposal pipeline (`proposeAgentImprovement`), never auto-promoted. + * + * The harvest is harness-agnostic by construction: it compares content hashes against the mount + * manifest, so it needs no harness event format, no refinement protocol, and no knowledge of WHY a + * surface changed. A harness that additionally reports structured self-edit events can enrich this + * signal; nothing requires it to. + * + * The kernel never reads workspace contents itself — the caller supplies the read seam (a box + * `fs.read`, a worktree read, a test double), mirroring how `recordMount` keeps mount hashing with + * the byte owner. Reads return typed outcomes; a failed read is reported as an `unreadable` diff + * carrying the error, never silently dropped. + * + * @experimental + */ + +import { createHash } from 'node:crypto' +import { readFile, realpath } from 'node:fs/promises' +import { isAbsolute, resolve, sep } from 'node:path' +import { readBoxPathWithRetry } from './box-read-retry' +import type { MountManifestEntry } from './types' + +/** Outcome of reading one surface back at settle. `missing: true` means the path no longer exists + * (a deletion — a valid, reportable outcome); any other failure carries its diagnostic. */ +export type SurfaceReadOutcome = + | { succeeded: true; value: Uint8Array } + | { succeeded: false; missing: boolean; error: string } + +/** The read seam: fetch the current bytes at a mounted path. Implemented by a sandbox box's + * `fs.read`, a local worktree read ({@link fsSurfaceReader}), or a test double. */ +export type SurfaceReader = (path: string) => Promise + +/** + * One watched surface whose settled state differs from what was mounted (or from absence). + * + * - `modified` — the surface exists with different bytes (`settledSha256`/`settledBytes` present). + * - `removed` — the surface no longer exists at its mounted path. + * - `created` — a watched path that was never mounted now exists (`settledSha256`/`settledBytes` + * present, no `mountedSha256`) — the shape a harness's new memory/skill file takes. + * - `unreadable` — the read seam failed for a reason other than absence; `error` carries the + * diagnostic. Reported rather than dropped so a permissions or transport failure cannot + * masquerade as "nothing changed". + */ +export interface SurfaceDiff { + /** The mounted/watched path, exactly as recorded. */ + path: string + status: 'modified' | 'removed' | 'created' | 'unreadable' + /** Hex SHA-256 of the bytes that were mounted (from the manifest). Absent for `created`. */ + mountedSha256?: string + /** Free-form origin: the manifest entry's `source`, or the watch entry's `source`. */ + source: string + /** Hex SHA-256 of the settled bytes. Present for `modified` and `created`. */ + settledSha256?: string + /** Size of the settled bytes. Present for `modified` and `created`. */ + settledBytes?: number + /** The read seam's diagnostic. Present only for `unreadable`. */ + error?: string +} + +/** A path to check at settle that was NOT necessarily mounted — where a harness is known to write + * self-authored surfaces (a memory dir's files, a refinement log). A watched path that was also + * mounted compares against its mount; one that wasn't reports `created` if it now exists. + * `created` is an inference from the mount manifest, not a proof of authorship: a file the box + * IMAGE shipped at a never-mounted path also reports `created`. Watch paths known absent at run + * start (or enumerate the tree at start AND settle and watch the difference) to make the label + * mean what it says. */ +export interface WatchedSurface { + path: string + /** Origin label carried onto the diff (default `'watched'`). */ + source?: string +} + +/** Inputs to {@link harvestSurfaceDiffs}: the run's mount manifest, the read seam, and optional + * watch paths for surfaces the agent may have created. */ +export interface HarvestSurfaceDiffsOptions { + /** The run's mount manifest (`RunProvenance.mounts`). Entries sharing a path are collapsed to the + * LAST entry — the bytes the agent actually saw at start. */ + mounts: readonly MountManifestEntry[] + /** How to read a mounted path's current bytes. */ + read: SurfaceReader + /** Additional paths to check that may not have been mounted (see {@link WatchedSurface}). The + * caller enumerates them (it knows the harness's state layout — e.g. via the box's file tree); + * the harvest stays layout-agnostic. */ + watch?: readonly WatchedSurface[] +} + +const sha256Hex = (bytes: Uint8Array): string => createHash('sha256').update(bytes).digest('hex') + +/** Contain a reader that violates its typed-outcome contract by THROWING: the module's law is that + * a failed read becomes an `unreadable` diff, so one bad path must not reject the whole harvest + * and silently drop every other diff. */ +const readOutcome = async (read: SurfaceReader, path: string): Promise => { + try { + return await read(path) + } catch (err) { + return { + succeeded: false, + missing: false, + error: err instanceof Error ? err.message : String(err), + } + } +} + +/** The collision/dedup key for a path: a leading `./` stripped, so `./AGENTS.md` (a mount + * recorder's form) and `AGENTS.md` (a file-tree enumeration's form) are one surface. Absolute vs + * relative cannot be reconciled here — keep those forms consistent between mounts and watches. */ +const pathKey = (p: string): string => p.replace(/^\.\//, '') + +/** + * Re-read every mounted (and watched) surface and report the ones whose settled state differs from + * the manifest — modified, removed, or created. Unchanged surfaces and still-absent watched paths + * produce no entry; reads run concurrently; output preserves record order, mounts before + * watch-only paths. Mounts and watches sharing a path key are each collapsed to the LAST entry, + * and a watched path that was also mounted compares against its mount (never reports `created`). + * + * The harvest takes no `AbortSignal`: it is pure fan-out over the read seam and waits on nothing + * itself, so every cancellable moment belongs to the reader. Pass a signal to the reader instead + * ({@link BoxSurfaceReaderOptions.signal}, or close over one in a custom {@link SurfaceReader}) — + * that cuts the backoff waits, and the harvest still returns the diffs it did establish rather + * than discarding settle-time evidence on a late cancellation. + */ +export async function harvestSurfaceDiffs( + options: HarvestSurfaceDiffsOptions, +): Promise { + const byPath = new Map() + for (const entry of options.mounts) byPath.set(pathKey(entry.path), entry) + const watchByPath = new Map() + for (const watched of options.watch ?? []) { + if (byPath.has(pathKey(watched.path))) continue + watchByPath.set(pathKey(watched.path), watched) + } + const mountDiffs = await Promise.all( + [...byPath.values()].map(async (entry): Promise => { + const outcome = await readOutcome(options.read, entry.path) + if (!outcome.succeeded) { + return outcome.missing + ? { + path: entry.path, + status: 'removed', + mountedSha256: entry.sha256, + source: entry.source, + } + : { + path: entry.path, + status: 'unreadable', + mountedSha256: entry.sha256, + source: entry.source, + error: outcome.error, + } + } + const settledSha256 = sha256Hex(outcome.value) + if (settledSha256 === entry.sha256.toLowerCase()) return undefined + return { + path: entry.path, + status: 'modified', + mountedSha256: entry.sha256, + source: entry.source, + settledSha256, + settledBytes: outcome.value.byteLength, + } + }), + ) + const watchDiffs = await Promise.all( + [...watchByPath.values()].map(async (watched): Promise => { + const outcome = await readOutcome(options.read, watched.path) + if (!outcome.succeeded) { + // A still-absent watched path is the expected no-op; a failing read is not. + if (outcome.missing) return undefined + return { + path: watched.path, + status: 'unreadable', + source: watched.source ?? 'watched', + error: outcome.error, + } + } + return { + path: watched.path, + status: 'created', + source: watched.source ?? 'watched', + settledSha256: sha256Hex(outcome.value), + settledBytes: outcome.value.byteLength, + } + }), + ) + return [...mountDiffs, ...watchDiffs].filter((d): d is SurfaceDiff => d !== undefined) +} + +/** The minimal box surface the box-backed reader needs — structurally typed so the real + * `@tangle-network/sandbox` box and a test double both satisfy it, no SDK import. */ +export interface SurfaceReadBox { + fs: { read(path: string): Promise } +} + +/** Retry and cancellation controls for {@link boxSurfaceReader}. */ +export interface BoxSurfaceReaderOptions { + /** Read attempts per path before settling on a failed outcome. The data plane can transiently + * 404 a just-written file (the same blip `openSandboxRun`'s deliverable read retries for), and a + * first-attempt 404 taken at face value turns a fresh self-edit into a false `removed`/dropped + * `created`. Default 3. */ + attempts?: number + /** Linear backoff base between attempts (delay = base × attempt). Default 250. */ + retryDelayMs?: number + /** Cuts the retry waits short when the run is abandoned. The reader still returns a typed + * outcome — the harvest reports what it managed to read rather than rejecting. */ + signal?: AbortSignal +} + +/** + * A {@link SurfaceReader} over a sandbox box's filesystem — the same `box.fs.read` seam + * `openSandboxRun` reads deliverables through, with the same transient-404 posture (bounded + * retry). The box wire returns UTF-8 TEXT (the SDK's binary path is `download()`), which profile + * surfaces are; hashes are computed over the UTF-8 encoding, and content the wire had to + * lossy-decode (a U+FFFD replacement character) is reported `unreadable` rather than hashed as + * mojibake. The SDK's not-found error is detected structurally (`err.name === 'NotFoundError'`) + * and maps to `missing: true` — unless its `resourceType` names something other than a file/path + * (the BOX or session being gone), which is a transport failure, not an absent surface. + */ +export function boxSurfaceReader( + box: SurfaceReadBox, + options: BoxSurfaceReaderOptions = {}, +): SurfaceReader { + return async (path) => { + const result = await readBoxPathWithRetry(box.fs.read.bind(box.fs), path, { + attempts: options.attempts ?? 3, + delayMs: options.retryDelayMs ?? 250, + signal: options.signal, + }) + if (result.succeeded) { + // A lossy decode is deterministic — retrying cannot recover the bytes, so this is a final + // outcome rather than another attempt. + if (result.text.includes('�')) { + return { + succeeded: false, + missing: false, + error: `boxSurfaceReader: content at ${JSON.stringify(path)} is not valid UTF-8 text (the box text wire lossy-decoded it); binary surfaces need a byte-faithful reader`, + } + } + return { succeeded: true, value: new TextEncoder().encode(result.text) } + } + const err = result.error + const notFound = err instanceof Error && err.name === 'NotFoundError' + const resourceType = + err && typeof err === 'object' && 'resourceType' in err + ? String((err as { resourceType: unknown }).resourceType) + : undefined + // A NotFoundError naming the BOX or the session means the TRANSPORT is gone — calling that + // `missing` would read as "the agent deleted every surface". Anything else is the file: the + // SDK's HTTP mapper defaults `resourceType` to `'Resource'` when the server does not name one, + // so matching only file-ish names would classify an ordinary deletion as `unreadable` and the + // harvest would never report `removed`. + const transportGone = resourceType !== undefined && /sandbox|session/i.test(resourceType) + return { + succeeded: false, + missing: notFound && !transportGone, + error: err instanceof Error ? err.message : String(err), + } + } +} + +/** + * A {@link SurfaceReader} over the local filesystem, for worktree/local workers. Every path — + * relative or absolute — must resolve INSIDE `root`: a path that escapes it (`../`, an absolute + * path elsewhere) fails as a contained non-missing outcome rather than reading outside the + * worktree, so a persisted or mistyped manifest path cannot turn the harvest into an + * existence/hash oracle over the host filesystem. Containment is checked twice — once on the + * lexical path, then again on the symlink-resolved path, because `readFile` follows a link and a + * link planted inside the root would otherwise read host bytes through a contained-looking name. + * Absence maps to `missing: true`; every other failure carries the error message. + */ +export function fsSurfaceReader(root: string): SurfaceReader { + const lexicalRoot = resolve(root) + let resolvedRoot: string | undefined + const escapes = (candidate: string, boundary: string): boolean => + candidate !== boundary && !candidate.startsWith(boundary + sep) + const outside = (path: string, boundary: string): SurfaceReadOutcome => ({ + succeeded: false, + missing: false, + error: `fsSurfaceReader: path ${JSON.stringify(path)} resolves outside the reader root ${JSON.stringify(boundary)}`, + }) + return async (path) => { + const target = isAbsolute(path) ? resolve(path) : resolve(lexicalRoot, path) + if (escapes(target, lexicalRoot)) return outside(path, lexicalRoot) + try { + // The root itself may be reached through a link (a worktree, a temp dir); compare + // link-resolved against link-resolved so a legitimate mount is not read as an escape. + resolvedRoot ??= await realpath(lexicalRoot) + } catch (err) { + // The WORKSPACE is gone, not the surface. Calling this `missing` would report every mount as + // a deliberate agent deletion — the filesystem twin of a dead box answering for each path. + return { + succeeded: false, + missing: false, + error: `fsSurfaceReader: reader root ${JSON.stringify(lexicalRoot)} is unreadable (${err instanceof Error ? err.message : String(err)})`, + } + } + try { + const resolvedTarget = await realpath(target) + if (escapes(resolvedTarget, resolvedRoot)) return outside(path, resolvedRoot) + return { succeeded: true, value: new Uint8Array(await readFile(resolvedTarget)) } + } catch (err) { + const code = (err as NodeJS.ErrnoException).code + return { + succeeded: false, + missing: code === 'ENOENT' || code === 'ENOTDIR', + error: err instanceof Error ? err.message : String(err), + } + } + } +} diff --git a/tests/runtime/sandbox-run.test.ts b/tests/runtime/sandbox-run.test.ts index 0f7d22d6..2fddeefd 100644 --- a/tests/runtime/sandbox-run.test.ts +++ b/tests/runtime/sandbox-run.test.ts @@ -574,6 +574,27 @@ describe('openSandboxRun — abort-aware artifact read preserves the partial tra // One read attempted, then the post-backoff guard short-circuits the retries. expect(reads).toBe(1) }) + + it('returns the turn with its readError when every read attempt fails without an abort', async () => { + let reads = 0 + const { client } = createFakeClient({ + events: [{ type: 'result', data: { ok: true } } as SandboxEvent], + fsRead: () => { + reads += 1 + throw new Error('Resource not found: never flushed') + }, + }) + const run = await openSandboxRun( + client, + { agentRun: spec(), signal: new AbortController().signal, readRetryDelayMs: 0 }, + artifactDeliverable('solution.patch'), + ) + const turn = await run.start('write the patch') + // The retry budget is spent, then the failure is REPORTED rather than thrown, so a caller + // still distinguishes "produced nothing" from a read fault. + expect(reads).toBe(4) + expect(turn.readError).toMatch(/never flushed/) + }) }) describe('openSandboxRun — runtime hooks', () => {