From 09486c8f56678bb488f9868ff4db8bad9526119d Mon Sep 17 00:00:00 2001 From: drewstone Date: Tue, 11 Aug 2026 20:16:40 +0000 Subject: [PATCH 01/10] feat(kernel): settle-time surface-diff harvest + Prime Agent harness integration contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit harvestSurfaceDiffs is the read-back dual of the mount manifest: re-hash the mounted profile surfaces (instructions/skills/memory) at settle and report what the agent itself changed — the harness-agnostic observation behind any self-refinement surface (claude-code memory edits, opencode AGENTS.md, Prime /refine). Observed session-scoped edits record; reuse-scoped edits go through the existing measured proposal pipeline, never auto-promoted. docs/design/prime-agent-harness-integration.md is the decision record for hosting Prime Agent as a sandbox-materialized harness: the boundary (harness owns its inner loop; Runtime owns identity/budget/lifecycle; Eval owns verdicts; Intelligence owns promotion), the anti-reinvention map of what the integration reuses, what is deliberately not built (TopologyAuthority, HarnessExecutionReceipt, an optional-method backend zoo), the numbered substrate wish-list, and the first gated /refine-transfer experiment. harness-compat.md gains a pending prime-agent column marker (measure, never inherit pi's row). --- docs/README.md | 1 + docs/api/primitive-catalog.md | 9 +- docs/api/runtime.md | 142 ++++++++++++++ docs/canonical-api.md | 1 + .../design/prime-agent-harness-integration.md | 176 ++++++++++++++++++ docs/research/harness-compat.md | 13 ++ src/runtime/index.ts | 8 + src/runtime/surface-diff.test.ts | 115 ++++++++++++ src/runtime/surface-diff.ts | 142 ++++++++++++++ 9 files changed, 605 insertions(+), 2 deletions(-) create mode 100644 docs/design/prime-agent-harness-integration.md create mode 100644 src/runtime/surface-diff.test.ts create mode 100644 src/runtime/surface-diff.ts diff --git a/docs/README.md b/docs/README.md index 2c23f257..bc915ad0 100644 --- a/docs/README.md +++ b/docs/README.md @@ -47,6 +47,7 @@ These are internal working documents: design theses, research narrative, and roa | Doc | Role | Purpose | |---|---|---| +| [design/prime-agent-harness-integration.md](./design/prime-agent-harness-integration.md) | integration contract | Prime Agent as a sandbox-materialized harness: the boundary, the anti-reinvention map, the substrate wish-list, and the first gated experiment. | | [simplification-plan.md](./research/simplification-plan.md) | historical tracker | Earlier simplification analysis. The active execution and API convergence plan is [agent-managed-compute/roadmap.md](./agent-managed-compute/roadmap.md). | | [research/README.md](./research/README.md) | research index | Forward-looking design threads + decision log. Not the canonical spine. | | [archive/](./archive/) | retired notes | Superseded/niche docs kept for history (delivery manifest, conversation economics, artifact-lifecycle, go-live, results, benchmark-matrix consolidation). | diff --git a/docs/api/primitive-catalog.md b/docs/api/primitive-catalog.md index f5b979af..d26ad6a7 100644 --- a/docs/api/primitive-catalog.md +++ b/docs/api/primitive-catalog.md @@ -504,7 +504,7 @@ Import from `@tangle-network/agent-runtime/intelligence` — 166 exports. ### Execution kernel — recursive atom, supervision, executors, round-synchronous loop -Import from `@tangle-network/agent-runtime/kernel` — 709 exports. +Import from `@tangle-network/agent-runtime/kernel` — 715 exports. | Symbol | Kind | Summary | |---|---|---| @@ -593,9 +593,11 @@ Import from `@tangle-network/agent-runtime/kernel` — 709 exports. | `flatWidenGate` | function | The flat default `ScopeWidenGate` — never widens, keeping the R2 selector≠judge collision | | `formatPromptHandle` | function | The string form of a handle: `/v`. | | `freeSlots` | function | Free worker slots under a simultaneity cap: `cap - live`, floored at 0, or `null` when there is | +| `fsSurfaceReader` | function | A {@link SurfaceReader} over the local filesystem, for worktree/local workers. Relative mount | | `gateOnDeliverable` | function | Wrap an `Executor` so its settlement `valid` reflects the deliverable check, not the | | `gitWorkspace` | function | A `Workspace` over a git checkout: materialize an isolated worktree at `ref`, commit produced changes (conflict-aware), and read `head` — hooks disabled, identity pinned. | | `harvestCorpus` | function | Batch the firewalled `observe()` analyst over completed runs and accrete the trace-derived facts into the durable corpus — the production-traces→corpus write side of the flywheel. | +| `harvestSurfaceDiffs` | function | Re-read every mounted surface and report the ones whose settled state differs from the manifest. | | `inlineSandboxClient` | function | Adapt an `ExecutorFactory` into a `SandboxClient` for `runAgentRounds`. The factory is | | `inProcessSandboxClient` | function | Adapt a single `onPrompt(prompt, ctx)` callback into a `SandboxClient` for | | `isWaitOutcome` | function | Narrow a settlement's `out` to a wait outcome — a wait settles on the SAME cursor as workers, | @@ -954,6 +956,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 709 exports. | `SupervisorSpanOutcome` | interface | How the supervised run ended, as `finish()` records it on the root span. | | `SupervisorToolDescriptor` | interface | One product-owned tool. It reuses the canonical MCP descriptor fields while Runtime supplies | | `SupervisorToolInvocationContext` | interface | Trusted context for one product-tool invocation. The node identity remains the same detached, | +| `SurfaceDiff` | interface | One mounted surface whose settled state differs from what was mounted. | | `SurfaceWorkerConfig` | interface | How a worker runs the surface task (its router substrate + per-attempt bounds). | | `SurfaceWorkerOut` | interface | What a surface worker settles with — the surface verdict the driver + deliverable read. `resolved` is | | `ToolLoopCompaction` | interface | Self-compaction — bound the loop's OWN context window the way a fresh-respawn (dumb-Ralph) loop | @@ -1051,6 +1054,8 @@ Import from `@tangle-network/agent-runtime/kernel` — 709 exports. | `SupervisorNodeContextSeed` | type | Context known before `Agent.act`; Runtime adds the concrete node, profile, and task. | | `SupervisorProfile` | type | A supervisor is an exact canonical AgentProfile; no looser model/prompt shape exists. | | `SupervisorSpanAttributes` | type | OTLP span attribute values. Exported because `SupervisorSpanOptions.attributes` is public and | +| `SurfaceReader` | type | The read seam: fetch the current bytes at a mounted path. Implemented by a sandbox box's | +| `SurfaceReadOutcome` | type | Outcome of reading one surface back at settle. `missing: true` means the path no longer exists | | `ToolLoopCompactionOptions` | type | Public supervisor-facing compaction config: same knobs as the primitive, but `distill` is optional | | `ToolLoopMessageRecord` | type | Provider-neutral conversation record accepted by a tool-loop brain. | | `TrajectoryReportFn` | type | `trajectoryReport(...)` — the tree+cost reconstructor. Async (reads journal + optionally blobs). | @@ -1070,7 +1075,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 709 exports. | `WorktreeCheckRunner` | type | The single shell-command-in-worktree runner seam (replaces the per-executor copies). | | `WorktreePatchArtifact` | type | Terminal artifact of one worktree-CLI run — the canonical worktree-harness result (the captured | -**Undocumented supporting types** (add a TSDoc line at the declaration to earn a table row): `AcquireOptions`, `AgentEnvironment`, `AgentEnvironmentCapabilities`, `AgentEnvironmentEvent`, `AgentEnvironmentProvider`, `AgentEnvironmentQuery`, `AgentEnvironmentSummary`, `AgentGraph`, `AgenticOptions`, `AgenticRunResult`, `AgenticTask`, `AgenticTool`, `AgentSession`, `AgentSessionRef`, `AgentTurnResult`, `AllWorkersStalledOptions`, `AnalystRegistry`, `AnytimeReport`, `AnytimeStrategySummary`, `AnytimeTaskCurve`, `ArtifactHandle`, `AuditIntentInput`, `AuditIntentOptions`, `AuthoredHarness`, `AuthoredStrategy`, `AuthorStrategyOptions`, `BenchmarkConfig`, `BenchmarkLift`, `BenchmarkStrategySummary`, `BenchmarkTaskRow`, `BudgetPool`, `BusStats`, `ChampionPick`, `CheckpointRef`, `CheckpointRequest`, `CheckRunContext`, `CliWorktreeBridgeSeam`, `CoordinationMcpHandle`, `CopyOptions`, `CorpusReadbackOptions`, `CreateAgentEnvironmentInput`, `CreateTangleSandboxExactProcessProviderOptions`, `DefinedLeaderboard`, `DispatchReport`, `Driver`, `EvolutionArchiveNode`, `EvolutionAuthor`, `EvolutionBandInfo`, `EvolutionCandidate`, `EvolutionGeneration`, `EvolutionReport`, `ExecRequest`, `ExecResult`, `ExecutorResultMapping`, `ForkRequest`, `GitWorkspaceOptions`, `GraphResult`, `HarvestCorpusOptions`, `HarvestFailure`, `HarvestReport`, `Inbox`, `InProcessSandboxClientOptions`, `IntentAudit`, `Iteration`, `Leaderboard`, `LeaderboardOptions`, `LocalSandboxClientOptions`, `LoopDecisionPayload`, `LoopDispatchOptions`, `LoopEndedPayload`, `LoopIterationEndedPayload`, `LoopIterationStartedPayload`, `LoopPlanDescription`, `LoopResult`, `LoopSandboxPlacement`, `LoopStartedPayload`, `LoopTraceEmitter`, `LoopWinner`, `MaterializeLocalMcpOptions`, `McpEnvironmentOptions`, `McpToolDescriptor`, `NodeSnapshot`, `NoProgressForOptions`, `Observation`, `ObserveInput`, `ObserveOptions`, `OpenSandboxRunOptions`, `PairwiseOptions`, `PatchDeliverableOptions`, `PlacementInfo`, `PlateauOptions`, `ProgressTrackerOptions`, `PromotionGateOptions`, `PromotionVerdict`, `PublishOptions`, `ReproductionCheck`, `ResolveSandboxClientOptions`, `ResourceRequest`, `RollingDispatchOptions`, `RunAgenticOptions`, `RunAgentRoundsOptions`, `RunGraphOptions`, `SandboxRun`, `ShotSpec`, `SpawnOpts`, `StdioMcpConnection`, `StdioMcpServerSpec`, `SteerableSandboxArgs`, `Strategy`, `StrategyEvolutionConfig`, `StrategyResult`, `StreamAgentTurnOptions`, `StructuralRolloutConfig`, `SuperviseOptions`, `SuperviseSurfaceOptions`, `SupervisorAgentDeps`, `SupervisorOpts`, `SupervisorSpanOptions`, `SupervisorSpanRecorder`, `SurfaceScore`, `ToolSpec`, `ToolStepInput`, `TraceSource`, `TrajectoryAnalysis`, `UntrackedCopyStats`, `ValidationCtx`, `Validator`, `VerifierEnvironmentOptions`, `WatchTraceOptions`, `WaterfallCollector`, `WaterfallReport`, `WaterfallSpan`, `WorkerEvidenceInput`, `Workspace`, `WorkspaceRequest`, `WorkspaceRun`, `WorktreeCliExecutorOptions`, `WorktreeFanoutOptions`, `AgentEnvironmentStatus`, `AgentSessionStatus`, `ChampionPolicy`, `EdgeDeliveryOutcome`, `GraphEdge`, `LoopTraceEvent`, `MakeWorkerAgent`, `RepairStop`, `SandboxControlClient`, `WorkspaceCommit`. +**Undocumented supporting types** (add a TSDoc line at the declaration to earn a table row): `AcquireOptions`, `AgentEnvironment`, `AgentEnvironmentCapabilities`, `AgentEnvironmentEvent`, `AgentEnvironmentProvider`, `AgentEnvironmentQuery`, `AgentEnvironmentSummary`, `AgentGraph`, `AgenticOptions`, `AgenticRunResult`, `AgenticTask`, `AgenticTool`, `AgentSession`, `AgentSessionRef`, `AgentTurnResult`, `AllWorkersStalledOptions`, `AnalystRegistry`, `AnytimeReport`, `AnytimeStrategySummary`, `AnytimeTaskCurve`, `ArtifactHandle`, `AuditIntentInput`, `AuditIntentOptions`, `AuthoredHarness`, `AuthoredStrategy`, `AuthorStrategyOptions`, `BenchmarkConfig`, `BenchmarkLift`, `BenchmarkStrategySummary`, `BenchmarkTaskRow`, `BudgetPool`, `BusStats`, `ChampionPick`, `CheckpointRef`, `CheckpointRequest`, `CheckRunContext`, `CliWorktreeBridgeSeam`, `CoordinationMcpHandle`, `CopyOptions`, `CorpusReadbackOptions`, `CreateAgentEnvironmentInput`, `CreateTangleSandboxExactProcessProviderOptions`, `DefinedLeaderboard`, `DispatchReport`, `Driver`, `EvolutionArchiveNode`, `EvolutionAuthor`, `EvolutionBandInfo`, `EvolutionCandidate`, `EvolutionGeneration`, `EvolutionReport`, `ExecRequest`, `ExecResult`, `ExecutorResultMapping`, `ForkRequest`, `GitWorkspaceOptions`, `GraphResult`, `HarvestCorpusOptions`, `HarvestFailure`, `HarvestReport`, `HarvestSurfaceDiffsOptions`, `Inbox`, `InProcessSandboxClientOptions`, `IntentAudit`, `Iteration`, `Leaderboard`, `LeaderboardOptions`, `LocalSandboxClientOptions`, `LoopDecisionPayload`, `LoopDispatchOptions`, `LoopEndedPayload`, `LoopIterationEndedPayload`, `LoopIterationStartedPayload`, `LoopPlanDescription`, `LoopResult`, `LoopSandboxPlacement`, `LoopStartedPayload`, `LoopTraceEmitter`, `LoopWinner`, `MaterializeLocalMcpOptions`, `McpEnvironmentOptions`, `McpToolDescriptor`, `NodeSnapshot`, `NoProgressForOptions`, `Observation`, `ObserveInput`, `ObserveOptions`, `OpenSandboxRunOptions`, `PairwiseOptions`, `PatchDeliverableOptions`, `PlacementInfo`, `PlateauOptions`, `ProgressTrackerOptions`, `PromotionGateOptions`, `PromotionVerdict`, `PublishOptions`, `ReproductionCheck`, `ResolveSandboxClientOptions`, `ResourceRequest`, `RollingDispatchOptions`, `RunAgenticOptions`, `RunAgentRoundsOptions`, `RunGraphOptions`, `SandboxRun`, `ShotSpec`, `SpawnOpts`, `StdioMcpConnection`, `StdioMcpServerSpec`, `SteerableSandboxArgs`, `Strategy`, `StrategyEvolutionConfig`, `StrategyResult`, `StreamAgentTurnOptions`, `StructuralRolloutConfig`, `SuperviseOptions`, `SuperviseSurfaceOptions`, `SupervisorAgentDeps`, `SupervisorOpts`, `SupervisorSpanOptions`, `SupervisorSpanRecorder`, `SurfaceScore`, `ToolSpec`, `ToolStepInput`, `TraceSource`, `TrajectoryAnalysis`, `UntrackedCopyStats`, `ValidationCtx`, `Validator`, `VerifierEnvironmentOptions`, `WatchTraceOptions`, `WaterfallCollector`, `WaterfallReport`, `WaterfallSpan`, `WorkerEvidenceInput`, `Workspace`, `WorkspaceRequest`, `WorkspaceRun`, `WorktreeCliExecutorOptions`, `WorktreeFanoutOptions`, `AgentEnvironmentStatus`, `AgentSessionStatus`, `ChampionPolicy`, `EdgeDeliveryOutcome`, `GraphEdge`, `LoopTraceEvent`, `MakeWorkerAgent`, `RepairStop`, `SandboxControlClient`, `WorkspaceCommit`. ### Environment provider adapters — generic sandbox/compute bridge diff --git a/docs/api/runtime.md b/docs/api/runtime.md index afde9792..9cb1635f 100644 --- a/docs/api/runtime.md +++ b/docs/api/runtime.md @@ -16025,6 +16025,79 @@ returning an incomplete reproducibility receipt. *** +### SurfaceDiff + +One mounted surface whose settled state differs from what was mounted. + +- `modified` — the surface exists with different bytes (`settledSha256`/`settledBytes` present). +- `removed` — the surface no longer exists at its mounted path. +- `unreadable` — the read seam failed for a reason other than absence; `error` carries the + diagnostic. Reported rather than dropped so a permissions or transport failure cannot + masquerade as "nothing changed". + +#### Properties + +##### path + +> **path**: `string` + +The mounted path, exactly as recorded in the manifest entry. + +##### status + +> **status**: `"modified"` \| `"removed"` \| `"unreadable"` + +##### mountedSha256 + +> **mountedSha256**: `string` + +Hex SHA-256 of the bytes that were mounted (from the manifest). + +##### source + +> **source**: `string` + +Free-form origin of the mounted resource, carried through from the manifest. + +##### settledSha256? + +> `optional` **settledSha256?**: `string` + +Hex SHA-256 of the settled bytes. Present only for `modified`. + +##### settledBytes? + +> `optional` **settledBytes?**: `number` + +Size of the settled bytes. Present only for `modified`. + +##### error? + +> `optional` **error?**: `string` + +The read seam's diagnostic. Present only for `unreadable`. + +*** + +### HarvestSurfaceDiffsOptions + +#### Properties + +##### mounts + +> **mounts**: readonly [`MountManifestEntry`](#mountmanifestentry)[] + +The run's mount manifest (`RunProvenance.mounts`). Entries sharing a path are collapsed to the + LAST entry — the bytes the agent actually saw at start. + +##### read + +> **read**: [`SurfaceReader`](#surfacereader) + +How to read a mounted path's current bytes. + +*** + ### CreateTangleSandboxExactProcessProviderOptions #### Properties @@ -19480,6 +19553,34 @@ Terminal artifact of one worktree-CLI run — the canonical worktree-harness res *** +### SurfaceReadOutcome + +> **SurfaceReadOutcome** = \{ `succeeded`: `true`; `value`: `Uint8Array`; \} \| \{ `succeeded`: `false`; `missing`: `boolean`; `error`: `string`; \} + +Outcome of reading one surface back at settle. `missing: true` means the path no longer exists + (a deletion — a valid, reportable outcome); any other failure carries its diagnostic. + +*** + +### SurfaceReader + +> **SurfaceReader** = (`path`) => `Promise`\<[`SurfaceReadOutcome`](#surfacereadoutcome)\> + +The read seam: fetch the current bytes at a mounted path. Implemented by a sandbox box's + `fs.read`, a local worktree read ([fsSurfaceReader](#fssurfacereader)), or a test double. + +#### Parameters + +##### path + +`string` + +#### Returns + +`Promise`\<[`SurfaceReadOutcome`](#surfacereadoutcome)\> + +*** + ### SandboxControlClient > **SandboxControlClient** = `Pick`\<`Sandbox`, `"create"` \| `"get"` \| `"list"`\> @@ -24930,6 +25031,47 @@ the shared valid-only `selectValidWinner` (never a judge). *** +### harvestSurfaceDiffs() + +> **harvestSurfaceDiffs**(`options`): `Promise`\<[`SurfaceDiff`](#surfacediff)[]\> + +Re-read every mounted surface and report the ones whose settled state differs from the manifest. +Unchanged surfaces produce no entry; output preserves manifest record order. Surfaces the agent +CREATED (paths never mounted) are outside this contract — the manifest cannot see them; a +substrate that enumerates harness-state directories can feed those paths in as additional mounts. + +#### Parameters + +##### options + +[`HarvestSurfaceDiffsOptions`](#harvestsurfacediffsoptions) + +#### Returns + +`Promise`\<[`SurfaceDiff`](#surfacediff)[]\> + +*** + +### fsSurfaceReader() + +> **fsSurfaceReader**(`root`): [`SurfaceReader`](#surfacereader) + +A [SurfaceReader](#surfacereader) over the local filesystem, for worktree/local workers. Relative mount +paths resolve against `root`. Absence maps to `missing: true`; every other failure carries the +error message. + +#### Parameters + +##### root + +`string` + +#### Returns + +[`SurfaceReader`](#surfacereader) + +*** + ### createTangleSandboxExactProcessProvider() > **createTangleSandboxExactProcessProvider**(`client`, `options?`): `AgentEnvironmentProvider` diff --git a/docs/canonical-api.md b/docs/canonical-api.md index fb640e0b..3b124e3c 100644 --- a/docs/canonical-api.md +++ b/docs/canonical-api.md @@ -158,6 +158,7 @@ A general "loop" primitive is the single most common modelling error in this rep | See a **supervised tree** in a trace viewer (one span per node, opened at spawn, closed at settle, parented to its parent node; driver turns as LLM child spans) | `supervise(profile, task, { otel: { exporter } })`, or `createSupervisorSpanRecorder({ runId, … }).hooks` on `SupervisorOpts.hooks`: `/kernel` — OPT-IN, and omitting `otel` installs no hook at all | parsing the spawn journal to reconstruct the tree; a second exporter; routing replay/resume through telemetry (the journal stays the only durable record) | | Run an ordered analyst pass where later analysts use findings from earlier analysts | `runAnalystLoop({ chainFindings: true })`: `/analyst-loop`; registration order defines the dependency order, while omission keeps analysts independent | manually invoke each analyst and pipe findings between calls | | Know **what got mounted into a run** / **why a candidate won** | `result.provenance.mounts` / `result.provenance.selectionReceipts` (`MountManifestEntry`/`SelectionReceipt`/`RunProvenance`); declare mounts via the `recordMount` recorder in `prepareBox`: root export | re-reading box contents to reconstruct what was mounted, or re-deriving which candidate the selector picked | +| Know **which of its own mounted surfaces an agent CHANGED** during a run (instructions/skills/memory self-edits — the observed, session-scoped evidence that may later become a measured proposal) | `harvestSurfaceDiffs({ mounts, read })` (+ `fsSurfaceReader` for local worktrees): `/kernel` | a per-harness refinement-event parser, or ad-hoc re-reads of box contents that silently skip unreadable surfaces | | State any benchmark/A-B claim | `pairedLift(...)` (bench) over `pairedBootstrap`/`heldoutSignificance` (substrate) | your own bootstrap loop/PRNG per gate; a point lift without `low/high/pairs` | | Let an agent **delegate ONE generic INTENT** (no fixed coder/researcher type) and get the result + real spend SYNCHRONOUSLY | the **`delegate` tool**: `createDelegateHandler` via `createMcpServer({ delegateSupervisor })`; mount it over the `agent-runtime mcp` bin with `MCP_ENABLE_DELEGATE=1` (the bin authors a supervisor over a `sandbox` backend): `/mcp` | a hardcoded coder/researcher profile, or task-specific `delegate_code`/`delegate_research` verbs (RETIRED): `delegate` is the ONE delegation path and the only one with a cost channel | | Run a coding task INSIDE the agent's OWN sandbox session (a sibling box, fresh branch, validated patch) | `detachedSessionDelegate({ sandboxClient \| executor, workerProfile, fanoutProfiles? })`: `/mcp` (the required exact worker profile owns harness, provider, model, prompt, and tools; optional exact profiles make fanout heterogeneous) | a hardcoded coder profile baked into the delegate; `delegate()` (that spawns workers in a *chosen* backend, not the agent's own session) | diff --git a/docs/design/prime-agent-harness-integration.md b/docs/design/prime-agent-harness-integration.md new file mode 100644 index 00000000..3f73a1d1 --- /dev/null +++ b/docs/design/prime-agent-harness-integration.md @@ -0,0 +1,176 @@ +# Prime Agent as a harness — boundary, adoption map, substrate wish-list + +> **Status: decision record + integration contract.** Prime Agent (Prime Intellect's +> local agent OS: persistent IPython kernel, daemon-backed session trees, native `rlm(…)` +> subagents, `/refine` continual-harness edits) is being added as a **sandbox-materialized +> harness** — the sandbox/adc side owns the image, the adapter, and the `prime-agent` +> harness id. This doc owns the Runtime side: the boundary we hold, which proposed +> integration machinery we adopt or reject (most of it already exists here), and the +> concrete asks on the substrate. On architecture conflict, +> [architecture.md](../architecture.md) wins. + +## 1. The boundary (the one non-negotiable) + +**A harness owns its native inner model/tool/subagent loop. Runtime owns the external +execution identity, budget, lifecycle contract, and cross-harness coordination. Agent +Eval owns verdicts. Intelligence owns promotion.** + +This is not new policy — it is [canonical-api.md §1.5](../canonical-api.md) applied to a +harness that happens to be unusually complete. Acceptance criteria: + +- No Prime-specific inner-loop logic enters Runtime. Prime's IPython surface, session + JSONL schema, daemon protocol, and TUI are harness features, not Runtime contracts. +- No Runtime tool loop wraps Prime's own loop. Prime is a **leaf** behind the one + `Executor` port / a sandbox backend, exactly like claude-code or codex. +- Prime's local `AgentConnection`/daemon protocol never becomes a public Runtime API. + Callers see opaque run ids, session ids, artifact refs, and receipts. +- A stronger harness is a reason to author better profiles, never to specialize Runtime: + a lever Prime exposes that the profile cannot yet express is a **materializer gap in + the substrate**, not a bespoke Prime adapter here. + +## 2. Anti-reinvention map — what the integration needs that already exists + +Most of the machinery a "host Prime properly" plan calls for is built. Reuse it. + +| Integration need | Existing primitive | Where | +|---|---|---| +| Harness identity + per-harness capability truth | `HarnessType` + the harness-capability layer (model lock, reasoning clamp, selector honoring) | `@tangle-network/agent-interface` (`harness.ts`, `harness-capabilities.ts`) | +| Environment capability negotiation (streaming, sessions, workspace, **checkpoint/fork**) | `AgentEnvironmentCapabilities` incl. `branching.checkpoint/fork`; `AgentEnvironment.checkpoint()/fork()` | `src/runtime/environment-provider.ts` | +| Portable capability with per-harness lowerings ("one skill, many bindings") | `CertifiedCapability` = `{ iface, binding, auth, provenance }` — interfaces closed, bindings open; a Prime Python skill is one binding arm (`file`/`sandbox-code`), never a new manifest format | `src/intelligence/capability.ts` | +| Execution-identity receipts (what exactly ran) | `materialized` journal event (authored/effective profile + platform-attachment digests), `RunProvenance.mounts`, `SelectionReceipt`, usage integrity guard | `src/runtime/supervise/`, `src/runtime/types.ts` | +| Frozen candidate identity + workspace bytes | `buildAgentCandidateBundle` / `captureAgentCandidateWorkspace` | `src/candidate-execution/` | +| Improvement governance (train/selection/final-test partitions, review, activation, rollback receipts) | `improve()` + `proposeAgentImprovement` → `reviewAgentImprovementProposal` → activation | `src/improvement/`, `src/intelligence/` | +| Equal-compute topology comparison + holdout gates | conserved budget pool, `pairedLift`, `promotionGate`, `heldOutGate` | `src/runtime/supervise/budget.ts`, bench/eval | +| Harness × model × task matrix | `defineLeaderboard` / `runProfileMatrix` (harness×model axes) | `src/runtime/define-leaderboard.ts` | +| Harness-agnostic trace normalization | `TraceSource` + the per-harness `toolPartDecoders` registry (add a harness = one validated decoder entry) | `src/runtime/supervise/trace-source.ts` | +| Uncertain-effect posture after restart | uncertain reservations charged at full declared budget with telemetry explicitly unknown; instruction receipts retained as evidence, never auto-redelivered | `src/runtime/supervise/budget.ts`, `supervise/coordination-mcp.ts` | +| Prime Intellect ecosystem bridge (verifiers env packaging, trace import) | `writePrimeIntellectPackage`, `importPrimeIntellectTraces` → `RunRecord` | `src/primeintellect/` | + +## 3. What Runtime adopts + +### 3.1 Surface-diff harvest (built — `harvestSurfaceDiffs`, `/kernel`) + +The harness-agnostic generalization of Prime's `/refine` observation, and the read-back +dual of the mount manifest: `RunProvenance.mounts` records what the substrate placed into +a run (instructions, skills, memory); `harvestSurfaceDiffs({ mounts, read })` re-hashes +those surfaces at settle and reports what the agent itself changed +(`src/runtime/surface-diff.ts`). Every harness self-mutates this way — claude-code edits +CLAUDE.md/memory files, opencode edits AGENTS.md, Prime edits its continual harness — and +today that evidence evaporates at box teardown. + +The scope law it feeds (Prime's `/refine` scopes, generalized to any harness): + +- **Session-scoped self-edits are observations.** They applied during the run, they are + recorded (path, mounted/settled hashes, source), they promote nothing. +- **Reuse-scoped edits become candidates.** A diff worth keeping is materialized through + the existing proposal pipeline (`proposeAgentImprovement` → review → gated activation) + and must clear the same held-out, equal-compute gate as any other candidate. An LLM + rationale ("expectedOutcome") is never promotion evidence — the trajectory that + motivated an edit is not an independent measurement of it. + +### 3.2 When the `prime-agent` id ships (blocked on substrate) + +Small, mechanical, and compile-pinned — do these in the release that picks up the new +`agent-interface`/`sandbox` versions: + +1. Add `'prime-agent'` to `harnessBackends` in `src/runtime/sandbox-backend.ts` (the + double-`satisfies` makes the absence a compile error, so this cannot be forgotten). +2. Register a `toolPartDecoders['prime-agent']` entry — see wish-list item 3 for why this + should be the canonical `ToolPart` decoder, not a bespoke one. +3. Add a **measured** column to + [research/harness-compat.md](../research/harness-compat.md). Prime is Pi-lineage but + not Pi: do not inherit `pi`'s capability row by assumption. + +## 4. What we deliberately do NOT build + +- **A `TopologyAuthority` type.** The fear it answers — Runtime spawns 5 Prime roots, + each spawns 5 native children, unbounded — is structurally impossible here: a worker's + native subagents spend **inside its conserved-budget reservation** (`supervise/budget.ts`), + so no arm can buy more compute regardless of what the harness does internally. The + layering doctrine already assigns native orchestration to the harness (layer one) and + cross-harness composition to Runtime. The three "modes" are expressible today as + authoring decisions: harness-owned = spawn one worker and let it expand; runtime-owned + = author a profile with native subagents disabled (where the harness supports it — see + the compat matrix); hybrid = the default. What is genuinely missing is *visibility* + into native children (wish-list 4) and *fan-out caps as a materializable profile lever* + (wish-list 8) — observability and configuration, not a new authority enum. +- **A `HarnessExecutionReceipt` envelope.** The fields exist across the `materialized` + journal event, `RunProvenance`, the budget ledger, and candidate bundles. Missing + fields (environment/image digest, harness version, harness-state digest, snapshot + refs) are substrate facts — the fix is the substrate *reporting* them (wish-list 5) so + existing receipts can carry them, not a parallel envelope Runtime must keep consistent + with the ones it already emits. +- **An `AdvancedHarnessBackend` optional-method interface.** Capability truth lives in + agent-interface (`harness-capabilities`, `AgentEnvironmentCapabilities`) and behavior + behind the existing ports (`Executor.progress`/`traceSource`, provider + `checkpoint`/`fork`). Growing the backend port into a 10-optional-method surface + re-creates the closed adapter zoo the one `Executor` port replaced. +- **A persistent-kernel / programmable-computer abstraction in Runtime.** Prime's IPython + surface is a strong cognitive substrate and a real reason to route long-horizon + data-heavy tasks to Prime — via the leaderboard matrix and profile authoring, not via a + Runtime kernel API. If the pattern proves out empirically, the portable expression is a + capability binding (a code-execution MCP) any harness can mount, measured like any + other candidate. +- **Prime's daemon/durability layer inside Runtime.** Reconnect, leases, adoption, + replay cursors belong to the sandbox/orchestrator boundary (see + [agent-managed-compute/](../agent-managed-compute/)). Prime's daemon patterns + (generation-aware cursors, claim-before-delivery, explicit uncertain mutations) are + good prior art for that work where Runtime doesn't already have the equivalent. + +## 5. Substrate wish-list (the asks on `@tangle-network/sandbox` / adc) + +Numbered so the sandbox-side work can check them off. Items 1–5 are required for +instrumented runs; 6–9 unlock the experiment tier. + +1. **Harness identity.** `'prime-agent'` in `HarnessType` (agent-interface) and as a + sandbox `backend.type` — a distinct id, not `pi`. Capability rows (model lock, + reasoning clamp, selector honoring) measured against the real CLI, not inherited. +2. **Profile materialization.** Render an `AgentProfile` into Prime's native surfaces — + system-prompt addendum, skills (as Prime executable skills or SKILL.md), MCP config, + subagent specs, hooks where expressible — with the standard materialization receipt + (exact mounted bytes + digests). A profile lever Prime cannot express must fail loud + in the materializer, not silently drop. +3. **Canonical tool parts.** Normalize Prime session output into agent-interface's + published `ToolPart`/`ToolState` (terminal-state semantics + `callId` dedup), the + shape every adc sdk-provider already targets. Then Runtime's existing decoder handles + Prime with zero new decode logic. If normalization is impossible for some part kinds, + publish the raw wire shape so a decoder entry can be *validated* (the registry rule: + a decoder lands only checked against the harness's real output). +4. **Native session tree exposure.** Prime's root session and `rlm` children as + session-tree metadata (session id + parent id + status + per-child usage) readable + off the box. Runtime projects these as child spans/runs; without it a Prime worker is + one opaque blob and per-child cost attribution is lost. +5. **Environment identity reporting.** Image digest, Prime version, Python-environment + digest, and a continual-harness state digest surfaced on the box/session so run + receipts can bind results to the *effective* behavioral identity (profile digest alone + under-identifies a harness with mutable local state). +6. **Kernel-cell and effect visibility.** One IPython cell can read 100 files, hit the + network, and spawn children. Surface cell start/end, shell subprocesses, and file + effects as parts or a metadata channel so trace analysts see causal structure, not + one giant tool call. (Order/count first; byte-accurate effect receipts can come + later.) +7. **Refinement events.** Prime `/refine` plans and applied edits (scope, surface, + before/after hashes, rationale) surfaced on the session stream, and scope policy + enforceable at materialization: session-local edits apply freely; project/global + edits are emitted as events for the host to lift into proposals, never self-applied + to shared state from inside a run. During gated evaluation: auto-refinement off, + candidate and baseline harness state frozen and separated. +8. **Fan-out caps as config.** Prime's native child limit (and equivalents on other + harnesses) as a materializable knob so an authored profile can bound native expansion. +9. **Whole-environment checkpoint/branch.** Box snapshot + branch behind the existing + `AgentEnvironment.checkpoint()/fork()` contract, covering workspace + harness state + (kernel heap where feasible). A transcript-tree branch is not a world branch; paired + baseline/candidate arms need same-state forks. Credentials stay short-lived and + injected — never serialized into snapshots, session JSONL, or harness state. + +## 6. The first gated experiment (when 1–5, 7, 9 land) + +One question, run through the existing machinery (no new rig): **does a +`/refine`-derived edit improve held-out tasks, or only continuity on the task that +produced it?** Source tasks generate candidate edits; `improve()`'s partitions select +and hold out; both arms run frozen harness state at equal compute under a deployable +verifier; `proposeAgentImprovement` carries the winner to review. Ablate edit kinds +(memory / skill / subagent spec / bundle) against a same-size generic-edit control. +Powered paired n per the eval substrate's rules — not anecdotes. Separately (never in +the same causal claim): Prime vs other harnesses on the same profile via the +leaderboard matrix. diff --git a/docs/research/harness-compat.md b/docs/research/harness-compat.md index 7f11eb0d..d986986b 100644 --- a/docs/research/harness-compat.md +++ b/docs/research/harness-compat.md @@ -28,5 +28,18 @@ Harnesses wired via cli-bridge: **claude-code** (2.1.177), **codex** (0.139.0), 5. **Three different MCP wiring mechanisms** (file-flag / synthetic-HOME-TOML / env-config-file) — no uniform `--mcp-config`. opencode http = `"type":"remote"`+`url` (the MEMORY `transport:'http'` note is the *claude/kimi* `--mcp-config` layer, a different file). 6. **Resume identity differs per harness** (claude uuid / codex thread_id / opencode session id) — a driver resuming across a sandbox boundary must keep the external→internal id map per-harness; ids are not cross-harness valid. +## Pending columns + +- **prime-agent** — Prime Intellect's agent OS (persistent IPython kernel, native `rlm(…)` + subagents, `/refine` continual-harness edits, daemon-backed session trees). Sandbox + backend in progress; see + [design/prime-agent-harness-integration.md](../design/prime-agent-harness-integration.md). + **Do not inherit `pi`'s row** — Prime is Pi-lineage but adds materially different + semantics; every cell must be measured against the real CLI before this column fills in. + Expected steering-relevant deltas to verify: native subagent fan-out cap and whether it + is configurable (runaway surface #3), whether the daemon protocol allows mid-step + interrupt (black-box harnesses today degrade `steer_agent({interrupt:true})` to next + spawn), and whether tool activity surfaces as structured parts or only via the kernel. + ## Files `~/code/cli-bridge/src/backends/{claude,codex,opencode}.ts` (invocation), `.../profile-support.ts` (MCP materializers), `.../modes.ts` (byob/hosted-safe gating); `~/code/agent-dev-container` sidecar registry (12 bindings). diff --git a/src/runtime/index.ts b/src/runtime/index.ts index bfde2e72..d5095af7 100644 --- a/src/runtime/index.ts +++ b/src/runtime/index.ts @@ -944,6 +944,14 @@ export { type SurfaceWorkerOut, superviseSurface, } from './supervise-surface' +export { + fsSurfaceReader, + type HarvestSurfaceDiffsOptions, + harvestSurfaceDiffs, + type SurfaceDiff, + type SurfaceReader, + type SurfaceReadOutcome, +} from './surface-diff' export type { SandboxControlClient } from './tangle-sandbox-exact-process-provider' // Profile-owned supervisor configuration. The arbitrary model callback and raw driver constructor // live only under `/testing`; production model execution enters through `supervise(AgentProfile)`. diff --git a/src/runtime/surface-diff.test.ts b/src/runtime/surface-diff.test.ts new file mode 100644 index 00000000..ad8f29df --- /dev/null +++ b/src/runtime/surface-diff.test.ts @@ -0,0 +1,115 @@ +import { createHash } from 'node:crypto' +import { mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterAll, describe, expect, it } from 'vitest' +import { fsSurfaceReader, harvestSurfaceDiffs, type SurfaceReader } from './surface-diff' +import type { MountManifestEntry } from './types' + +const sha = (s: string) => createHash('sha256').update(s).digest('hex') + +const mount = (path: string, content: string, source = 'test'): MountManifestEntry => ({ + path, + sha256: sha(content), + bytes: Buffer.byteLength(content), + source, +}) + +const readerOf = (files: Record): SurfaceReader => { + return (path) => { + const content = files[path] + if (content === undefined) + return Promise.resolve({ succeeded: false, missing: true, error: 'ENOENT' }) + return Promise.resolve({ succeeded: true, value: new TextEncoder().encode(content) }) + } +} + +describe('harvestSurfaceDiffs', () => { + it('reports nothing when every surface settles with its mounted bytes', async () => { + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('CLAUDE.md', 'instructions'), mount('memory/notes.md', 'notes')], + read: readerOf({ 'CLAUDE.md': 'instructions', 'memory/notes.md': 'notes' }), + }) + expect(diffs).toEqual([]) + }) + + it('reports a modified surface with the settled hash and size, preserving record order', async () => { + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('a.md', 'aaa'), mount('b.md', 'bbb', 'corpus:42'), mount('c.md', 'ccc')], + read: readerOf({ 'a.md': 'aaa', 'b.md': 'bbb-edited-by-agent', 'c.md': 'ccc!' }), + }) + expect(diffs.map((d) => d.path)).toEqual(['b.md', 'c.md']) + expect(diffs[0]).toEqual({ + path: 'b.md', + status: 'modified', + mountedSha256: sha('bbb'), + source: 'corpus:42', + settledSha256: sha('bbb-edited-by-agent'), + settledBytes: Buffer.byteLength('bbb-edited-by-agent'), + }) + }) + + it('reports a missing surface as removed and a failed read as unreadable with its error', async () => { + const read: SurfaceReader = (path) => { + if (path === 'gone.md') + return Promise.resolve({ succeeded: false, missing: true, error: 'ENOENT' }) + return Promise.resolve({ succeeded: false, missing: false, error: 'EACCES: denied' }) + } + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('gone.md', 'x'), mount('locked.md', 'y')], + read, + }) + expect(diffs).toEqual([ + { path: 'gone.md', status: 'removed', mountedSha256: sha('x'), source: 'test' }, + { + path: 'locked.md', + status: 'unreadable', + mountedSha256: sha('y'), + source: 'test', + error: 'EACCES: denied', + }, + ]) + }) + + it('collapses duplicate paths to the last mount — the bytes the agent actually saw', async () => { + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('a.md', 'first'), mount('a.md', 'second')], + read: readerOf({ 'a.md': 'second' }), + }) + expect(diffs).toEqual([]) + }) + + it('treats an uppercase manifest hash as equal to the settled lowercase hash', async () => { + const entry = { ...mount('a.md', 'same'), sha256: sha('same').toUpperCase() } + const diffs = await harvestSurfaceDiffs({ + mounts: [entry], + read: readerOf({ 'a.md': 'same' }), + }) + expect(diffs).toEqual([]) + }) +}) + +describe('fsSurfaceReader', () => { + const root = mkdtempSync(join(tmpdir(), 'surface-diff-')) + afterAll(() => rmSync(root, { recursive: true, force: true })) + + it('reads relative paths against the root and maps absence to missing', async () => { + writeFileSync(join(root, 'notes.md'), 'hello') + const read = fsSurfaceReader(root) + const hit = await read('notes.md') + expect(hit.succeeded).toBe(true) + if (hit.succeeded) expect(new TextDecoder().decode(hit.value)).toBe('hello') + const miss = await read('absent.md') + expect(miss).toMatchObject({ succeeded: false, missing: true }) + }) + + it('composes with the harvest over a real worktree edit', async () => { + writeFileSync(join(root, 'skill.md'), 'v1') + const mounts = [mount('skill.md', 'v1')] + writeFileSync(join(root, 'skill.md'), 'v2') + const diffs = await harvestSurfaceDiffs({ mounts, read: fsSurfaceReader(root) }) + expect(diffs).toMatchObject([ + { path: 'skill.md', status: 'modified', settledSha256: sha('v2') }, + ]) + }) +}) diff --git a/src/runtime/surface-diff.ts b/src/runtime/surface-diff.ts new file mode 100644 index 00000000..648141f3 --- /dev/null +++ b/src/runtime/surface-diff.ts @@ -0,0 +1,142 @@ +/** + * + * Settle-time surface-diff harvest — the read-back dual of the mount manifest. + * + * `RunProvenance.mounts` records what the caller placed INTO a run's workspace before the agent saw + * it (instructions, skills, memory files — the profile surfaces the substrate materialized). This + * module answers the reverse question at settle: **which of those mounted surfaces did the agent + * itself change while working?** Harnesses mutate their own profile-adjacent surfaces as a matter of + * course — a memory file appended to, an instructions file edited, a skill rewritten — and that + * self-mutation is improvement-relevant evidence: an OBSERVED, session-scoped edit that may later be + * lifted into the measured proposal pipeline (`proposeAgentImprovement`), never auto-promoted. + * + * The harvest is harness-agnostic by construction: it compares content hashes against the mount + * manifest, so it needs no harness event format, no refinement protocol, and no knowledge of WHY a + * surface changed. A harness that additionally reports structured self-edit events can enrich this + * signal; nothing requires it to. + * + * The kernel never reads workspace contents itself — the caller supplies the read seam (a box + * `fs.read`, a worktree read, a test double), mirroring how `recordMount` keeps mount hashing with + * the byte owner. Reads return typed outcomes; a failed read is reported as an `unreadable` diff + * carrying the error, never silently dropped. + * + * @experimental + */ + +import { createHash } from 'node:crypto' +import { readFile } from 'node:fs/promises' +import { isAbsolute, resolve } from 'node:path' +import type { MountManifestEntry } from './types' + +/** Outcome of reading one surface back at settle. `missing: true` means the path no longer exists + * (a deletion — a valid, reportable outcome); any other failure carries its diagnostic. */ +export type SurfaceReadOutcome = + | { succeeded: true; value: Uint8Array } + | { succeeded: false; missing: boolean; error: string } + +/** The read seam: fetch the current bytes at a mounted path. Implemented by a sandbox box's + * `fs.read`, a local worktree read ({@link fsSurfaceReader}), or a test double. */ +export type SurfaceReader = (path: string) => Promise + +/** + * One mounted surface whose settled state differs from what was mounted. + * + * - `modified` — the surface exists with different bytes (`settledSha256`/`settledBytes` present). + * - `removed` — the surface no longer exists at its mounted path. + * - `unreadable` — the read seam failed for a reason other than absence; `error` carries the + * diagnostic. Reported rather than dropped so a permissions or transport failure cannot + * masquerade as "nothing changed". + */ +export interface SurfaceDiff { + /** The mounted path, exactly as recorded in the manifest entry. */ + path: string + status: 'modified' | 'removed' | 'unreadable' + /** Hex SHA-256 of the bytes that were mounted (from the manifest). */ + mountedSha256: string + /** Free-form origin of the mounted resource, carried through from the manifest. */ + source: string + /** Hex SHA-256 of the settled bytes. Present only for `modified`. */ + settledSha256?: string + /** Size of the settled bytes. Present only for `modified`. */ + settledBytes?: number + /** The read seam's diagnostic. Present only for `unreadable`. */ + error?: string +} + +export interface HarvestSurfaceDiffsOptions { + /** The run's mount manifest (`RunProvenance.mounts`). Entries sharing a path are collapsed to the + * LAST entry — the bytes the agent actually saw at start. */ + mounts: readonly MountManifestEntry[] + /** How to read a mounted path's current bytes. */ + read: SurfaceReader +} + +const sha256Hex = (bytes: Uint8Array): string => createHash('sha256').update(bytes).digest('hex') + +/** + * Re-read every mounted surface and report the ones whose settled state differs from the manifest. + * Unchanged surfaces produce no entry; output preserves manifest record order. Surfaces the agent + * CREATED (paths never mounted) are outside this contract — the manifest cannot see them; a + * substrate that enumerates harness-state directories can feed those paths in as additional mounts. + */ +export async function harvestSurfaceDiffs( + options: HarvestSurfaceDiffsOptions, +): Promise { + const byPath = new Map() + for (const entry of options.mounts) byPath.set(entry.path, entry) + const diffs: SurfaceDiff[] = [] + for (const entry of byPath.values()) { + const outcome = await options.read(entry.path) + if (!outcome.succeeded) { + diffs.push( + outcome.missing + ? { + path: entry.path, + status: 'removed', + mountedSha256: entry.sha256, + source: entry.source, + } + : { + path: entry.path, + status: 'unreadable', + mountedSha256: entry.sha256, + source: entry.source, + error: outcome.error, + }, + ) + continue + } + const settledSha256 = sha256Hex(outcome.value) + if (settledSha256 === entry.sha256.toLowerCase()) continue + diffs.push({ + path: entry.path, + status: 'modified', + mountedSha256: entry.sha256, + source: entry.source, + settledSha256, + settledBytes: outcome.value.byteLength, + }) + } + return diffs +} + +/** + * A {@link SurfaceReader} over the local filesystem, for worktree/local workers. Relative mount + * paths resolve against `root`. Absence maps to `missing: true`; every other failure carries the + * error message. + */ +export function fsSurfaceReader(root: string): SurfaceReader { + return async (path) => { + const target = isAbsolute(path) ? path : resolve(root, path) + try { + return { succeeded: true, value: new Uint8Array(await readFile(target)) } + } catch (err) { + const code = (err as NodeJS.ErrnoException).code + return { + succeeded: false, + missing: code === 'ENOENT' || code === 'ENOTDIR', + error: err instanceof Error ? err.message : String(err), + } + } + } +} From 569f02e28f7a63515b48735f1826e02722543935 Mon Sep 17 00:00:00 2001 From: drewstone Date: Tue, 11 Aug 2026 20:16:40 +0000 Subject: [PATCH 02/10] feat(kernel): box-backed surface reader + created-surface watch; correct checkpoint/branch claims MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit boxSurfaceReader joins harvestSurfaceDiffs to the real box.fs.read seam (the one openSandboxRun reads deliverables through), with the SDK not-found error detected structurally. watch entries cover surfaces the agent CREATED at never-mounted paths — the common shape of a harness-authored memory/skill file — while the harvest stays layout-agnostic (the caller enumerates). Integration doc corrections: box.snapshot/branch, the CRIU capability probe, and the fork lineage already exist, so wish-list item 9 narrows to verifying snapshot coverage of the Prime daemon + kernel process tree; and a daemon-backed harness maps naturally onto the AgentEnvironmentProvider contract, so the adapter shape is per-tier (box backend vs provider), not forced. --- docs/api/primitive-catalog.md | 9 +- docs/api/runtime.md | 104 +++++++++++++++--- docs/canonical-api.md | 2 +- .../design/prime-agent-harness-integration.md | 32 +++++- src/runtime/index.ts | 3 + src/runtime/surface-diff.test.ts | 89 ++++++++++++++- src/runtime/surface-diff.ts | 87 +++++++++++++-- 7 files changed, 291 insertions(+), 35 deletions(-) diff --git a/docs/api/primitive-catalog.md b/docs/api/primitive-catalog.md index d26ad6a7..15b487a5 100644 --- a/docs/api/primitive-catalog.md +++ b/docs/api/primitive-catalog.md @@ -504,7 +504,7 @@ Import from `@tangle-network/agent-runtime/intelligence` — 166 exports. ### Execution kernel — recursive atom, supervision, executors, round-synchronous loop -Import from `@tangle-network/agent-runtime/kernel` — 715 exports. +Import from `@tangle-network/agent-runtime/kernel` — 718 exports. | Symbol | Kind | Summary | |---|---|---| @@ -524,6 +524,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 715 exports. | `auditIntent` | function | The route-rigor analyst: compare declared vs revealed vs user intent over a trajectory and return aligned / drifting / diverged with evidence and one recommended intervention. | | `authorStrategy` | function | Author + load a strategy from losses. Throws when the author emits no loadable module; | | `bestSoFar` | function | The best-so-far fold — the ONE definition of "how good was the run after k results", shared by | +| `boxSurfaceReader` | function | A {@link SurfaceReader} over a sandbox box's filesystem — the same `box.fs.read` seam | | `breadthStrategy` | function | BREADTH: K independent rollouts (each own artifact), verifier picks the best. | | `buildSteerContext` | function | Build the `SteerContext` a combinator reads to steer (its `loopUntil.until`, `widen` gate, any | | `canDisplace` | function | The repair keep-best guard: a challenger displaces the incumbent only when it is | @@ -597,7 +598,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 715 exports. | `gateOnDeliverable` | function | Wrap an `Executor` so its settlement `valid` reflects the deliverable check, not the | | `gitWorkspace` | function | A `Workspace` over a git checkout: materialize an isolated worktree at `ref`, commit produced changes (conflict-aware), and read `head` — hooks disabled, identity pinned. | | `harvestCorpus` | function | Batch the firewalled `observe()` analyst over completed runs and accrete the trace-derived facts into the durable corpus — the production-traces→corpus write side of the flywheel. | -| `harvestSurfaceDiffs` | function | Re-read every mounted surface and report the ones whose settled state differs from the manifest. | +| `harvestSurfaceDiffs` | function | Re-read every mounted (and watched) surface and report the ones whose settled state differs from | | `inlineSandboxClient` | function | Adapt an `ExecutorFactory` into a `SandboxClient` for `runAgentRounds`. The factory is | | `inProcessSandboxClient` | function | Adapt a single `onPrompt(prompt, ctx)` callback into a `SandboxClient` for | | `isWaitOutcome` | function | Narrow a settlement's `out` to a wait outcome — a wait settles on the SAME cursor as workers, | @@ -956,7 +957,8 @@ Import from `@tangle-network/agent-runtime/kernel` — 715 exports. | `SupervisorSpanOutcome` | interface | How the supervised run ended, as `finish()` records it on the root span. | | `SupervisorToolDescriptor` | interface | One product-owned tool. It reuses the canonical MCP descriptor fields while Runtime supplies | | `SupervisorToolInvocationContext` | interface | Trusted context for one product-tool invocation. The node identity remains the same detached, | -| `SurfaceDiff` | interface | One mounted surface whose settled state differs from what was mounted. | +| `SurfaceDiff` | interface | One watched surface whose settled state differs from what was mounted (or from absence). | +| `SurfaceReadBox` | interface | The minimal box surface the box-backed reader needs — structurally typed so the real | | `SurfaceWorkerConfig` | interface | How a worker runs the surface task (its router substrate + per-attempt bounds). | | `SurfaceWorkerOut` | interface | What a surface worker settles with — the surface verdict the driver + deliverable read. `resolved` is | | `ToolLoopCompaction` | interface | Self-compaction — bound the loop's OWN context window the way a fresh-respawn (dumb-Ralph) loop | @@ -971,6 +973,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 715 exports. | `WaitOpts` | interface | Options for `Scope.wait`. `label` is the wait's identity within its parent scope — it is what | | `WaitOutcome` | interface | The `out` a settled wait node delivers through `Scope.next()`. `settled` is the outcome the | | `WaitProbeRegistry` | interface | Resolves a `poll` spec's `probe` name to its predicate. Threaded through `SupervisorOpts` so | +| `WatchedSurface` | interface | A path to check at settle that was NOT necessarily mounted — where a harness is known to write | | `WidenGate` | interface | The progressive-widening gate (MCTS-PW). Decides whether a settled child is | | `WidenLineage` | interface | A lineage the gate may widen toward — the settled child that looked promising + the findings | | `WidenSpec` | interface | `widen({ gate })` (G5) — the STREAMING spawn-on-completion driver. Unlike the static-fanout | diff --git a/docs/api/runtime.md b/docs/api/runtime.md index 9cb1635f..55975ec5 100644 --- a/docs/api/runtime.md +++ b/docs/api/runtime.md @@ -16027,10 +16027,12 @@ returning an incomplete reproducibility receipt. ### SurfaceDiff -One mounted surface whose settled state differs from what was mounted. +One watched surface whose settled state differs from what was mounted (or from absence). - `modified` — the surface exists with different bytes (`settledSha256`/`settledBytes` present). - `removed` — the surface no longer exists at its mounted path. +- `created` — a watched path that was never mounted now exists (`settledSha256`/`settledBytes` + present, no `mountedSha256`) — the shape a harness's new memory/skill file takes. - `unreadable` — the read seam failed for a reason other than absence; `error` carries the diagnostic. Reported rather than dropped so a permissions or transport failure cannot masquerade as "nothing changed". @@ -16041,35 +16043,35 @@ One mounted surface whose settled state differs from what was mounted. > **path**: `string` -The mounted path, exactly as recorded in the manifest entry. +The mounted/watched path, exactly as recorded. ##### status -> **status**: `"modified"` \| `"removed"` \| `"unreadable"` +> **status**: `"modified"` \| `"removed"` \| `"created"` \| `"unreadable"` -##### mountedSha256 +##### mountedSha256? -> **mountedSha256**: `string` +> `optional` **mountedSha256?**: `string` -Hex SHA-256 of the bytes that were mounted (from the manifest). +Hex SHA-256 of the bytes that were mounted (from the manifest). Absent for `created`. ##### source > **source**: `string` -Free-form origin of the mounted resource, carried through from the manifest. +Free-form origin: the manifest entry's `source`, or the watch entry's `source`. ##### settledSha256? > `optional` **settledSha256?**: `string` -Hex SHA-256 of the settled bytes. Present only for `modified`. +Hex SHA-256 of the settled bytes. Present for `modified` and `created`. ##### settledBytes? > `optional` **settledBytes?**: `number` -Size of the settled bytes. Present only for `modified`. +Size of the settled bytes. Present for `modified` and `created`. ##### error? @@ -16079,6 +16081,26 @@ The read seam's diagnostic. Present only for `unreadable`. *** +### WatchedSurface + +A path to check at settle that was NOT necessarily mounted — where a harness is known to write + self-authored surfaces (a memory dir's files, a refinement log). A watched path that was also + mounted compares against its mount; one that wasn't reports `created` if it now exists. + +#### Properties + +##### path + +> **path**: `string` + +##### source? + +> `optional` **source?**: `string` + +Origin label carried onto the diff (default `'watched'`). + +*** + ### HarvestSurfaceDiffsOptions #### Properties @@ -16096,6 +16118,41 @@ The run's mount manifest (`RunProvenance.mounts`). Entries sharing a path are co How to read a mounted path's current bytes. +##### watch? + +> `optional` **watch?**: readonly [`WatchedSurface`](#watchedsurface)[] + +Additional paths to check that may not have been mounted (see [WatchedSurface](#watchedsurface)). The + caller enumerates them (it knows the harness's state layout — e.g. via the box's file tree); + the harvest stays layout-agnostic. + +*** + +### SurfaceReadBox + +The minimal box surface the box-backed reader needs — structurally typed so the real + `@tangle-network/sandbox` box and a test double both satisfy it, no SDK import. + +#### Properties + +##### fs + +> **fs**: `object` + +###### read() + +> **read**(`path`): `Promise`\<`string`\> + +###### Parameters + +###### path + +`string` + +###### Returns + +`Promise`\<`string`\> + *** ### CreateTangleSandboxExactProcessProviderOptions @@ -25035,10 +25092,9 @@ the shared valid-only `selectValidWinner` (never a judge). > **harvestSurfaceDiffs**(`options`): `Promise`\<[`SurfaceDiff`](#surfacediff)[]\> -Re-read every mounted surface and report the ones whose settled state differs from the manifest. -Unchanged surfaces produce no entry; output preserves manifest record order. Surfaces the agent -CREATED (paths never mounted) are outside this contract — the manifest cannot see them; a -substrate that enumerates harness-state directories can feed those paths in as additional mounts. +Re-read every mounted (and watched) surface and report the ones whose settled state differs from +the manifest — modified, removed, or created. Unchanged surfaces and still-absent watched paths +produce no entry; output preserves record order, mounts before watch-only paths. #### Parameters @@ -25052,6 +25108,28 @@ substrate that enumerates harness-state directories can feed those paths in as a *** +### boxSurfaceReader() + +> **boxSurfaceReader**(`box`): [`SurfaceReader`](#surfacereader) + +A [SurfaceReader](#surfacereader) over a sandbox box's filesystem — the same `box.fs.read` seam +`openSandboxRun` reads deliverables through. The box wire returns UTF-8 text, so this reader +covers TEXT surfaces (which profile surfaces are); hashes are computed over the UTF-8 encoding. +The SDK's not-found error is detected structurally (`err.name === 'NotFoundError'`) and maps to +`missing: true`; every other failure carries its message. + +#### Parameters + +##### box + +[`SurfaceReadBox`](#surfacereadbox) + +#### Returns + +[`SurfaceReader`](#surfacereader) + +*** + ### fsSurfaceReader() > **fsSurfaceReader**(`root`): [`SurfaceReader`](#surfacereader) diff --git a/docs/canonical-api.md b/docs/canonical-api.md index 3b124e3c..eb4f9289 100644 --- a/docs/canonical-api.md +++ b/docs/canonical-api.md @@ -158,7 +158,7 @@ A general "loop" primitive is the single most common modelling error in this rep | See a **supervised tree** in a trace viewer (one span per node, opened at spawn, closed at settle, parented to its parent node; driver turns as LLM child spans) | `supervise(profile, task, { otel: { exporter } })`, or `createSupervisorSpanRecorder({ runId, … }).hooks` on `SupervisorOpts.hooks`: `/kernel` — OPT-IN, and omitting `otel` installs no hook at all | parsing the spawn journal to reconstruct the tree; a second exporter; routing replay/resume through telemetry (the journal stays the only durable record) | | Run an ordered analyst pass where later analysts use findings from earlier analysts | `runAnalystLoop({ chainFindings: true })`: `/analyst-loop`; registration order defines the dependency order, while omission keeps analysts independent | manually invoke each analyst and pipe findings between calls | | Know **what got mounted into a run** / **why a candidate won** | `result.provenance.mounts` / `result.provenance.selectionReceipts` (`MountManifestEntry`/`SelectionReceipt`/`RunProvenance`); declare mounts via the `recordMount` recorder in `prepareBox`: root export | re-reading box contents to reconstruct what was mounted, or re-deriving which candidate the selector picked | -| Know **which of its own mounted surfaces an agent CHANGED** during a run (instructions/skills/memory self-edits — the observed, session-scoped evidence that may later become a measured proposal) | `harvestSurfaceDiffs({ mounts, read })` (+ `fsSurfaceReader` for local worktrees): `/kernel` | a per-harness refinement-event parser, or ad-hoc re-reads of box contents that silently skip unreadable surfaces | +| Know **which of its own mounted surfaces an agent CHANGED or CREATED** during a run (instructions/skills/memory self-edits — the observed, session-scoped evidence that may later become a measured proposal) | `harvestSurfaceDiffs({ mounts, read, watch? })` + `boxSurfaceReader` (box) / `fsSurfaceReader` (worktree): `/kernel` | a per-harness refinement-event parser, or ad-hoc re-reads of box contents that silently skip unreadable surfaces | | State any benchmark/A-B claim | `pairedLift(...)` (bench) over `pairedBootstrap`/`heldoutSignificance` (substrate) | your own bootstrap loop/PRNG per gate; a point lift without `low/high/pairs` | | Let an agent **delegate ONE generic INTENT** (no fixed coder/researcher type) and get the result + real spend SYNCHRONOUSLY | the **`delegate` tool**: `createDelegateHandler` via `createMcpServer({ delegateSupervisor })`; mount it over the `agent-runtime mcp` bin with `MCP_ENABLE_DELEGATE=1` (the bin authors a supervisor over a `sandbox` backend): `/mcp` | a hardcoded coder/researcher profile, or task-specific `delegate_code`/`delegate_research` verbs (RETIRED): `delegate` is the ONE delegation path and the only one with a cost channel | | Run a coding task INSIDE the agent's OWN sandbox session (a sibling box, fresh branch, validated patch) | `detachedSessionDelegate({ sandboxClient \| executor, workerProfile, fanoutProfiles? })`: `/mcp` (the required exact worker profile owns harness, provider, model, prompt, and tools; optional exact profiles make fanout heterogeneous) | a hardcoded coder profile baked into the delegate; `delegate()` (that spawns workers in a *chosen* backend, not the agent's own session) | diff --git a/docs/design/prime-agent-harness-integration.md b/docs/design/prime-agent-harness-integration.md index 3f73a1d1..5e630d5f 100644 --- a/docs/design/prime-agent-harness-integration.md +++ b/docs/design/prime-agent-harness-integration.md @@ -58,6 +58,15 @@ those surfaces at settle and reports what the agent itself changed CLAUDE.md/memory files, opencode edits AGENTS.md, Prime edits its continual harness — and today that evidence evaporates at box teardown. +The harvest is **caller-invoked at settle by the same caller that recorded the mounts** +(the kernel never reads workspace contents itself — same law as `recordMount`). Two +shipped readers join it to real substrates: `boxSurfaceReader` rides the same +`box.fs.read` seam `openSandboxRun` reads deliverables through; `fsSurfaceReader` +covers worktree/local workers. Surfaces the agent *created* (a new memory file — the +common `/refine` outcome) are covered by `watch` entries: the caller enumerates the +harness-state paths it cares about (e.g. via the box file tree) and never-mounted +paths that now exist report as `created`. + The scope law it feeds (Prime's `/refine` scopes, generalized to any harness): - **Session-scoped self-edits are observations.** They applied during the run, they are @@ -157,11 +166,24 @@ instrumented runs; 6–9 unlock the experiment tier. candidate and baseline harness state frozen and separated. 8. **Fan-out caps as config.** Prime's native child limit (and equivalents on other harnesses) as a materializable knob so an authored profile can bound native expansion. -9. **Whole-environment checkpoint/branch.** Box snapshot + branch behind the existing - `AgentEnvironment.checkpoint()/fork()` contract, covering workspace + harness state - (kernel heap where feasible). A transcript-tree branch is not a world branch; paired - baseline/candidate arms need same-state forks. Credentials stay short-lived and - injected — never serialized into snapshots, session JSONL, or harness state. +9. **Checkpoint/branch coverage for the Prime process tree.** The mechanism already + exists — `box.snapshot()` / `box.branch(count)` in the sandbox SDK, the kernel's + CRIU capability probe (`SandboxClient.criuStatus`) and fork lineage + (`LoopLineageOptions.forkFanout`), and the `AgentEnvironment.checkpoint()/fork()` + contract. The ask reduces to: **verify** a CRIU/box snapshot actually covers the + Prime daemon + IPython kernel process tree (a whole-box checkpoint should capture + kernel heap by construction — verify, don't assume), and report the answer through + the environment capability row. A transcript-tree branch is not a world branch; + paired baseline/candidate arms need same-state forks. Credentials stay short-lived + and injected — never serialized into snapshots, session JSONL, or harness state. + +**Adapter shape note.** Prime is daemon-backed with persistent sessions, which maps more +naturally onto the `AgentEnvironmentProvider` contract (sessions, capability +negotiation, `checkpoint`/`fork` — `src/runtime/environment-provider.ts`) than onto a +plain one-shot `backend.type`. Both integration shapes reach Runtime through existing +ports (`providerAsExecutor` / `providerAsSandboxClient` on one side, `buildBackendOptions` +on the other); the sandbox side should pick per tier — box backend for basic runs, +provider for instrumented/experiment tiers — rather than forcing one. ## 6. The first gated experiment (when 1–5, 7, 9 land) diff --git a/src/runtime/index.ts b/src/runtime/index.ts index d5095af7..a5722e39 100644 --- a/src/runtime/index.ts +++ b/src/runtime/index.ts @@ -945,12 +945,15 @@ export { superviseSurface, } from './supervise-surface' export { + boxSurfaceReader, fsSurfaceReader, type HarvestSurfaceDiffsOptions, harvestSurfaceDiffs, type SurfaceDiff, + type SurfaceReadBox, type SurfaceReader, type SurfaceReadOutcome, + type WatchedSurface, } from './surface-diff' export type { SandboxControlClient } from './tangle-sandbox-exact-process-provider' // Profile-owned supervisor configuration. The arbitrary model callback and raw driver constructor diff --git a/src/runtime/surface-diff.test.ts b/src/runtime/surface-diff.test.ts index ad8f29df..2425723d 100644 --- a/src/runtime/surface-diff.test.ts +++ b/src/runtime/surface-diff.test.ts @@ -3,7 +3,12 @@ import { mkdtempSync, rmSync, writeFileSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' import { afterAll, describe, expect, it } from 'vitest' -import { fsSurfaceReader, harvestSurfaceDiffs, type SurfaceReader } from './surface-diff' +import { + boxSurfaceReader, + fsSurfaceReader, + harvestSurfaceDiffs, + type SurfaceReader, +} from './surface-diff' import type { MountManifestEntry } from './types' const sha = (s: string) => createHash('sha256').update(s).digest('hex') @@ -79,6 +84,35 @@ describe('harvestSurfaceDiffs', () => { expect(diffs).toEqual([]) }) + it('reports a watched never-mounted path that now exists as created, and stays silent while absent', async () => { + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('CLAUDE.md', 'base')], + read: readerOf({ 'CLAUDE.md': 'base', 'memory/new-lesson.md': 'learned it' }), + watch: [ + { path: 'memory/new-lesson.md', source: 'harness-state' }, + { path: 'memory/still-absent.md' }, + ], + }) + expect(diffs).toEqual([ + { + path: 'memory/new-lesson.md', + status: 'created', + source: 'harness-state', + settledSha256: sha('learned it'), + settledBytes: Buffer.byteLength('learned it'), + }, + ]) + }) + + it('compares a watched path that was ALSO mounted against its mount, not as created', async () => { + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('a.md', 'v1')], + read: readerOf({ 'a.md': 'v2' }), + watch: [{ path: 'a.md' }], + }) + expect(diffs.map((d) => d.status)).toEqual(['modified']) + }) + it('treats an uppercase manifest hash as equal to the settled lowercase hash', async () => { const entry = { ...mount('a.md', 'same'), sha256: sha('same').toUpperCase() } const diffs = await harvestSurfaceDiffs({ @@ -89,6 +123,59 @@ describe('harvestSurfaceDiffs', () => { }) }) +describe('boxSurfaceReader', () => { + it('reads through box.fs.read, maps the SDK NotFoundError to missing, and reports other errors', async () => { + const notFound = new Error('no such file') + notFound.name = 'NotFoundError' + const box = { + fs: { + read: (path: string) => { + if (path === 'AGENTS.md') return Promise.resolve('edited') + if (path === 'gone.md') return Promise.reject(notFound) + return Promise.reject(new Error('transport down')) + }, + }, + } + const read = boxSurfaceReader(box) + const hit = await read('AGENTS.md') + expect(hit.succeeded).toBe(true) + if (hit.succeeded) expect(new TextDecoder().decode(hit.value)).toBe('edited') + expect(await read('gone.md')).toEqual({ + succeeded: false, + missing: true, + error: 'no such file', + }) + expect(await read('other.md')).toEqual({ + succeeded: false, + missing: false, + error: 'transport down', + }) + }) + + it('composes with the harvest over a box double: modified mount + created watch', async () => { + const box = { + fs: { + read: (path: string) => { + if (path === 'CLAUDE.md') return Promise.resolve('rewritten') + if (path === 'memory/lesson.md') return Promise.resolve('new note') + const err = new Error('missing') + err.name = 'NotFoundError' + return Promise.reject(err) + }, + }, + } + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('CLAUDE.md', 'original')], + read: boxSurfaceReader(box), + watch: [{ path: 'memory/lesson.md' }, { path: 'memory/other.md' }], + }) + expect(diffs.map((d) => [d.path, d.status])).toEqual([ + ['CLAUDE.md', 'modified'], + ['memory/lesson.md', 'created'], + ]) + }) +}) + describe('fsSurfaceReader', () => { const root = mkdtempSync(join(tmpdir(), 'surface-diff-')) afterAll(() => rmSync(root, { recursive: true, force: true })) diff --git a/src/runtime/surface-diff.ts b/src/runtime/surface-diff.ts index 648141f3..ddf91a87 100644 --- a/src/runtime/surface-diff.ts +++ b/src/runtime/surface-diff.ts @@ -39,45 +39,59 @@ export type SurfaceReadOutcome = export type SurfaceReader = (path: string) => Promise /** - * One mounted surface whose settled state differs from what was mounted. + * One watched surface whose settled state differs from what was mounted (or from absence). * * - `modified` — the surface exists with different bytes (`settledSha256`/`settledBytes` present). * - `removed` — the surface no longer exists at its mounted path. + * - `created` — a watched path that was never mounted now exists (`settledSha256`/`settledBytes` + * present, no `mountedSha256`) — the shape a harness's new memory/skill file takes. * - `unreadable` — the read seam failed for a reason other than absence; `error` carries the * diagnostic. Reported rather than dropped so a permissions or transport failure cannot * masquerade as "nothing changed". */ export interface SurfaceDiff { - /** The mounted path, exactly as recorded in the manifest entry. */ + /** The mounted/watched path, exactly as recorded. */ path: string - status: 'modified' | 'removed' | 'unreadable' - /** Hex SHA-256 of the bytes that were mounted (from the manifest). */ - mountedSha256: string - /** Free-form origin of the mounted resource, carried through from the manifest. */ + status: 'modified' | 'removed' | 'created' | 'unreadable' + /** Hex SHA-256 of the bytes that were mounted (from the manifest). Absent for `created`. */ + mountedSha256?: string + /** Free-form origin: the manifest entry's `source`, or the watch entry's `source`. */ source: string - /** Hex SHA-256 of the settled bytes. Present only for `modified`. */ + /** Hex SHA-256 of the settled bytes. Present for `modified` and `created`. */ settledSha256?: string - /** Size of the settled bytes. Present only for `modified`. */ + /** Size of the settled bytes. Present for `modified` and `created`. */ settledBytes?: number /** The read seam's diagnostic. Present only for `unreadable`. */ error?: string } +/** A path to check at settle that was NOT necessarily mounted — where a harness is known to write + * self-authored surfaces (a memory dir's files, a refinement log). A watched path that was also + * mounted compares against its mount; one that wasn't reports `created` if it now exists. */ +export interface WatchedSurface { + path: string + /** Origin label carried onto the diff (default `'watched'`). */ + source?: string +} + export interface HarvestSurfaceDiffsOptions { /** The run's mount manifest (`RunProvenance.mounts`). Entries sharing a path are collapsed to the * LAST entry — the bytes the agent actually saw at start. */ mounts: readonly MountManifestEntry[] /** How to read a mounted path's current bytes. */ read: SurfaceReader + /** Additional paths to check that may not have been mounted (see {@link WatchedSurface}). The + * caller enumerates them (it knows the harness's state layout — e.g. via the box's file tree); + * the harvest stays layout-agnostic. */ + watch?: readonly WatchedSurface[] } const sha256Hex = (bytes: Uint8Array): string => createHash('sha256').update(bytes).digest('hex') /** - * Re-read every mounted surface and report the ones whose settled state differs from the manifest. - * Unchanged surfaces produce no entry; output preserves manifest record order. Surfaces the agent - * CREATED (paths never mounted) are outside this contract — the manifest cannot see them; a - * substrate that enumerates harness-state directories can feed those paths in as additional mounts. + * Re-read every mounted (and watched) surface and report the ones whose settled state differs from + * the manifest — modified, removed, or created. Unchanged surfaces and still-absent watched paths + * produce no entry; output preserves record order, mounts before watch-only paths. */ export async function harvestSurfaceDiffs( options: HarvestSurfaceDiffsOptions, @@ -117,9 +131,58 @@ export async function harvestSurfaceDiffs( settledBytes: outcome.value.byteLength, }) } + for (const watched of options.watch ?? []) { + if (byPath.has(watched.path)) continue + const outcome = await options.read(watched.path) + if (!outcome.succeeded) { + // A still-absent watched path is the expected no-op; a failing read is not. + if (!outcome.missing) + diffs.push({ + path: watched.path, + status: 'unreadable', + source: watched.source ?? 'watched', + error: outcome.error, + }) + continue + } + diffs.push({ + path: watched.path, + status: 'created', + source: watched.source ?? 'watched', + settledSha256: sha256Hex(outcome.value), + settledBytes: outcome.value.byteLength, + }) + } return diffs } +/** The minimal box surface the box-backed reader needs — structurally typed so the real + * `@tangle-network/sandbox` box and a test double both satisfy it, no SDK import. */ +export interface SurfaceReadBox { + fs: { read(path: string): Promise } +} + +/** + * A {@link SurfaceReader} over a sandbox box's filesystem — the same `box.fs.read` seam + * `openSandboxRun` reads deliverables through. The box wire returns UTF-8 text, so this reader + * covers TEXT surfaces (which profile surfaces are); hashes are computed over the UTF-8 encoding. + * The SDK's not-found error is detected structurally (`err.name === 'NotFoundError'`) and maps to + * `missing: true`; every other failure carries its message. + */ +export function boxSurfaceReader(box: SurfaceReadBox): SurfaceReader { + return async (path) => { + try { + return { succeeded: true, value: new TextEncoder().encode(await box.fs.read(path)) } + } catch (err) { + return { + succeeded: false, + missing: err instanceof Error && err.name === 'NotFoundError', + error: err instanceof Error ? err.message : String(err), + } + } + } +} + /** * A {@link SurfaceReader} over the local filesystem, for worktree/local workers. Relative mount * paths resolve against `root`. Absence maps to `missing: true`; every other failure carries the From 5fc802a29a63793fd962d11dcf1163135c30d529 Mon Sep 17 00:00:00 2001 From: drewstone Date: Tue, 11 Aug 2026 20:35:58 +0000 Subject: [PATCH 03/10] fix(kernel): harden the surface harvest against transport lies and path-form drift Review findings applied: boxSurfaceReader retries transient first-attempt 404s (the same write-flush blip openSandboxRun's deliverable read retries for) so a fresh self-edit is not reported missing; a NotFoundError whose resourceType names a non-file resource (the box or session being gone) is a transport failure, never 'every surface was removed'; lossy-decoded content (U+FFFD) is reported unreadable instead of hashed as a false modification. harvestSurfaceDiffs normalizes './'-prefixed paths for mount/watch collision (a mount-recorder form vs a file-tree form is one surface), dedupes watch entries by path, and runs all reads concurrently while preserving record order. --- docs/api/primitive-catalog.md | 4 +- docs/api/runtime.md | 42 ++++++-- src/runtime/index.ts | 1 + src/runtime/surface-diff.test.ts | 61 +++++++++++- src/runtime/surface-diff.ts | 164 +++++++++++++++++++++---------- 5 files changed, 209 insertions(+), 63 deletions(-) diff --git a/docs/api/primitive-catalog.md b/docs/api/primitive-catalog.md index 15b487a5..3ceaa832 100644 --- a/docs/api/primitive-catalog.md +++ b/docs/api/primitive-catalog.md @@ -504,7 +504,7 @@ Import from `@tangle-network/agent-runtime/intelligence` — 166 exports. ### Execution kernel — recursive atom, supervision, executors, round-synchronous loop -Import from `@tangle-network/agent-runtime/kernel` — 718 exports. +Import from `@tangle-network/agent-runtime/kernel` — 719 exports. | Symbol | Kind | Summary | |---|---|---| @@ -1078,7 +1078,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 718 exports. | `WorktreeCheckRunner` | type | The single shell-command-in-worktree runner seam (replaces the per-executor copies). | | `WorktreePatchArtifact` | type | Terminal artifact of one worktree-CLI run — the canonical worktree-harness result (the captured | -**Undocumented supporting types** (add a TSDoc line at the declaration to earn a table row): `AcquireOptions`, `AgentEnvironment`, `AgentEnvironmentCapabilities`, `AgentEnvironmentEvent`, `AgentEnvironmentProvider`, `AgentEnvironmentQuery`, `AgentEnvironmentSummary`, `AgentGraph`, `AgenticOptions`, `AgenticRunResult`, `AgenticTask`, `AgenticTool`, `AgentSession`, `AgentSessionRef`, `AgentTurnResult`, `AllWorkersStalledOptions`, `AnalystRegistry`, `AnytimeReport`, `AnytimeStrategySummary`, `AnytimeTaskCurve`, `ArtifactHandle`, `AuditIntentInput`, `AuditIntentOptions`, `AuthoredHarness`, `AuthoredStrategy`, `AuthorStrategyOptions`, `BenchmarkConfig`, `BenchmarkLift`, `BenchmarkStrategySummary`, `BenchmarkTaskRow`, `BudgetPool`, `BusStats`, `ChampionPick`, `CheckpointRef`, `CheckpointRequest`, `CheckRunContext`, `CliWorktreeBridgeSeam`, `CoordinationMcpHandle`, `CopyOptions`, `CorpusReadbackOptions`, `CreateAgentEnvironmentInput`, `CreateTangleSandboxExactProcessProviderOptions`, `DefinedLeaderboard`, `DispatchReport`, `Driver`, `EvolutionArchiveNode`, `EvolutionAuthor`, `EvolutionBandInfo`, `EvolutionCandidate`, `EvolutionGeneration`, `EvolutionReport`, `ExecRequest`, `ExecResult`, `ExecutorResultMapping`, `ForkRequest`, `GitWorkspaceOptions`, `GraphResult`, `HarvestCorpusOptions`, `HarvestFailure`, `HarvestReport`, `HarvestSurfaceDiffsOptions`, `Inbox`, `InProcessSandboxClientOptions`, `IntentAudit`, `Iteration`, `Leaderboard`, `LeaderboardOptions`, `LocalSandboxClientOptions`, `LoopDecisionPayload`, `LoopDispatchOptions`, `LoopEndedPayload`, `LoopIterationEndedPayload`, `LoopIterationStartedPayload`, `LoopPlanDescription`, `LoopResult`, `LoopSandboxPlacement`, `LoopStartedPayload`, `LoopTraceEmitter`, `LoopWinner`, `MaterializeLocalMcpOptions`, `McpEnvironmentOptions`, `McpToolDescriptor`, `NodeSnapshot`, `NoProgressForOptions`, `Observation`, `ObserveInput`, `ObserveOptions`, `OpenSandboxRunOptions`, `PairwiseOptions`, `PatchDeliverableOptions`, `PlacementInfo`, `PlateauOptions`, `ProgressTrackerOptions`, `PromotionGateOptions`, `PromotionVerdict`, `PublishOptions`, `ReproductionCheck`, `ResolveSandboxClientOptions`, `ResourceRequest`, `RollingDispatchOptions`, `RunAgenticOptions`, `RunAgentRoundsOptions`, `RunGraphOptions`, `SandboxRun`, `ShotSpec`, `SpawnOpts`, `StdioMcpConnection`, `StdioMcpServerSpec`, `SteerableSandboxArgs`, `Strategy`, `StrategyEvolutionConfig`, `StrategyResult`, `StreamAgentTurnOptions`, `StructuralRolloutConfig`, `SuperviseOptions`, `SuperviseSurfaceOptions`, `SupervisorAgentDeps`, `SupervisorOpts`, `SupervisorSpanOptions`, `SupervisorSpanRecorder`, `SurfaceScore`, `ToolSpec`, `ToolStepInput`, `TraceSource`, `TrajectoryAnalysis`, `UntrackedCopyStats`, `ValidationCtx`, `Validator`, `VerifierEnvironmentOptions`, `WatchTraceOptions`, `WaterfallCollector`, `WaterfallReport`, `WaterfallSpan`, `WorkerEvidenceInput`, `Workspace`, `WorkspaceRequest`, `WorkspaceRun`, `WorktreeCliExecutorOptions`, `WorktreeFanoutOptions`, `AgentEnvironmentStatus`, `AgentSessionStatus`, `ChampionPolicy`, `EdgeDeliveryOutcome`, `GraphEdge`, `LoopTraceEvent`, `MakeWorkerAgent`, `RepairStop`, `SandboxControlClient`, `WorkspaceCommit`. +**Undocumented supporting types** (add a TSDoc line at the declaration to earn a table row): `AcquireOptions`, `AgentEnvironment`, `AgentEnvironmentCapabilities`, `AgentEnvironmentEvent`, `AgentEnvironmentProvider`, `AgentEnvironmentQuery`, `AgentEnvironmentSummary`, `AgentGraph`, `AgenticOptions`, `AgenticRunResult`, `AgenticTask`, `AgenticTool`, `AgentSession`, `AgentSessionRef`, `AgentTurnResult`, `AllWorkersStalledOptions`, `AnalystRegistry`, `AnytimeReport`, `AnytimeStrategySummary`, `AnytimeTaskCurve`, `ArtifactHandle`, `AuditIntentInput`, `AuditIntentOptions`, `AuthoredHarness`, `AuthoredStrategy`, `AuthorStrategyOptions`, `BenchmarkConfig`, `BenchmarkLift`, `BenchmarkStrategySummary`, `BenchmarkTaskRow`, `BoxSurfaceReaderOptions`, `BudgetPool`, `BusStats`, `ChampionPick`, `CheckpointRef`, `CheckpointRequest`, `CheckRunContext`, `CliWorktreeBridgeSeam`, `CoordinationMcpHandle`, `CopyOptions`, `CorpusReadbackOptions`, `CreateAgentEnvironmentInput`, `CreateTangleSandboxExactProcessProviderOptions`, `DefinedLeaderboard`, `DispatchReport`, `Driver`, `EvolutionArchiveNode`, `EvolutionAuthor`, `EvolutionBandInfo`, `EvolutionCandidate`, `EvolutionGeneration`, `EvolutionReport`, `ExecRequest`, `ExecResult`, `ExecutorResultMapping`, `ForkRequest`, `GitWorkspaceOptions`, `GraphResult`, `HarvestCorpusOptions`, `HarvestFailure`, `HarvestReport`, `HarvestSurfaceDiffsOptions`, `Inbox`, `InProcessSandboxClientOptions`, `IntentAudit`, `Iteration`, `Leaderboard`, `LeaderboardOptions`, `LocalSandboxClientOptions`, `LoopDecisionPayload`, `LoopDispatchOptions`, `LoopEndedPayload`, `LoopIterationEndedPayload`, `LoopIterationStartedPayload`, `LoopPlanDescription`, `LoopResult`, `LoopSandboxPlacement`, `LoopStartedPayload`, `LoopTraceEmitter`, `LoopWinner`, `MaterializeLocalMcpOptions`, `McpEnvironmentOptions`, `McpToolDescriptor`, `NodeSnapshot`, `NoProgressForOptions`, `Observation`, `ObserveInput`, `ObserveOptions`, `OpenSandboxRunOptions`, `PairwiseOptions`, `PatchDeliverableOptions`, `PlacementInfo`, `PlateauOptions`, `ProgressTrackerOptions`, `PromotionGateOptions`, `PromotionVerdict`, `PublishOptions`, `ReproductionCheck`, `ResolveSandboxClientOptions`, `ResourceRequest`, `RollingDispatchOptions`, `RunAgenticOptions`, `RunAgentRoundsOptions`, `RunGraphOptions`, `SandboxRun`, `ShotSpec`, `SpawnOpts`, `StdioMcpConnection`, `StdioMcpServerSpec`, `SteerableSandboxArgs`, `Strategy`, `StrategyEvolutionConfig`, `StrategyResult`, `StreamAgentTurnOptions`, `StructuralRolloutConfig`, `SuperviseOptions`, `SuperviseSurfaceOptions`, `SupervisorAgentDeps`, `SupervisorOpts`, `SupervisorSpanOptions`, `SupervisorSpanRecorder`, `SurfaceScore`, `ToolSpec`, `ToolStepInput`, `TraceSource`, `TrajectoryAnalysis`, `UntrackedCopyStats`, `ValidationCtx`, `Validator`, `VerifierEnvironmentOptions`, `WatchTraceOptions`, `WaterfallCollector`, `WaterfallReport`, `WaterfallSpan`, `WorkerEvidenceInput`, `Workspace`, `WorkspaceRequest`, `WorkspaceRun`, `WorktreeCliExecutorOptions`, `WorktreeFanoutOptions`, `AgentEnvironmentStatus`, `AgentSessionStatus`, `ChampionPolicy`, `EdgeDeliveryOutcome`, `GraphEdge`, `LoopTraceEvent`, `MakeWorkerAgent`, `RepairStop`, `SandboxControlClient`, `WorkspaceCommit`. ### Environment provider adapters — generic sandbox/compute bridge diff --git a/docs/api/runtime.md b/docs/api/runtime.md index 55975ec5..3d5dfa96 100644 --- a/docs/api/runtime.md +++ b/docs/api/runtime.md @@ -16155,6 +16155,27 @@ The minimal box surface the box-backed reader needs — structurally typed so th *** +### BoxSurfaceReaderOptions + +#### Properties + +##### attempts? + +> `optional` **attempts?**: `number` + +Read attempts per path before settling on a failed outcome. The data plane can transiently + 404 a just-written file (the same blip `openSandboxRun`'s deliverable read retries for), and a + first-attempt 404 taken at face value turns a fresh self-edit into a false `removed`/dropped + `created`. Default 3. + +##### retryDelayMs? + +> `optional` **retryDelayMs?**: `number` + +Linear backoff base between attempts (delay = base × attempt). Default 250. + +*** + ### CreateTangleSandboxExactProcessProviderOptions #### Properties @@ -25094,7 +25115,9 @@ the shared valid-only `selectValidWinner` (never a judge). Re-read every mounted (and watched) surface and report the ones whose settled state differs from the manifest — modified, removed, or created. Unchanged surfaces and still-absent watched paths -produce no entry; output preserves record order, mounts before watch-only paths. +produce no entry; reads run concurrently; output preserves record order, mounts before +watch-only paths. Mounts and watches sharing a path key are each collapsed to the LAST entry, +and a watched path that was also mounted compares against its mount (never reports `created`). #### Parameters @@ -25110,13 +25133,16 @@ produce no entry; output preserves record order, mounts before watch-only paths. ### boxSurfaceReader() -> **boxSurfaceReader**(`box`): [`SurfaceReader`](#surfacereader) +> **boxSurfaceReader**(`box`, `options?`): [`SurfaceReader`](#surfacereader) A [SurfaceReader](#surfacereader) over a sandbox box's filesystem — the same `box.fs.read` seam -`openSandboxRun` reads deliverables through. The box wire returns UTF-8 text, so this reader -covers TEXT surfaces (which profile surfaces are); hashes are computed over the UTF-8 encoding. -The SDK's not-found error is detected structurally (`err.name === 'NotFoundError'`) and maps to -`missing: true`; every other failure carries its message. +`openSandboxRun` reads deliverables through, with the same transient-404 posture (bounded +retry). The box wire returns UTF-8 TEXT (the SDK's binary path is `download()`), which profile +surfaces are; hashes are computed over the UTF-8 encoding, and content the wire had to +lossy-decode (a U+FFFD replacement character) is reported `unreadable` rather than hashed as +mojibake. The SDK's not-found error is detected structurally (`err.name === 'NotFoundError'`) +and maps to `missing: true` — unless its `resourceType` names something other than a file/path +(the BOX or session being gone), which is a transport failure, not an absent surface. #### Parameters @@ -25124,6 +25150,10 @@ The SDK's not-found error is detected structurally (`err.name === 'NotFoundError [`SurfaceReadBox`](#surfacereadbox) +##### options? + +[`BoxSurfaceReaderOptions`](#boxsurfacereaderoptions) = `{}` + #### Returns [`SurfaceReader`](#surfacereader) diff --git a/src/runtime/index.ts b/src/runtime/index.ts index a5722e39..c14be0a8 100644 --- a/src/runtime/index.ts +++ b/src/runtime/index.ts @@ -945,6 +945,7 @@ export { superviseSurface, } from './supervise-surface' export { + type BoxSurfaceReaderOptions, boxSurfaceReader, fsSurfaceReader, type HarvestSurfaceDiffsOptions, diff --git a/src/runtime/surface-diff.test.ts b/src/runtime/surface-diff.test.ts index 2425723d..534a9aa9 100644 --- a/src/runtime/surface-diff.test.ts +++ b/src/runtime/surface-diff.test.ts @@ -113,6 +113,27 @@ describe('harvestSurfaceDiffs', () => { expect(diffs.map((d) => d.status)).toEqual(['modified']) }) + it("treats a './'-prefixed mount and a bare watched path as one surface, and dedupes duplicate watches", async () => { + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('./AGENTS.md', 'given')], + read: readerOf({ './AGENTS.md': 'given', 'memory/x.md': 'note' }), + watch: [ + { path: 'AGENTS.md' }, + { path: 'memory/x.md' }, + { path: 'memory/x.md', source: 'tree' }, + ], + }) + expect(diffs).toEqual([ + { + path: 'memory/x.md', + status: 'created', + source: 'tree', + settledSha256: sha('note'), + settledBytes: Buffer.byteLength('note'), + }, + ]) + }) + it('treats an uppercase manifest hash as equal to the settled lowercase hash', async () => { const entry = { ...mount('a.md', 'same'), sha256: sha('same').toUpperCase() } const diffs = await harvestSurfaceDiffs({ @@ -136,7 +157,7 @@ describe('boxSurfaceReader', () => { }, }, } - const read = boxSurfaceReader(box) + const read = boxSurfaceReader(box, { retryDelayMs: 0 }) const hit = await read('AGENTS.md') expect(hit.succeeded).toBe(true) if (hit.succeeded) expect(new TextDecoder().decode(hit.value)).toBe('edited') @@ -152,6 +173,44 @@ describe('boxSurfaceReader', () => { }) }) + it('retries a transient first-attempt 404 instead of reporting a fresh write as missing', async () => { + const blip = new Error('not flushed yet') + blip.name = 'NotFoundError' + let calls = 0 + const box = { + fs: { + read: () => { + calls += 1 + return calls < 3 ? Promise.reject(blip) : Promise.resolve('finally visible') + }, + }, + } + const outcome = await boxSurfaceReader(box, { retryDelayMs: 0 })('memory/new.md') + expect(calls).toBe(3) + expect(outcome.succeeded).toBe(true) + }) + + it('reports a box-level NotFoundError (resourceType names the sandbox) as unreadable, never missing', async () => { + const boxGone = Object.assign(new Error('sandbox sb-1 not found'), { + name: 'NotFoundError', + resourceType: 'Sandbox', + resourceId: 'sb-1', + }) + const box = { fs: { read: () => Promise.reject(boxGone) } } + expect(await boxSurfaceReader(box, { attempts: 1 })('CLAUDE.md')).toEqual({ + succeeded: false, + missing: false, + error: 'sandbox sb-1 not found', + }) + }) + + it('refuses to hash content the text wire lossy-decoded instead of reporting a false modification', async () => { + const box = { fs: { read: () => Promise.resolve('binary�garbage') } } + const outcome = await boxSurfaceReader(box, { attempts: 1 })('memory/store.db') + expect(outcome).toMatchObject({ succeeded: false, missing: false }) + if (!outcome.succeeded) expect(outcome.error).toContain('not valid UTF-8') + }) + it('composes with the harvest over a box double: modified mount + created watch', async () => { const box = { fs: { diff --git a/src/runtime/surface-diff.ts b/src/runtime/surface-diff.ts index ddf91a87..a19270c4 100644 --- a/src/runtime/surface-diff.ts +++ b/src/runtime/surface-diff.ts @@ -88,22 +88,33 @@ export interface HarvestSurfaceDiffsOptions { const sha256Hex = (bytes: Uint8Array): string => createHash('sha256').update(bytes).digest('hex') +/** The collision/dedup key for a path: a leading `./` stripped, so `./AGENTS.md` (a mount + * recorder's form) and `AGENTS.md` (a file-tree enumeration's form) are one surface. Absolute vs + * relative cannot be reconciled here — keep those forms consistent between mounts and watches. */ +const pathKey = (p: string): string => p.replace(/^\.\//, '') + /** * Re-read every mounted (and watched) surface and report the ones whose settled state differs from * the manifest — modified, removed, or created. Unchanged surfaces and still-absent watched paths - * produce no entry; output preserves record order, mounts before watch-only paths. + * produce no entry; reads run concurrently; output preserves record order, mounts before + * watch-only paths. Mounts and watches sharing a path key are each collapsed to the LAST entry, + * and a watched path that was also mounted compares against its mount (never reports `created`). */ export async function harvestSurfaceDiffs( options: HarvestSurfaceDiffsOptions, ): Promise { const byPath = new Map() - for (const entry of options.mounts) byPath.set(entry.path, entry) - const diffs: SurfaceDiff[] = [] - for (const entry of byPath.values()) { - const outcome = await options.read(entry.path) - if (!outcome.succeeded) { - diffs.push( - outcome.missing + for (const entry of options.mounts) byPath.set(pathKey(entry.path), entry) + const watchByPath = new Map() + for (const watched of options.watch ?? []) { + if (byPath.has(pathKey(watched.path))) continue + watchByPath.set(pathKey(watched.path), watched) + } + const mountDiffs = await Promise.all( + [...byPath.values()].map(async (entry): Promise => { + const outcome = await options.read(entry.path) + if (!outcome.succeeded) { + return outcome.missing ? { path: entry.path, status: 'removed', @@ -116,44 +127,43 @@ export async function harvestSurfaceDiffs( mountedSha256: entry.sha256, source: entry.source, error: outcome.error, - }, - ) - continue - } - const settledSha256 = sha256Hex(outcome.value) - if (settledSha256 === entry.sha256.toLowerCase()) continue - diffs.push({ - path: entry.path, - status: 'modified', - mountedSha256: entry.sha256, - source: entry.source, - settledSha256, - settledBytes: outcome.value.byteLength, - }) - } - for (const watched of options.watch ?? []) { - if (byPath.has(watched.path)) continue - const outcome = await options.read(watched.path) - if (!outcome.succeeded) { - // A still-absent watched path is the expected no-op; a failing read is not. - if (!outcome.missing) - diffs.push({ + } + } + const settledSha256 = sha256Hex(outcome.value) + if (settledSha256 === entry.sha256.toLowerCase()) return undefined + return { + path: entry.path, + status: 'modified', + mountedSha256: entry.sha256, + source: entry.source, + settledSha256, + settledBytes: outcome.value.byteLength, + } + }), + ) + const watchDiffs = await Promise.all( + [...watchByPath.values()].map(async (watched): Promise => { + const outcome = await options.read(watched.path) + if (!outcome.succeeded) { + // A still-absent watched path is the expected no-op; a failing read is not. + if (outcome.missing) return undefined + return { path: watched.path, status: 'unreadable', source: watched.source ?? 'watched', error: outcome.error, - }) - continue - } - diffs.push({ - path: watched.path, - status: 'created', - source: watched.source ?? 'watched', - settledSha256: sha256Hex(outcome.value), - settledBytes: outcome.value.byteLength, - }) - } - return diffs + } + } + return { + path: watched.path, + status: 'created', + source: watched.source ?? 'watched', + settledSha256: sha256Hex(outcome.value), + settledBytes: outcome.value.byteLength, + } + }), + ) + return [...mountDiffs, ...watchDiffs].filter((d): d is SurfaceDiff => d !== undefined) } /** The minimal box surface the box-backed reader needs — structurally typed so the real @@ -162,24 +172,70 @@ export interface SurfaceReadBox { fs: { read(path: string): Promise } } +export interface BoxSurfaceReaderOptions { + /** Read attempts per path before settling on a failed outcome. The data plane can transiently + * 404 a just-written file (the same blip `openSandboxRun`'s deliverable read retries for), and a + * first-attempt 404 taken at face value turns a fresh self-edit into a false `removed`/dropped + * `created`. Default 3. */ + attempts?: number + /** Linear backoff base between attempts (delay = base × attempt). Default 250. */ + retryDelayMs?: number +} + +const sleep = (ms: number) => new Promise((res) => setTimeout(res, ms)) + /** * A {@link SurfaceReader} over a sandbox box's filesystem — the same `box.fs.read` seam - * `openSandboxRun` reads deliverables through. The box wire returns UTF-8 text, so this reader - * covers TEXT surfaces (which profile surfaces are); hashes are computed over the UTF-8 encoding. - * The SDK's not-found error is detected structurally (`err.name === 'NotFoundError'`) and maps to - * `missing: true`; every other failure carries its message. + * `openSandboxRun` reads deliverables through, with the same transient-404 posture (bounded + * retry). The box wire returns UTF-8 TEXT (the SDK's binary path is `download()`), which profile + * surfaces are; hashes are computed over the UTF-8 encoding, and content the wire had to + * lossy-decode (a U+FFFD replacement character) is reported `unreadable` rather than hashed as + * mojibake. The SDK's not-found error is detected structurally (`err.name === 'NotFoundError'`) + * and maps to `missing: true` — unless its `resourceType` names something other than a file/path + * (the BOX or session being gone), which is a transport failure, not an absent surface. */ -export function boxSurfaceReader(box: SurfaceReadBox): SurfaceReader { +export function boxSurfaceReader( + box: SurfaceReadBox, + options: BoxSurfaceReaderOptions = {}, +): SurfaceReader { + const attempts = options.attempts ?? 3 + const retryDelayMs = options.retryDelayMs ?? 250 return async (path) => { - try { - return { succeeded: true, value: new TextEncoder().encode(await box.fs.read(path)) } - } catch (err) { - return { - succeeded: false, - missing: err instanceof Error && err.name === 'NotFoundError', - error: err instanceof Error ? err.message : String(err), + let last: SurfaceReadOutcome = { + succeeded: false, + missing: false, + error: 'boxSurfaceReader: no read attempted', + } + for (let attempt = 1; attempt <= attempts; attempt += 1) { + try { + const text = await box.fs.read(path) + if (text.includes('�')) { + return { + succeeded: false, + missing: false, + error: `boxSurfaceReader: content at ${JSON.stringify(path)} is not valid UTF-8 text (the box text wire lossy-decoded it); binary surfaces need a byte-faithful reader`, + } + } + return { succeeded: true, value: new TextEncoder().encode(text) } + } catch (err) { + const notFound = err instanceof Error && err.name === 'NotFoundError' + const resourceType = + err && typeof err === 'object' && 'resourceType' in err + ? String((err as { resourceType: unknown }).resourceType) + : undefined + // A NotFoundError naming a non-file resource (Sandbox / session) means the TRANSPORT is + // gone — reporting it as `missing` would read as "every surface was removed". + const fileNotFound = + notFound && (resourceType === undefined || /file|path/i.test(resourceType)) + last = { + succeeded: false, + missing: fileNotFound, + error: err instanceof Error ? err.message : String(err), + } + if (attempt < attempts && retryDelayMs > 0) await sleep(retryDelayMs * attempt) } } + return last } } From 36ec6ec1bf597c34971986f1676e9f93842a19c2 Mon Sep 17 00:00:00 2001 From: drewstone Date: Tue, 11 Aug 2026 21:45:10 +0000 Subject: [PATCH 04/10] feat(kernel): wire the shipped prime harness id; apply PR review findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit agent-interface 0.46.x shipped the id as 'prime' (a distinct harness from pi — the fork's daemon rejects pi-line clients) and the sandbox backend enum accepts it, so 'prime' joins harnessBackends and the docs drop the provisional 'prime-agent' name. The decoder entry stays gated on validated wire output; the compat column stays gated on measurement. Review findings applied: a reader that throws is contained to an unreadable diff for that path instead of rejecting the whole harvest; fsSurfaceReader refuses paths that resolve outside its root (an escaping manifest path was an existence/hash oracle over the host filesystem); HarvestSurfaceDiffsOptions gains its declaration summary; 'created' is documented as a mount-manifest inference, not proof of authorship; the anti-reinvention table's holdout-gate and matrix rows point at the files that actually hold the symbols; the canonical-api version pin matches package.json at 0.132.2. --- docs/api/primitive-catalog.md | 15 ++++-- docs/api/runtime.md | 16 +++++-- docs/canonical-api.md | 2 +- .../design/prime-agent-harness-integration.md | 38 ++++++++------- docs/research/harness-compat.md | 11 +++-- src/runtime/sandbox-backend.ts | 1 + src/runtime/surface-diff.test.ts | 27 +++++++++++ src/runtime/surface-diff.ts | 48 +++++++++++++++---- 8 files changed, 122 insertions(+), 36 deletions(-) diff --git a/docs/api/primitive-catalog.md b/docs/api/primitive-catalog.md index cf8a10e0..81e973cb 100644 --- a/docs/api/primitive-catalog.md +++ b/docs/api/primitive-catalog.md @@ -7,7 +7,7 @@ # Primitive catalog — the never-stale anti-reinvention inventory -> **GENERATED** from `@tangle-network/agent-runtime@0.132.1` and `@tangle-network/agent-eval@0.144.12` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`. +> **GENERATED** from `@tangle-network/agent-runtime@0.132.2` and `@tangle-network/agent-eval@0.144.12` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`. ## 1. agent-runtime — own public surface @@ -514,7 +514,7 @@ Import from `@tangle-network/agent-runtime/intelligence` — 166 exports. ### Execution kernel — recursive atom, supervision, executors, round-synchronous loop -Import from `@tangle-network/agent-runtime/kernel` — 734 exports. +Import from `@tangle-network/agent-runtime/kernel` — 744 exports. | Symbol | Kind | Summary | |---|---|---| @@ -534,6 +534,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 734 exports. | `auditIntent` | function | The route-rigor analyst: compare declared vs revealed vs user intent over a trajectory and return aligned / drifting / diverged with evidence and one recommended intervention. | | `authorStrategy` | function | Author + load a strategy from losses. Throws when the author emits no loadable module; | | `bestSoFar` | function | The best-so-far fold — the ONE definition of "how good was the run after k results", shared by | +| `boxSurfaceReader` | function | A {@link SurfaceReader} over a sandbox box's filesystem — the same `box.fs.read` seam | | `breadthStrategy` | function | BREADTH: K independent rollouts (each own artifact), verifier picks the best. | | `buildSteerContext` | function | Build the `SteerContext` a combinator reads to steer (its `loopUntil.until`, `widen` gate, any | | `canDisplace` | function | The repair keep-best guard: a challenger displaces the incumbent only when it is | @@ -604,9 +605,11 @@ Import from `@tangle-network/agent-runtime/kernel` — 734 exports. | `flatWidenGate` | function | The flat default `ScopeWidenGate` — never widens, keeping the R2 selector≠judge collision | | `formatPromptHandle` | function | The string form of a handle: `/v`. | | `freeSlots` | function | Free worker slots under a simultaneity cap: `cap - live`, floored at 0, or `null` when there is | +| `fsSurfaceReader` | function | A {@link SurfaceReader} over the local filesystem, for worktree/local workers. Every path — | | `gateOnDeliverable` | function | Wrap an `Executor` so its settlement `valid` reflects the deliverable check, not the | | `gitWorkspace` | function | A `Workspace` over a git checkout: materialize an isolated worktree at `ref`, commit produced changes (conflict-aware), and read `head` — hooks disabled, identity pinned. | | `harvestCorpus` | function | Batch the firewalled `observe()` analyst over completed runs and accrete the trace-derived facts into the durable corpus — the production-traces→corpus write side of the flywheel. | +| `harvestSurfaceDiffs` | function | Re-read every mounted (and watched) surface and report the ones whose settled state differs from | | `inlineSandboxClient` | function | Adapt an `ExecutorFactory` into a `SandboxClient` for `runAgentRounds`. The factory is | | `inProcessSandboxClient` | function | Adapt a single `onPrompt(prompt, ctx)` callback into a `SandboxClient` for | | `isWaitOutcome` | function | Narrow a settlement's `out` to a wait outcome — a wait settles on the SAME cursor as workers, | @@ -860,6 +863,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 734 exports. | `ForkCapableBox` | interface | Loop-side widening of the box's optional fork method. | | `GraphNode` | interface | A graph node: an id and a canonical `AgentProfile`. The profile is the ONLY way a node is | | `Handle` | interface | A live child handle. `abort()` is defined over the ACQUIRE lifecycle: it chains into | +| `HarvestSurfaceDiffsOptions` | interface | Inputs to {@link harvestSurfaceDiffs}: the run's mount manifest, the read seam, and optional | | `InboxMessage` | interface | The worker-side receive end of the down-leg: a per-worker inbox an executor exposes as | | `InMemoryRunContext` | interface | The bundle of stores a supervised run needs, shaped to spread into `SupervisorOpts`. | | `InMemoryRunContextOptions` | interface | Options for a supervised run context. | @@ -983,6 +987,8 @@ Import from `@tangle-network/agent-runtime/kernel` — 734 exports. | `SupervisorSpanOutcome` | interface | How the supervised run ended, as `finish()` records it on the root span. | | `SupervisorToolDescriptor` | interface | One product-owned tool. It reuses the canonical MCP descriptor fields while Runtime supplies | | `SupervisorToolInvocationContext` | interface | Trusted context for one product-tool invocation. The node identity remains the same detached, | +| `SurfaceDiff` | interface | One watched surface whose settled state differs from what was mounted (or from absence). | +| `SurfaceReadBox` | interface | The minimal box surface the box-backed reader needs — structurally typed so the real | | `SurfaceWorkerConfig` | interface | How a worker runs the surface task (its router substrate + per-attempt bounds). | | `SurfaceWorkerOut` | interface | What a surface worker settles with — the surface verdict the driver + deliverable read. `resolved` is | | `ToolLoopCompaction` | interface | Self-compaction — bound the loop's OWN context window the way a fresh-respawn (dumb-Ralph) loop | @@ -997,6 +1003,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 734 exports. | `WaitOpts` | interface | Options for `Scope.wait`. `label` is the wait's identity within its parent scope — it is what | | `WaitOutcome` | interface | The `out` a settled wait node delivers through `Scope.next()`. `settled` is the outcome the | | `WaitProbeRegistry` | interface | Resolves a `poll` spec's `probe` name to its predicate. Threaded through `SupervisorOpts` so | +| `WatchedSurface` | interface | A path to check at settle that was NOT necessarily mounted — where a harness is known to write | | `WidenGate` | interface | The progressive-widening gate (MCTS-PW). Decides whether a settled child is | | `WidenLineage` | interface | A lineage the gate may widen toward — the settled child that looked promising + the findings | | `WidenSpec` | interface | `widen({ gate })` (G5) — the STREAMING spawn-on-completion driver. Unlike the static-fanout | @@ -1086,6 +1093,8 @@ Import from `@tangle-network/agent-runtime/kernel` — 734 exports. | `SupervisorNodeContextSeed` | type | Context known before `Agent.act`; Runtime adds the concrete node, profile, and task. | | `SupervisorProfile` | type | A supervisor is an exact canonical AgentProfile; no looser model/prompt shape exists. | | `SupervisorSpanAttributes` | type | OTLP span attribute values. Exported because `SupervisorSpanOptions.attributes` is public and | +| `SurfaceReader` | type | The read seam: fetch the current bytes at a mounted path. Implemented by a sandbox box's | +| `SurfaceReadOutcome` | type | Outcome of reading one surface back at settle. `missing: true` means the path no longer exists | | `ToolLoopCompactionOptions` | type | Public supervisor-facing compaction config: same knobs as the primitive, but `distill` is optional | | `ToolLoopMessageRecord` | type | Provider-neutral conversation record accepted by a tool-loop brain. | | `TrajectoryReportFn` | type | `trajectoryReport(...)` — the tree+cost reconstructor. Async (reads journal + optionally blobs). | @@ -1105,7 +1114,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 734 exports. | `WorktreeCheckRunner` | type | The single shell-command-in-worktree runner seam (replaces the per-executor copies). | | `WorktreePatchArtifact` | type | Terminal artifact of one worktree-CLI run — the canonical worktree-harness result (the captured | -**Undocumented supporting types** (add a TSDoc line at the declaration to earn a table row): `AcquireOptions`, `AgentEnvironment`, `AgentEnvironmentCapabilities`, `AgentEnvironmentEvent`, `AgentEnvironmentProvider`, `AgentEnvironmentQuery`, `AgentEnvironmentSummary`, `AgentGraph`, `AgenticOptions`, `AgenticRunResult`, `AgenticTask`, `AgenticTool`, `AgentSession`, `AgentSessionRef`, `AgentTurnResult`, `AllWorkersStalledOptions`, `AnalystRegistry`, `AnytimeReport`, `AnytimeStrategySummary`, `AnytimeTaskCurve`, `ArtifactHandle`, `AuditIntentInput`, `AuditIntentOptions`, `AuthoredHarness`, `AuthoredStrategy`, `AuthorStrategyOptions`, `BenchmarkConfig`, `BenchmarkLift`, `BenchmarkStrategySummary`, `BenchmarkTaskRow`, `BudgetPool`, `BusStats`, `ChampionPick`, `CheckpointRef`, `CheckpointRequest`, `CheckRunContext`, `CliWorktreeBridgeSeam`, `CoordinationMcpHandle`, `CopyOptions`, `CorpusReadbackOptions`, `CreateAgentEnvironmentInput`, `CreateTangleSandboxExactProcessProviderOptions`, `DefinedLeaderboard`, `DispatchReport`, `Driver`, `EvolutionArchiveNode`, `EvolutionAuthor`, `EvolutionBandInfo`, `EvolutionCandidate`, `EvolutionGeneration`, `EvolutionReport`, `ExecRequest`, `ExecResult`, `ExecutorResultMapping`, `ForkRequest`, `GitWorkspaceOptions`, `GraphResult`, `HarvestCorpusOptions`, `HarvestFailure`, `HarvestReport`, `Inbox`, `InProcessSandboxClientOptions`, `IntentAudit`, `Iteration`, `Leaderboard`, `LeaderboardOptions`, `LocalSandboxClientOptions`, `LoopDecisionPayload`, `LoopDispatchOptions`, `LoopEndedPayload`, `LoopIterationEndedPayload`, `LoopIterationStartedPayload`, `LoopPlanDescription`, `LoopResult`, `LoopSandboxPlacement`, `LoopStartedPayload`, `LoopTraceEmitter`, `LoopWinner`, `MaterializeLocalMcpOptions`, `McpEnvironmentOptions`, `McpToolDescriptor`, `NodeSnapshot`, `NoProgressForOptions`, `Observation`, `ObserveInput`, `ObserveOptions`, `OpenSandboxRunOptions`, `PairwiseOptions`, `PatchDeliverableOptions`, `PlacementInfo`, `PlateauOptions`, `ProgressTrackerOptions`, `PromotionGateOptions`, `PromotionVerdict`, `PublishOptions`, `ReproductionCheck`, `ResolveSandboxClientOptions`, `ResourceRequest`, `RollingDispatchOptions`, `RunAgenticOptions`, `RunAgentRoundsOptions`, `RunGraphOptions`, `SandboxRun`, `ShotSpec`, `SpawnOpts`, `StdioMcpConnection`, `StdioMcpServerSpec`, `SteerableSandboxArgs`, `Strategy`, `StrategyEvolutionConfig`, `StrategyResult`, `StreamAgentTurnOptions`, `StructuralRolloutConfig`, `SuperviseOptions`, `SuperviseSurfaceOptions`, `SupervisorAgentDeps`, `SupervisorOpts`, `SupervisorSpanOptions`, `SupervisorSpanRecorder`, `SurfaceScore`, `ToolSpec`, `ToolStepInput`, `TraceSource`, `TrajectoryAnalysis`, `UntrackedCopyStats`, `ValidationCtx`, `Validator`, `VerifierEnvironmentOptions`, `WatchTraceOptions`, `WaterfallCollector`, `WaterfallReport`, `WaterfallSpan`, `WorkerEvidenceInput`, `Workspace`, `WorkspaceRequest`, `WorkspaceRun`, `WorktreeCliExecutorOptions`, `WorktreeFanoutOptions`, `AgentEnvironmentStatus`, `AgentSessionStatus`, `ChampionPolicy`, `EdgeDeliveryOutcome`, `GraphEdge`, `LoopTraceEvent`, `MakeWorkerAgent`, `RepairStop`, `SandboxControlClient`, `WorkspaceCommit`. +**Undocumented supporting types** (add a TSDoc line at the declaration to earn a table row): `AcquireOptions`, `AgentEnvironment`, `AgentEnvironmentCapabilities`, `AgentEnvironmentEvent`, `AgentEnvironmentProvider`, `AgentEnvironmentQuery`, `AgentEnvironmentSummary`, `AgentGraph`, `AgenticOptions`, `AgenticRunResult`, `AgenticTask`, `AgenticTool`, `AgentSession`, `AgentSessionRef`, `AgentTurnResult`, `AllWorkersStalledOptions`, `AnalystRegistry`, `AnytimeReport`, `AnytimeStrategySummary`, `AnytimeTaskCurve`, `ArtifactHandle`, `AuditIntentInput`, `AuditIntentOptions`, `AuthoredHarness`, `AuthoredStrategy`, `AuthorStrategyOptions`, `BenchmarkConfig`, `BenchmarkLift`, `BenchmarkStrategySummary`, `BenchmarkTaskRow`, `BoxSurfaceReaderOptions`, `BudgetPool`, `BusStats`, `ChampionPick`, `CheckpointRef`, `CheckpointRequest`, `CheckRunContext`, `CliWorktreeBridgeSeam`, `CoordinationMcpHandle`, `CopyOptions`, `CorpusReadbackOptions`, `CreateAgentEnvironmentInput`, `CreateTangleSandboxExactProcessProviderOptions`, `DefinedLeaderboard`, `DispatchReport`, `Driver`, `EvolutionArchiveNode`, `EvolutionAuthor`, `EvolutionBandInfo`, `EvolutionCandidate`, `EvolutionGeneration`, `EvolutionReport`, `ExecRequest`, `ExecResult`, `ExecutorResultMapping`, `ForkRequest`, `GitWorkspaceOptions`, `GraphResult`, `HarvestCorpusOptions`, `HarvestFailure`, `HarvestReport`, `Inbox`, `InProcessSandboxClientOptions`, `IntentAudit`, `Iteration`, `Leaderboard`, `LeaderboardOptions`, `LocalSandboxClientOptions`, `LoopDecisionPayload`, `LoopDispatchOptions`, `LoopEndedPayload`, `LoopIterationEndedPayload`, `LoopIterationStartedPayload`, `LoopPlanDescription`, `LoopResult`, `LoopSandboxPlacement`, `LoopStartedPayload`, `LoopTraceEmitter`, `LoopWinner`, `MaterializeLocalMcpOptions`, `McpEnvironmentOptions`, `McpToolDescriptor`, `NodeSnapshot`, `NoProgressForOptions`, `Observation`, `ObserveInput`, `ObserveOptions`, `OpenSandboxRunOptions`, `PairwiseOptions`, `PatchDeliverableOptions`, `PlacementInfo`, `PlateauOptions`, `ProgressTrackerOptions`, `PromotionGateOptions`, `PromotionVerdict`, `PublishOptions`, `ReproductionCheck`, `ResolveSandboxClientOptions`, `ResourceRequest`, `RollingDispatchOptions`, `RunAgenticOptions`, `RunAgentRoundsOptions`, `RunGraphOptions`, `SandboxRun`, `ShotSpec`, `SpawnOpts`, `StdioMcpConnection`, `StdioMcpServerSpec`, `SteerableSandboxArgs`, `Strategy`, `StrategyEvolutionConfig`, `StrategyResult`, `StreamAgentTurnOptions`, `StructuralRolloutConfig`, `SuperviseOptions`, `SuperviseSurfaceOptions`, `SupervisorAgentDeps`, `SupervisorOpts`, `SupervisorSpanOptions`, `SupervisorSpanRecorder`, `SurfaceScore`, `ToolSpec`, `ToolStepInput`, `TraceSource`, `TrajectoryAnalysis`, `UntrackedCopyStats`, `ValidationCtx`, `Validator`, `VerifierEnvironmentOptions`, `WatchTraceOptions`, `WaterfallCollector`, `WaterfallReport`, `WaterfallSpan`, `WorkerEvidenceInput`, `Workspace`, `WorkspaceRequest`, `WorkspaceRun`, `WorktreeCliExecutorOptions`, `WorktreeFanoutOptions`, `AgentEnvironmentStatus`, `AgentSessionStatus`, `ChampionPolicy`, `EdgeDeliveryOutcome`, `GraphEdge`, `LoopTraceEvent`, `MakeWorkerAgent`, `RepairStop`, `SandboxControlClient`, `WorkspaceCommit`. ### Environment provider adapters — generic sandbox/compute bridge diff --git a/docs/api/runtime.md b/docs/api/runtime.md index 19ea7fd6..1e1b1a7d 100644 --- a/docs/api/runtime.md +++ b/docs/api/runtime.md @@ -16765,6 +16765,10 @@ The read seam's diagnostic. Present only for `unreadable`. A path to check at settle that was NOT necessarily mounted — where a harness is known to write self-authored surfaces (a memory dir's files, a refinement log). A watched path that was also mounted compares against its mount; one that wasn't reports `created` if it now exists. + `created` is an inference from the mount manifest, not a proof of authorship: a file the box + IMAGE shipped at a never-mounted path also reports `created`. Watch paths known absent at run + start (or enumerate the tree at start AND settle and watch the difference) to make the label + mean what it says. #### Properties @@ -16782,6 +16786,9 @@ Origin label carried onto the diff (default `'watched'`). ### HarvestSurfaceDiffsOptions +Inputs to [harvestSurfaceDiffs](#harvestsurfacediffs): the run's mount manifest, the read seam, and optional + watch paths for surfaces the agent may have created. + #### Properties ##### mounts @@ -26058,9 +26065,12 @@ and maps to `missing: true` — unless its `resourceType` names something other > **fsSurfaceReader**(`root`): [`SurfaceReader`](#surfacereader) -A [SurfaceReader](#surfacereader) over the local filesystem, for worktree/local workers. Relative mount -paths resolve against `root`. Absence maps to `missing: true`; every other failure carries the -error message. +A [SurfaceReader](#surfacereader) over the local filesystem, for worktree/local workers. Every path — +relative or absolute — must resolve INSIDE `root`: a path that escapes it (`../`, an absolute +path elsewhere) fails as a contained non-missing outcome rather than reading outside the +worktree, so a persisted or mistyped manifest path cannot turn the harvest into an +existence/hash oracle over the host filesystem. Absence maps to `missing: true`; every other +failure carries the error message. #### Parameters diff --git a/docs/canonical-api.md b/docs/canonical-api.md index 7d5fbbce..aa2939c6 100644 --- a/docs/canonical-api.md +++ b/docs/canonical-api.md @@ -4,7 +4,7 @@ Generated signatures and the complete export list live in docs/api/. Run pnpm docs:freshness after editing this file. --> -> **Version 0.132.1.** +> **Version 0.132.2.** > [`docs/api/primitive-catalog.md`](./api/primitive-catalog.md) lists every export and import path. > `agent-eval` must satisfy `>=0.144.12 <0.145.0`. > `sandbox` must satisfy `>=0.19.4 <0.20.0`. diff --git a/docs/design/prime-agent-harness-integration.md b/docs/design/prime-agent-harness-integration.md index 5e630d5f..2cefd7d9 100644 --- a/docs/design/prime-agent-harness-integration.md +++ b/docs/design/prime-agent-harness-integration.md @@ -3,8 +3,8 @@ > **Status: decision record + integration contract.** Prime Agent (Prime Intellect's > local agent OS: persistent IPython kernel, daemon-backed session trees, native `rlm(…)` > subagents, `/refine` continual-harness edits) is being added as a **sandbox-materialized -> harness** — the sandbox/adc side owns the image, the adapter, and the `prime-agent` -> harness id. This doc owns the Runtime side: the boundary we hold, which proposed +> harness** — the sandbox/adc side owns the image and the adapter; the harness id shipped +> as `prime` in `agent-interface` and the sandbox backend enum. This doc owns the Runtime side: the boundary we hold, which proposed > integration machinery we adopt or reject (most of it already exists here), and the > concrete asks on the substrate. On architecture conflict, > [architecture.md](../architecture.md) wins. @@ -40,8 +40,8 @@ Most of the machinery a "host Prime properly" plan calls for is built. Reuse it. | Execution-identity receipts (what exactly ran) | `materialized` journal event (authored/effective profile + platform-attachment digests), `RunProvenance.mounts`, `SelectionReceipt`, usage integrity guard | `src/runtime/supervise/`, `src/runtime/types.ts` | | Frozen candidate identity + workspace bytes | `buildAgentCandidateBundle` / `captureAgentCandidateWorkspace` | `src/candidate-execution/` | | Improvement governance (train/selection/final-test partitions, review, activation, rollback receipts) | `improve()` + `proposeAgentImprovement` → `reviewAgentImprovementProposal` → activation | `src/improvement/`, `src/intelligence/` | -| Equal-compute topology comparison + holdout gates | conserved budget pool, `pairedLift`, `promotionGate`, `heldOutGate` | `src/runtime/supervise/budget.ts`, bench/eval | -| Harness × model × task matrix | `defineLeaderboard` / `runProfileMatrix` (harness×model axes) | `src/runtime/define-leaderboard.ts` | +| Equal-compute topology comparison + holdout gates | conserved budget pool (`src/runtime/supervise/budget.ts`), `promotionGate`, `pairedLift`, `heldOutGate` | `src/runtime/promotion-gate.ts`, `bench/src/stats.mts`, `@tangle-network/agent-eval` contract | +| Harness × model × task matrix | `defineLeaderboard` / `runProfileMatrix` (harness×model axes) | `src/runtime/define-leaderboard.ts`, `@tangle-network/agent-eval` campaign | | Harness-agnostic trace normalization | `TraceSource` + the per-harness `toolPartDecoders` registry (add a harness = one validated decoder entry) | `src/runtime/supervise/trace-source.ts` | | Uncertain-effect posture after restart | uncertain reservations charged at full declared budget with telemetry explicitly unknown; instruction receipts retained as evidence, never auto-redelivered | `src/runtime/supervise/budget.ts`, `supervise/coordination-mcp.ts` | | Prime Intellect ecosystem bridge (verifiers env packaging, trace import) | `writePrimeIntellectPackage`, `importPrimeIntellectTraces` → `RunRecord` | `src/primeintellect/` | @@ -77,18 +77,23 @@ The scope law it feeds (Prime's `/refine` scopes, generalized to any harness): rationale ("expectedOutcome") is never promotion evidence — the trajectory that motivated an edit is not an independent measurement of it. -### 3.2 When the `prime-agent` id ships (blocked on substrate) +### 3.2 The `prime` id — shipped and wired -Small, mechanical, and compile-pinned — do these in the release that picks up the new -`agent-interface`/`sandbox` versions: +The id shipped as **`prime`** in `agent-interface` (`HarnessType`; capability rows for +its reasoning ladder and system-prompt semantics) and in the sandbox backend enum, and +`'prime'` is in `harnessBackends` (`src/runtime/sandbox-backend.ts`) — the +double-`satisfies` pin keeps the three enums aligned at compile time. Two follow-ups +remain, each gated on evidence rather than releases: -1. Add `'prime-agent'` to `harnessBackends` in `src/runtime/sandbox-backend.ts` (the - double-`satisfies` makes the absence a compile error, so this cannot be forgotten). -2. Register a `toolPartDecoders['prime-agent']` entry — see wish-list item 3 for why this - should be the canonical `ToolPart` decoder, not a bespoke one. -3. Add a **measured** column to +1. Register a `toolPartDecoders['prime']` entry — ONLY once validated against Prime's + real session output (the registry law). If the substrate normalizes into the + canonical `ToolPart` (wish-list item 3), the existing decoder covers it and the + entry is a one-line alias; until then unregistered harnesses fall through to the + try-all composite, which is correct. +2. Fill the **measured** `prime` column in [research/harness-compat.md](../research/harness-compat.md). Prime is Pi-lineage but - not Pi: do not inherit `pi`'s capability row by assumption. + not Pi (the fork's daemon rejects pi-line clients): do not inherit `pi`'s capability + row by assumption. ## 4. What we deliberately do NOT build @@ -131,9 +136,10 @@ Small, mechanical, and compile-pinned — do these in the release that picks up Numbered so the sandbox-side work can check them off. Items 1–5 are required for instrumented runs; 6–9 unlock the experiment tier. -1. **Harness identity.** `'prime-agent'` in `HarnessType` (agent-interface) and as a - sandbox `backend.type` — a distinct id, not `pi`. Capability rows (model lock, - reasoning clamp, selector honoring) measured against the real CLI, not inherited. +1. **Harness identity — DONE.** Shipped as `'prime'` in `HarnessType` (agent-interface, + with reasoning-ladder and prompt-channel capability rows) and as a sandbox + `backend.type` — a distinct id, not `pi`. The remaining capability facts (model lock, + selector honoring per the real CLI) still land measured, not inherited. 2. **Profile materialization.** Render an `AgentProfile` into Prime's native surfaces — system-prompt addendum, skills (as Prime executable skills or SKILL.md), MCP config, subagent specs, hooks where expressible — with the standard materialization receipt diff --git a/docs/research/harness-compat.md b/docs/research/harness-compat.md index d986986b..31df835e 100644 --- a/docs/research/harness-compat.md +++ b/docs/research/harness-compat.md @@ -30,12 +30,13 @@ Harnesses wired via cli-bridge: **claude-code** (2.1.177), **codex** (0.139.0), ## Pending columns -- **prime-agent** — Prime Intellect's agent OS (persistent IPython kernel, native `rlm(…)` - subagents, `/refine` continual-harness edits, daemon-backed session trees). Sandbox - backend in progress; see +- **prime** — Prime Intellect's agent OS (persistent IPython kernel, native `rlm(…)` + subagents, `/refine` continual-harness edits, daemon-backed session trees). The id + shipped in `agent-interface` `HarnessType` + the sandbox backend enum; see [design/prime-agent-harness-integration.md](../design/prime-agent-harness-integration.md). - **Do not inherit `pi`'s row** — Prime is Pi-lineage but adds materially different - semantics; every cell must be measured against the real CLI before this column fills in. + **Do not inherit `pi`'s row** — Prime is Pi-lineage but the fork's wire protocol has + diverged (its daemon rejects pi-line clients); every cell must be measured against the + real CLI before this column fills in. Expected steering-relevant deltas to verify: native subagent fan-out cap and whether it is configurable (runaway surface #3), whether the daemon protocol allows mid-step interrupt (black-box harnesses today degrade `steer_agent({interrupt:true})` to next diff --git a/src/runtime/sandbox-backend.ts b/src/runtime/sandbox-backend.ts index 48ae47f7..19ea023e 100644 --- a/src/runtime/sandbox-backend.ts +++ b/src/runtime/sandbox-backend.ts @@ -30,6 +30,7 @@ const harnessBackends = [ 'opencode', 'kimi-code', 'pi', + 'prime', 'hermes', 'openclaw', 'amp', diff --git a/src/runtime/surface-diff.test.ts b/src/runtime/surface-diff.test.ts index 534a9aa9..675d931a 100644 --- a/src/runtime/surface-diff.test.ts +++ b/src/runtime/surface-diff.test.ts @@ -134,6 +134,22 @@ describe('harvestSurfaceDiffs', () => { ]) }) + it('contains a reader that throws: the bad path reports unreadable, every other diff survives', async () => { + const read: SurfaceReader = (path) => { + if (path === 'boom.md') throw new Error('reader contract violation') + return Promise.resolve({ succeeded: true, value: new TextEncoder().encode('v2') }) + } + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('boom.md', 'x'), mount('a.md', 'v1')], + read, + }) + expect(diffs.map((d) => [d.path, d.status])).toEqual([ + ['boom.md', 'unreadable'], + ['a.md', 'modified'], + ]) + expect(diffs[0]?.error).toBe('reader contract violation') + }) + it('treats an uppercase manifest hash as equal to the settled lowercase hash', async () => { const entry = { ...mount('a.md', 'same'), sha256: sha('same').toUpperCase() } const diffs = await harvestSurfaceDiffs({ @@ -249,6 +265,17 @@ describe('fsSurfaceReader', () => { expect(miss).toMatchObject({ succeeded: false, missing: true }) }) + it('contains paths inside the root: ../ escapes and outside absolute paths fail without reading', async () => { + const read = fsSurfaceReader(root) + const escaped = await read('../outside.md') + expect(escaped).toMatchObject({ succeeded: false, missing: false }) + if (!escaped.succeeded) expect(escaped.error).toContain('outside the reader root') + const absolute = await read('/etc/hostname') + expect(absolute).toMatchObject({ succeeded: false, missing: false }) + const insideAbsolute = await read(join(root, 'notes.md')) + expect(insideAbsolute.succeeded).toBe(true) + }) + it('composes with the harvest over a real worktree edit', async () => { writeFileSync(join(root, 'skill.md'), 'v1') const mounts = [mount('skill.md', 'v1')] diff --git a/src/runtime/surface-diff.ts b/src/runtime/surface-diff.ts index a19270c4..dcf57abf 100644 --- a/src/runtime/surface-diff.ts +++ b/src/runtime/surface-diff.ts @@ -25,7 +25,7 @@ import { createHash } from 'node:crypto' import { readFile } from 'node:fs/promises' -import { isAbsolute, resolve } from 'node:path' +import { isAbsolute, resolve, sep } from 'node:path' import type { MountManifestEntry } from './types' /** Outcome of reading one surface back at settle. `missing: true` means the path no longer exists @@ -67,13 +67,19 @@ export interface SurfaceDiff { /** A path to check at settle that was NOT necessarily mounted — where a harness is known to write * self-authored surfaces (a memory dir's files, a refinement log). A watched path that was also - * mounted compares against its mount; one that wasn't reports `created` if it now exists. */ + * mounted compares against its mount; one that wasn't reports `created` if it now exists. + * `created` is an inference from the mount manifest, not a proof of authorship: a file the box + * IMAGE shipped at a never-mounted path also reports `created`. Watch paths known absent at run + * start (or enumerate the tree at start AND settle and watch the difference) to make the label + * mean what it says. */ export interface WatchedSurface { path: string /** Origin label carried onto the diff (default `'watched'`). */ source?: string } +/** Inputs to {@link harvestSurfaceDiffs}: the run's mount manifest, the read seam, and optional + * watch paths for surfaces the agent may have created. */ export interface HarvestSurfaceDiffsOptions { /** The run's mount manifest (`RunProvenance.mounts`). Entries sharing a path are collapsed to the * LAST entry — the bytes the agent actually saw at start. */ @@ -88,6 +94,21 @@ export interface HarvestSurfaceDiffsOptions { const sha256Hex = (bytes: Uint8Array): string => createHash('sha256').update(bytes).digest('hex') +/** Contain a reader that violates its typed-outcome contract by THROWING: the module's law is that + * a failed read becomes an `unreadable` diff, so one bad path must not reject the whole harvest + * and silently drop every other diff. */ +const readOutcome = async (read: SurfaceReader, path: string): Promise => { + try { + return await read(path) + } catch (err) { + return { + succeeded: false, + missing: false, + error: err instanceof Error ? err.message : String(err), + } + } +} + /** The collision/dedup key for a path: a leading `./` stripped, so `./AGENTS.md` (a mount * recorder's form) and `AGENTS.md` (a file-tree enumeration's form) are one surface. Absolute vs * relative cannot be reconciled here — keep those forms consistent between mounts and watches. */ @@ -112,7 +133,7 @@ export async function harvestSurfaceDiffs( } const mountDiffs = await Promise.all( [...byPath.values()].map(async (entry): Promise => { - const outcome = await options.read(entry.path) + const outcome = await readOutcome(options.read, entry.path) if (!outcome.succeeded) { return outcome.missing ? { @@ -143,7 +164,7 @@ export async function harvestSurfaceDiffs( ) const watchDiffs = await Promise.all( [...watchByPath.values()].map(async (watched): Promise => { - const outcome = await options.read(watched.path) + const outcome = await readOutcome(options.read, watched.path) if (!outcome.succeeded) { // A still-absent watched path is the expected no-op; a failing read is not. if (outcome.missing) return undefined @@ -240,13 +261,24 @@ export function boxSurfaceReader( } /** - * A {@link SurfaceReader} over the local filesystem, for worktree/local workers. Relative mount - * paths resolve against `root`. Absence maps to `missing: true`; every other failure carries the - * error message. + * A {@link SurfaceReader} over the local filesystem, for worktree/local workers. Every path — + * relative or absolute — must resolve INSIDE `root`: a path that escapes it (`../`, an absolute + * path elsewhere) fails as a contained non-missing outcome rather than reading outside the + * worktree, so a persisted or mistyped manifest path cannot turn the harvest into an + * existence/hash oracle over the host filesystem. Absence maps to `missing: true`; every other + * failure carries the error message. */ export function fsSurfaceReader(root: string): SurfaceReader { + const boundary = resolve(root) return async (path) => { - const target = isAbsolute(path) ? path : resolve(root, path) + const target = isAbsolute(path) ? resolve(path) : resolve(boundary, path) + if (target !== boundary && !target.startsWith(boundary + sep)) { + return { + succeeded: false, + missing: false, + error: `fsSurfaceReader: path ${JSON.stringify(path)} resolves outside the reader root ${JSON.stringify(boundary)}`, + } + } try { return { succeeded: true, value: new Uint8Array(await readFile(target)) } } catch (err) { From 678b7e4775afc6245e5e79e69b67ba602d754447 Mon Sep 17 00:00:00 2001 From: drewstone Date: Tue, 11 Aug 2026 21:59:45 +0000 Subject: [PATCH 05/10] chore(testing): regenerate proposal fixtures for 0.132.2 The release bump left the stored fixtures at runtimeVersion 0.132.1, so the fixture round-trip test fails against package.json on any branch carrying current main. Regenerated via generate:testing-fixture (the canonical path, so embedded digests stay valid). --- src/testing/fixtures/agent-improvement-proposal.json | 10 +++++----- .../fixtures/agent-profile-improvement-proposal.json | 6 +++--- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/src/testing/fixtures/agent-improvement-proposal.json b/src/testing/fixtures/agent-improvement-proposal.json index d21ff3dd..37a61fcb 100644 --- a/src/testing/fixtures/agent-improvement-proposal.json +++ b/src/testing/fixtures/agent-improvement-proposal.json @@ -1,6 +1,6 @@ { "changedSurfaces": ["prompt"], - "digest": "sha256:d7e382f866e8b8f947210b2f8c8c52bd56ff616cb10f8e872ba94912e13165a7", + "digest": "sha256:cd672d2488107486ee0e9bcbbd2057c540a8ad1e2b42de6e104ec0f1d710759c", "evaluation": { "decision": { "contributingChecks": [ @@ -4870,7 +4870,7 @@ ], "metadata": { "fixture": "agent-improvement-proposal", - "runtimeVersion": "0.132.1" + "runtimeVersion": "0.132.2" }, "objectives": [ { @@ -4981,8 +4981,8 @@ "baselineContentHash": "sha256:5c21ee53e513fc604cb09754e21c392b24a424da0ef37dbf8f1ee4a8a0b08f09", "candidateContentHash": "sha256:60fcbb1c728194bd51d7d19cb732d1c3f1881dce7e0a6266b41c8b98cfd65693", "kind": "agent-eval-loop", - "recordDigest": "sha256:81894ad1952e77f0b1e66765854897f16538eee7a57e3e370a2d53b102fa263e", - "runId": "agent-runtime-0.132.1-proposal-fixture", + "recordDigest": "sha256:aa37429c995c3a69d6b198ea08011b10fe3941bf983ff976efdcc29fbabad1d9", + "runId": "agent-runtime-0.132.2-proposal-fixture", "schema": "agent-candidate-experiment" } }, @@ -5009,5 +5009,5 @@ ], "kind": "agent-improvement-proposal", "proposedAt": "2026-07-10T01:00:00.000Z", - "runId": "agent-runtime-0.132.1-proposal-fixture" + "runId": "agent-runtime-0.132.2-proposal-fixture" } diff --git a/src/testing/fixtures/agent-profile-improvement-proposal.json b/src/testing/fixtures/agent-profile-improvement-proposal.json index b9adc00a..debd5fa5 100644 --- a/src/testing/fixtures/agent-profile-improvement-proposal.json +++ b/src/testing/fixtures/agent-profile-improvement-proposal.json @@ -1,6 +1,6 @@ { "changedSurfaces": ["prompt", "skills"], - "digest": "sha256:1637f21204f32f323a46abf1093a57fe8db5ad2fa2f918f1678937262209015b", + "digest": "sha256:d831d6e38120f962f83440e07a226fff41f198126cc214b6b9b946230182a2d9", "evaluation": { "decision": { "contributingChecks": [ @@ -1715,7 +1715,7 @@ ], "metadata": { "fixture": "agent-profile-improvement-proposal", - "runtimeVersion": "0.132.1" + "runtimeVersion": "0.132.2" }, "objectives": [ { @@ -1826,7 +1826,7 @@ "baselineContentHash": "sha256:21c495a37c418c10bde64fbaa188beddeed31f1f051ea60a6a6582a9ee0db704", "candidateContentHash": "sha256:103f77bc8481601eef1ad5fe6ba84a40dffabc3a44f421f8c8559121edab84e9", "kind": "agent-eval-loop", - "recordDigest": "sha256:d63fed2a06ba61e109301816ddc66a6915314098a67b1b881d64730093e6a867", + "recordDigest": "sha256:5c28507ecfe1857682bf0504d22e9dbb81cc40f6fa12bff3c8b87c685343f692", "runId": "profile-improvement-1", "schema": "agent-profile-improvement-experiment" } From e3894a4f45fe60e266a37b46c5fb6c89acff0b01 Mon Sep 17 00:00:00 2001 From: drewstone Date: Tue, 11 Aug 2026 22:20:13 +0000 Subject: [PATCH 06/10] docs: align version pin and regenerate catalog for 0.132.3 --- docs/api/primitive-catalog.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/api/primitive-catalog.md b/docs/api/primitive-catalog.md index 81e973cb..7d53813b 100644 --- a/docs/api/primitive-catalog.md +++ b/docs/api/primitive-catalog.md @@ -7,7 +7,7 @@ # Primitive catalog — the never-stale anti-reinvention inventory -> **GENERATED** from `@tangle-network/agent-runtime@0.132.2` and `@tangle-network/agent-eval@0.144.12` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`. +> **GENERATED** from `@tangle-network/agent-runtime@0.132.3` and `@tangle-network/agent-eval@0.144.12` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`. ## 1. agent-runtime — own public surface From 1f5016f97c3824abfe96b2c34a7ec68490ba218a Mon Sep 17 00:00:00 2001 From: Drew Stone Date: Wed, 12 Aug 2026 17:25:25 -0600 Subject: [PATCH 07/10] refactor(kernel): share one box-read retry between the deliverable read and the surface harvest The box data plane can transiently 404 a just-written file. `openSandboxRun` already retried its deliverable read for that reason with an inline loop, and `boxSurfaceReader` added a second backoff of its own. Two copies of one policy drift apart. Extract `readBoxPathWithRetry` and call it from both sites. `beforeAttempt` keeps the caller's authority to abandon the read with an error only it can build, so `openSandboxRun` still throws `SandboxRunAbortError` with its drained events. Thread an `AbortSignal` into `boxSurfaceReader`. The harvest itself waits on nothing, so the reader owns every cancellable moment; `harvestSurfaceDiffs` documents why it takes no signal. --- docs/api/runtime.md | 13 ++++++ src/runtime/box-read-retry.test.ts | 63 ++++++++++++++++++++++++++ src/runtime/box-read-retry.ts | 60 +++++++++++++++++++++++++ src/runtime/sandbox-run.ts | 30 ++++++------- src/runtime/surface-diff.test.ts | 29 ++++++++++++ src/runtime/surface-diff.ts | 72 +++++++++++++++--------------- 6 files changed, 215 insertions(+), 52 deletions(-) create mode 100644 src/runtime/box-read-retry.test.ts create mode 100644 src/runtime/box-read-retry.ts diff --git a/docs/api/runtime.md b/docs/api/runtime.md index d1884dc9..40b21da9 100644 --- a/docs/api/runtime.md +++ b/docs/api/runtime.md @@ -17023,6 +17023,13 @@ Read attempts per path before settling on a failed outcome. The data plane can t Linear backoff base between attempts (delay = base × attempt). Default 250. +##### signal? + +> `optional` **signal?**: `AbortSignal` + +Cuts the retry waits short when the run is abandoned. The reader still returns a typed + outcome — the harvest reports what it managed to read rather than rejecting. + *** ### CreateTangleSandboxExactProcessProviderOptions @@ -26274,6 +26281,12 @@ produce no entry; reads run concurrently; output preserves record order, mounts watch-only paths. Mounts and watches sharing a path key are each collapsed to the LAST entry, and a watched path that was also mounted compares against its mount (never reports `created`). +The harvest takes no `AbortSignal`: it is pure fan-out over the read seam and waits on nothing +itself, so every cancellable moment belongs to the reader. Pass a signal to the reader instead +([BoxSurfaceReaderOptions.signal](#signal-23), or close over one in a custom [SurfaceReader](#surfacereader)) — +that cuts the backoff waits, and the harvest still returns the diffs it did establish rather +than discarding settle-time evidence on a late cancellation. + #### Parameters ##### options diff --git a/src/runtime/box-read-retry.test.ts b/src/runtime/box-read-retry.test.ts new file mode 100644 index 00000000..2773f65d --- /dev/null +++ b/src/runtime/box-read-retry.test.ts @@ -0,0 +1,63 @@ +import { describe, expect, it } from 'vitest' +import { boxReadErrorMessage, readBoxPathWithRetry } from './box-read-retry' + +describe('readBoxPathWithRetry', () => { + it('returns the text as soon as an attempt succeeds, without spending the rest', async () => { + let calls = 0 + const read = () => { + calls += 1 + return calls < 3 ? Promise.reject(new Error('404')) : Promise.resolve('flushed') + } + const result = await readBoxPathWithRetry(read, 'a.md', { attempts: 4, delayMs: 0 }) + expect(result).toEqual({ succeeded: true, text: 'flushed' }) + expect(calls).toBe(3) + }) + + it('carries the LAST error out after every attempt fails', async () => { + let calls = 0 + const read = () => { + calls += 1 + return Promise.reject(new Error(`attempt ${calls}`)) + } + const result = await readBoxPathWithRetry(read, 'a.md', { attempts: 3, delayMs: 0 }) + expect(calls).toBe(3) + expect(result.succeeded).toBe(false) + if (!result.succeeded) expect(boxReadErrorMessage(result.error)).toBe('attempt 3') + }) + + it('lets beforeAttempt abandon the read with the caller’s own error', async () => { + let calls = 0 + const read = () => { + calls += 1 + return Promise.reject(new Error('transient')) + } + await expect( + readBoxPathWithRetry(read, 'a.md', { + attempts: 5, + delayMs: 0, + beforeAttempt: (lastError) => { + if (lastError !== undefined) + throw new Error(`gave up after: ${boxReadErrorMessage(lastError)}`) + }, + }), + ).rejects.toThrow('gave up after: transient') + expect(calls).toBe(1) + }) + + it('treats a sub-1 attempt count as a single attempt rather than skipping the read', async () => { + let calls = 0 + const read = () => { + calls += 1 + return Promise.resolve('once') + } + const result = await readBoxPathWithRetry(read, 'a.md', { attempts: 0, delayMs: 0 }) + expect(result).toEqual({ succeeded: true, text: 'once' }) + expect(calls).toBe(1) + }) + + it('passes undefined through boxReadErrorMessage so "no failure yet" stays distinguishable', () => { + expect(boxReadErrorMessage(undefined)).toBeUndefined() + expect(boxReadErrorMessage(new Error('boom'))).toBe('boom') + expect(boxReadErrorMessage('plain string')).toBe('plain string') + }) +}) diff --git a/src/runtime/box-read-retry.ts b/src/runtime/box-read-retry.ts new file mode 100644 index 00000000..0bc472b4 --- /dev/null +++ b/src/runtime/box-read-retry.ts @@ -0,0 +1,60 @@ +/** + * One bounded-retry read over a sandbox box's filesystem, shared by every caller that reads a + * path the agent may have just written. + * + * The box data plane can transiently 404 a file that exists: the write is not yet flushed, or an + * edge read lands on a stale view. A first-attempt failure is therefore not evidence of absence, + * and a caller that treats it as such records a lie — "the agent produced nothing" for a + * deliverable, or "the agent deleted this surface" for a mounted profile file. + * + * Retries wait `delayMs × attempt` (linear), so the total wait for n attempts is + * `delayMs × n(n-1)/2`. Set `delayMs` to 0 to retry without waiting (tests). + */ + +import { sleep } from './util' + +/** Text read from the box, or the last error after every attempt failed. */ +export type BoxReadAttemptResult = + | { succeeded: true; text: string } + | { succeeded: false; error: unknown } + +export interface BoxReadRetryOptions { + /** Total read attempts, including the first. Values below 1 are treated as 1. */ + attempts: number + /** Linear backoff base in ms; the i-th retry waits `delayMs × i`. 0 disables the wait. */ + delayMs: number + /** Cuts the backoff waits short. The loop still ends through `beforeAttempt`, so a caller that + * must throw its own abort error keeps that authority. */ + signal?: AbortSignal + /** Runs before every attempt, carrying the previous attempt's error (`undefined` on the first). + * Throw from here to abandon the read — the way an aborting caller cancels with an error only + * it can construct. */ + beforeAttempt?: (lastError: unknown) => void +} + +/** The diagnostic for a failed attempt. `undefined` in, `undefined` out — so a caller can pass the + * "no attempt has failed yet" state through without inventing a message for it. */ +export function boxReadErrorMessage(error: unknown): string | undefined { + if (error === undefined) return undefined + return error instanceof Error ? error.message : String(error) +} + +export async function readBoxPathWithRetry( + read: (path: string) => Promise, + path: string, + options: BoxReadRetryOptions, +): Promise { + const attempts = Math.max(1, options.attempts) + let lastError: unknown + for (let attempt = 1; attempt <= attempts; attempt += 1) { + options.beforeAttempt?.(lastError) + try { + return { succeeded: true, text: await read(path) } + } catch (err) { + lastError = err + if (attempt < attempts && options.delayMs > 0) + await sleep(options.delayMs * attempt, options.signal) + } + } + return { succeeded: false, error: lastError } +} diff --git a/src/runtime/sandbox-run.ts b/src/runtime/sandbox-run.ts index a3d8db70..4212df6a 100644 --- a/src/runtime/sandbox-run.ts +++ b/src/runtime/sandbox-run.ts @@ -34,11 +34,12 @@ import type { PromptOptions, SandboxEvent, SandboxInstance } from '@tangle-network/sandbox' import type { RuntimeHooks, RuntimeHookTarget } from '../runtime-hooks' import { notifyRuntimeHookEvent } from '../runtime-hooks' +import { boxReadErrorMessage, readBoxPathWithRetry } from './box-read-retry' import { probeSandboxCapabilities } from './sandbox-capabilities' import { notifySandboxEventObserver } from './sandbox-events' import { createSandboxLineage, type SandboxLineageHandle } from './sandbox-lineage' import type { AgentRunSpec, SandboxClient } from './types' -import { isAbortError, randomSuffix, sleep } from './util' +import { isAbortError, randomSuffix } from './util' /** * How a typed deliverable `Out` is materialized from a finished turn. @@ -269,26 +270,21 @@ export async function openSandboxRun( return { out: deliverable.fromEvents(collected), events: collected } } if (options.signal.aborted) throw new SandboxRunAbortError(collected) - let raw = '' - let readError: string | undefined // The data plane can transiently 404 a just-written artifact (write not yet // flushed, or an edge-read blip) — retry a few times with backoff before // declaring the deliverable empty, so a transient read failure is not recorded // as "the agent produced nothing". - const readAttempts = 4 - const readDelayMs = options.readRetryDelayMs ?? 1000 - for (let attempt = 0; attempt < readAttempts; attempt += 1) { - if (options.signal.aborted) throw new SandboxRunAbortError(collected, readError) - try { - raw = await box.fs.read(deliverable.path) - readError = undefined - break - } catch (err) { - readError = err instanceof Error ? err.message : String(err) - if (attempt < readAttempts - 1 && readDelayMs > 0) - await sleep(readDelayMs * (attempt + 1), options.signal) - } - } + const attempted = await readBoxPathWithRetry(box.fs.read.bind(box.fs), deliverable.path, { + attempts: 4, + delayMs: options.readRetryDelayMs ?? 1000, + signal: options.signal, + beforeAttempt: (lastError) => { + if (options.signal.aborted) + throw new SandboxRunAbortError(collected, boxReadErrorMessage(lastError)) + }, + }) + const raw = attempted.succeeded ? attempted.text : '' + const readError = attempted.succeeded ? undefined : boxReadErrorMessage(attempted.error) return { out: deliverable.fromArtifact(raw, collected), events: collected, diff --git a/src/runtime/surface-diff.test.ts b/src/runtime/surface-diff.test.ts index 675d931a..b535ad76 100644 --- a/src/runtime/surface-diff.test.ts +++ b/src/runtime/surface-diff.test.ts @@ -206,6 +206,35 @@ describe('boxSurfaceReader', () => { expect(outcome.succeeded).toBe(true) }) + it('does not report a still-flushing mounted file as removed when the first read 404s', async () => { + const blip = new Error('not flushed yet') + blip.name = 'NotFoundError' + let calls = 0 + const box = { + fs: { + read: () => { + calls += 1 + return calls === 1 ? Promise.reject(blip) : Promise.resolve('edited by the agent') + }, + }, + } + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('CLAUDE.md', 'original')], + read: boxSurfaceReader(box, { retryDelayMs: 0 }), + }) + expect(diffs.map((d) => d.status)).not.toContain('removed') + expect(diffs).toEqual([ + { + path: 'CLAUDE.md', + status: 'modified', + mountedSha256: sha('original'), + source: 'test', + settledSha256: sha('edited by the agent'), + settledBytes: Buffer.byteLength('edited by the agent'), + }, + ]) + }) + it('reports a box-level NotFoundError (resourceType names the sandbox) as unreadable, never missing', async () => { const boxGone = Object.assign(new Error('sandbox sb-1 not found'), { name: 'NotFoundError', diff --git a/src/runtime/surface-diff.ts b/src/runtime/surface-diff.ts index dcf57abf..a88fbe59 100644 --- a/src/runtime/surface-diff.ts +++ b/src/runtime/surface-diff.ts @@ -26,6 +26,7 @@ import { createHash } from 'node:crypto' import { readFile } from 'node:fs/promises' import { isAbsolute, resolve, sep } from 'node:path' +import { readBoxPathWithRetry } from './box-read-retry' import type { MountManifestEntry } from './types' /** Outcome of reading one surface back at settle. `missing: true` means the path no longer exists @@ -120,6 +121,12 @@ const pathKey = (p: string): string => p.replace(/^\.\//, '') * produce no entry; reads run concurrently; output preserves record order, mounts before * watch-only paths. Mounts and watches sharing a path key are each collapsed to the LAST entry, * and a watched path that was also mounted compares against its mount (never reports `created`). + * + * The harvest takes no `AbortSignal`: it is pure fan-out over the read seam and waits on nothing + * itself, so every cancellable moment belongs to the reader. Pass a signal to the reader instead + * ({@link BoxSurfaceReaderOptions.signal}, or close over one in a custom {@link SurfaceReader}) — + * that cuts the backoff waits, and the harvest still returns the diffs it did establish rather + * than discarding settle-time evidence on a late cancellation. */ export async function harvestSurfaceDiffs( options: HarvestSurfaceDiffsOptions, @@ -201,10 +208,11 @@ export interface BoxSurfaceReaderOptions { attempts?: number /** Linear backoff base between attempts (delay = base × attempt). Default 250. */ retryDelayMs?: number + /** Cuts the retry waits short when the run is abandoned. The reader still returns a typed + * outcome — the harvest reports what it managed to read rather than rejecting. */ + signal?: AbortSignal } -const sleep = (ms: number) => new Promise((res) => setTimeout(res, ms)) - /** * A {@link SurfaceReader} over a sandbox box's filesystem — the same `box.fs.read` seam * `openSandboxRun` reads deliverables through, with the same transient-404 posture (bounded @@ -219,44 +227,38 @@ export function boxSurfaceReader( box: SurfaceReadBox, options: BoxSurfaceReaderOptions = {}, ): SurfaceReader { - const attempts = options.attempts ?? 3 - const retryDelayMs = options.retryDelayMs ?? 250 return async (path) => { - let last: SurfaceReadOutcome = { - succeeded: false, - missing: false, - error: 'boxSurfaceReader: no read attempted', - } - for (let attempt = 1; attempt <= attempts; attempt += 1) { - try { - const text = await box.fs.read(path) - if (text.includes('�')) { - return { - succeeded: false, - missing: false, - error: `boxSurfaceReader: content at ${JSON.stringify(path)} is not valid UTF-8 text (the box text wire lossy-decoded it); binary surfaces need a byte-faithful reader`, - } - } - return { succeeded: true, value: new TextEncoder().encode(text) } - } catch (err) { - const notFound = err instanceof Error && err.name === 'NotFoundError' - const resourceType = - err && typeof err === 'object' && 'resourceType' in err - ? String((err as { resourceType: unknown }).resourceType) - : undefined - // A NotFoundError naming a non-file resource (Sandbox / session) means the TRANSPORT is - // gone — reporting it as `missing` would read as "every surface was removed". - const fileNotFound = - notFound && (resourceType === undefined || /file|path/i.test(resourceType)) - last = { + const result = await readBoxPathWithRetry(box.fs.read.bind(box.fs), path, { + attempts: options.attempts ?? 3, + delayMs: options.retryDelayMs ?? 250, + signal: options.signal, + }) + if (result.succeeded) { + // A lossy decode is deterministic — retrying cannot recover the bytes, so this is a final + // outcome rather than another attempt. + if (result.text.includes('�')) { + return { succeeded: false, - missing: fileNotFound, - error: err instanceof Error ? err.message : String(err), + missing: false, + error: `boxSurfaceReader: content at ${JSON.stringify(path)} is not valid UTF-8 text (the box text wire lossy-decoded it); binary surfaces need a byte-faithful reader`, } - if (attempt < attempts && retryDelayMs > 0) await sleep(retryDelayMs * attempt) } + return { succeeded: true, value: new TextEncoder().encode(result.text) } + } + const err = result.error + const notFound = err instanceof Error && err.name === 'NotFoundError' + const resourceType = + err && typeof err === 'object' && 'resourceType' in err + ? String((err as { resourceType: unknown }).resourceType) + : undefined + // A NotFoundError naming a non-file resource (Sandbox / session) means the TRANSPORT is + // gone — reporting it as `missing` would read as "every surface was removed". + const fileNotFound = notFound && (resourceType === undefined || /file|path/i.test(resourceType)) + return { + succeeded: false, + missing: fileNotFound, + error: err instanceof Error ? err.message : String(err), } - return last } } From d90d82f49a3851104d2000f3014e5b242e29030c Mon Sep 17 00:00:00 2001 From: Drew Stone Date: Wed, 12 Aug 2026 18:15:32 -0600 Subject: [PATCH 08/10] fix(kernel): classify a box file 404 as a removed surface, not an unreadable one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The SDK's HTTP mapper builds `new NotFoundError(data.resourceType || 'Resource', ...)`, so a file 404 carries `'Resource'` whenever the server does not name a type. Matching only file-ish names therefore classified an ordinary deletion as `unreadable`, and the harvest could never report `removed` — the one status that says the agent deleted a mounted surface. Invert the test: a NotFoundError is the file unless its `resourceType` names the box or the session, which stays `unreadable` because the transport is gone. Also stop the retry loop once the run is abandoned, and give `BoxSurfaceReaderOptions` the declaration summary the catalog asks for. --- docs/api/primitive-catalog.md | 3 ++- docs/api/runtime.md | 2 ++ src/runtime/box-read-retry.ts | 11 +++++++-- src/runtime/surface-diff.test.ts | 42 ++++++++++++++++++++++++++++++++ src/runtime/surface-diff.ts | 12 ++++++--- 5 files changed, 63 insertions(+), 7 deletions(-) diff --git a/docs/api/primitive-catalog.md b/docs/api/primitive-catalog.md index 135d37ad..f4fe8f98 100644 --- a/docs/api/primitive-catalog.md +++ b/docs/api/primitive-catalog.md @@ -805,6 +805,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 753 exports. | `AuthorizedSpawnContext` | interface | Exact trusted context after a manager-authored spawn has passed product authorization. | | `BenchmarkCell` | interface | One strategy's outcome on one task — the per-task cell an optimizer consumes. | | `BenchmarkReport` | interface | Benchmark output: per-strategy means plus the full per-task × per-strategy losses table an optimizer mines. | +| `BoxSurfaceReaderOptions` | interface | Retry and cancellation controls for {@link boxSurfaceReader}. | | `BridgeModelCredential` | interface | A live, request-scoped model credential reference for a local cli-bridge. | | `BridgeSeam` | interface | cli-bridge seam. A local OpenAI-compatible bridge that fronts harness CLIs | | `Budget` | interface | A budget envelope on a spawn or the root. All ceilings; the pool reserves against them. | @@ -1131,7 +1132,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 753 exports. | `WorktreeCheckRunner` | type | The single shell-command-in-worktree runner seam (replaces the per-executor copies). | | `WorktreePatchArtifact` | type | Terminal artifact of one worktree-CLI run — the canonical worktree-harness result (the captured | -**Undocumented supporting types** (add a TSDoc line at the declaration to earn a table row): `AcquireOptions`, `AgentEnvironment`, `AgentEnvironmentCapabilities`, `AgentEnvironmentEvent`, `AgentEnvironmentProvider`, `AgentEnvironmentQuery`, `AgentEnvironmentSummary`, `AgentGraph`, `AgenticOptions`, `AgenticRunResult`, `AgenticTask`, `AgenticTool`, `AgentSession`, `AgentSessionRef`, `AgentTurnResult`, `AllWorkersStalledOptions`, `AnalystRegistry`, `AnytimeReport`, `AnytimeStrategySummary`, `AnytimeTaskCurve`, `ArtifactHandle`, `AuditIntentInput`, `AuditIntentOptions`, `AuthoredHarness`, `AuthoredStrategy`, `AuthorStrategyOptions`, `BenchmarkConfig`, `BenchmarkLift`, `BenchmarkStrategySummary`, `BenchmarkTaskRow`, `BoxSurfaceReaderOptions`, `BudgetPool`, `BusStats`, `ChampionPick`, `CheckpointRef`, `CheckpointRequest`, `CheckRunContext`, `CliWorktreeBridgeSeam`, `CoordinationMcpHandle`, `CopyOptions`, `CorpusReadbackOptions`, `CreateAgentEnvironmentInput`, `CreateTangleSandboxExactProcessProviderOptions`, `DefinedLeaderboard`, `DispatchReport`, `Driver`, `EvolutionArchiveNode`, `EvolutionAuthor`, `EvolutionBandInfo`, `EvolutionCandidate`, `EvolutionGeneration`, `EvolutionReport`, `ExecRequest`, `ExecResult`, `ExecutorResultMapping`, `ForkRequest`, `GitWorkspaceOptions`, `GraphResult`, `HarvestCorpusOptions`, `HarvestFailure`, `HarvestReport`, `Inbox`, `InProcessSandboxClientOptions`, `IntentAudit`, `Iteration`, `Leaderboard`, `LeaderboardOptions`, `LocalSandboxClientOptions`, `LoopDecisionPayload`, `LoopDispatchOptions`, `LoopEndedPayload`, `LoopIterationEndedPayload`, `LoopIterationStartedPayload`, `LoopPlanDescription`, `LoopResult`, `LoopSandboxPlacement`, `LoopStartedPayload`, `LoopTraceEmitter`, `LoopWinner`, `MaterializeLocalMcpOptions`, `McpEnvironmentOptions`, `McpToolDescriptor`, `NodeSnapshot`, `NoProgressForOptions`, `Observation`, `ObserveInput`, `ObserveOptions`, `OpenSandboxRunOptions`, `PairwiseOptions`, `PatchDeliverableOptions`, `PlacementInfo`, `PlateauOptions`, `ProgressTrackerOptions`, `PromotionGateOptions`, `PromotionVerdict`, `PublishOptions`, `ReproductionCheck`, `ResolveSandboxClientOptions`, `ResourceRequest`, `RollingDispatchOptions`, `RunAgenticOptions`, `RunAgentRoundsOptions`, `RunGraphOptions`, `SandboxRun`, `ShotSpec`, `SpawnOpts`, `StdioMcpConnection`, `StdioMcpServerSpec`, `SteerableSandboxArgs`, `Strategy`, `StrategyEvolutionConfig`, `StrategyResult`, `StreamAgentTurnOptions`, `StructuralRolloutConfig`, `SuperviseOptions`, `SuperviseSurfaceOptions`, `SupervisorAgentDeps`, `SupervisorOpts`, `SupervisorSpanOptions`, `SupervisorSpanRecorder`, `SurfaceScore`, `ToolSpec`, `ToolStepInput`, `TraceSource`, `TrajectoryAnalysis`, `UntrackedCopyStats`, `ValidationCtx`, `Validator`, `VerifierEnvironmentOptions`, `WatchTraceOptions`, `WaterfallCollector`, `WaterfallReport`, `WaterfallSpan`, `WorkerEvidenceInput`, `Workspace`, `WorkspaceRequest`, `WorkspaceRun`, `WorktreeCliExecutorOptions`, `WorktreeFanoutOptions`, `AgentEnvironmentStatus`, `AgentSessionStatus`, `ChampionPolicy`, `EdgeDeliveryOutcome`, `GraphEdge`, `LoopTraceEvent`, `MakeWorkerAgent`, `RepairStop`, `SandboxControlClient`, `WorkspaceCommit`. +**Undocumented supporting types** (add a TSDoc line at the declaration to earn a table row): `AcquireOptions`, `AgentEnvironment`, `AgentEnvironmentCapabilities`, `AgentEnvironmentEvent`, `AgentEnvironmentProvider`, `AgentEnvironmentQuery`, `AgentEnvironmentSummary`, `AgentGraph`, `AgenticOptions`, `AgenticRunResult`, `AgenticTask`, `AgenticTool`, `AgentSession`, `AgentSessionRef`, `AgentTurnResult`, `AllWorkersStalledOptions`, `AnalystRegistry`, `AnytimeReport`, `AnytimeStrategySummary`, `AnytimeTaskCurve`, `ArtifactHandle`, `AuditIntentInput`, `AuditIntentOptions`, `AuthoredHarness`, `AuthoredStrategy`, `AuthorStrategyOptions`, `BenchmarkConfig`, `BenchmarkLift`, `BenchmarkStrategySummary`, `BenchmarkTaskRow`, `BudgetPool`, `BusStats`, `ChampionPick`, `CheckpointRef`, `CheckpointRequest`, `CheckRunContext`, `CliWorktreeBridgeSeam`, `CoordinationMcpHandle`, `CopyOptions`, `CorpusReadbackOptions`, `CreateAgentEnvironmentInput`, `CreateTangleSandboxExactProcessProviderOptions`, `DefinedLeaderboard`, `DispatchReport`, `Driver`, `EvolutionArchiveNode`, `EvolutionAuthor`, `EvolutionBandInfo`, `EvolutionCandidate`, `EvolutionGeneration`, `EvolutionReport`, `ExecRequest`, `ExecResult`, `ExecutorResultMapping`, `ForkRequest`, `GitWorkspaceOptions`, `GraphResult`, `HarvestCorpusOptions`, `HarvestFailure`, `HarvestReport`, `Inbox`, `InProcessSandboxClientOptions`, `IntentAudit`, `Iteration`, `Leaderboard`, `LeaderboardOptions`, `LocalSandboxClientOptions`, `LoopDecisionPayload`, `LoopDispatchOptions`, `LoopEndedPayload`, `LoopIterationEndedPayload`, `LoopIterationStartedPayload`, `LoopPlanDescription`, `LoopResult`, `LoopSandboxPlacement`, `LoopStartedPayload`, `LoopTraceEmitter`, `LoopWinner`, `MaterializeLocalMcpOptions`, `McpEnvironmentOptions`, `McpToolDescriptor`, `NodeSnapshot`, `NoProgressForOptions`, `Observation`, `ObserveInput`, `ObserveOptions`, `OpenSandboxRunOptions`, `PairwiseOptions`, `PatchDeliverableOptions`, `PlacementInfo`, `PlateauOptions`, `ProgressTrackerOptions`, `PromotionGateOptions`, `PromotionVerdict`, `PublishOptions`, `ReproductionCheck`, `ResolveSandboxClientOptions`, `ResourceRequest`, `RollingDispatchOptions`, `RunAgenticOptions`, `RunAgentRoundsOptions`, `RunGraphOptions`, `SandboxRun`, `ShotSpec`, `SpawnOpts`, `StdioMcpConnection`, `StdioMcpServerSpec`, `SteerableSandboxArgs`, `Strategy`, `StrategyEvolutionConfig`, `StrategyResult`, `StreamAgentTurnOptions`, `StructuralRolloutConfig`, `SuperviseOptions`, `SuperviseSurfaceOptions`, `SupervisorAgentDeps`, `SupervisorOpts`, `SupervisorSpanOptions`, `SupervisorSpanRecorder`, `SurfaceScore`, `ToolSpec`, `ToolStepInput`, `TraceSource`, `TrajectoryAnalysis`, `UntrackedCopyStats`, `ValidationCtx`, `Validator`, `VerifierEnvironmentOptions`, `WatchTraceOptions`, `WaterfallCollector`, `WaterfallReport`, `WaterfallSpan`, `WorkerEvidenceInput`, `Workspace`, `WorkspaceRequest`, `WorkspaceRun`, `WorktreeCliExecutorOptions`, `WorktreeFanoutOptions`, `AgentEnvironmentStatus`, `AgentSessionStatus`, `ChampionPolicy`, `EdgeDeliveryOutcome`, `GraphEdge`, `LoopTraceEvent`, `MakeWorkerAgent`, `RepairStop`, `SandboxControlClient`, `WorkspaceCommit`. ### Environment provider adapters — generic sandbox/compute bridge diff --git a/docs/api/runtime.md b/docs/api/runtime.md index 40b21da9..ae9eea69 100644 --- a/docs/api/runtime.md +++ b/docs/api/runtime.md @@ -17006,6 +17006,8 @@ The minimal box surface the box-backed reader needs — structurally typed so th ### BoxSurfaceReaderOptions +Retry and cancellation controls for [boxSurfaceReader](#boxsurfacereader). + #### Properties ##### attempts? diff --git a/src/runtime/box-read-retry.ts b/src/runtime/box-read-retry.ts index 0bc472b4..1787d02b 100644 --- a/src/runtime/box-read-retry.ts +++ b/src/runtime/box-read-retry.ts @@ -23,8 +23,9 @@ export interface BoxReadRetryOptions { attempts: number /** Linear backoff base in ms; the i-th retry waits `delayMs × i`. 0 disables the wait. */ delayMs: number - /** Cuts the backoff waits short. The loop still ends through `beforeAttempt`, so a caller that - * must throw its own abort error keeps that authority. */ + /** Stops the retries once aborted: the pending wait ends and no further attempt is spent on an + * abandoned run. The last error is still returned, so a caller that must raise its own abort + * error keeps that authority through `beforeAttempt`. */ signal?: AbortSignal /** Runs before every attempt, carrying the previous attempt's error (`undefined` on the first). * Throw from here to abandon the read — the way an aborting caller cancels with an error only @@ -54,6 +55,12 @@ export async function readBoxPathWithRetry( lastError = err if (attempt < attempts && options.delayMs > 0) await sleep(options.delayMs * attempt, options.signal) + // An abandoned run earns no further attempts. `beforeAttempt` still runs first, so a caller + // that must raise its own abort error keeps that authority instead of getting a bare result. + if (options.signal?.aborted) { + options.beforeAttempt?.(lastError) + break + } } } return { succeeded: false, error: lastError } diff --git a/src/runtime/surface-diff.test.ts b/src/runtime/surface-diff.test.ts index b535ad76..586f797b 100644 --- a/src/runtime/surface-diff.test.ts +++ b/src/runtime/surface-diff.test.ts @@ -235,6 +235,48 @@ describe('boxSurfaceReader', () => { ]) }) + it("treats the SDK's default 'Resource' resourceType as a missing FILE, so a deletion reports removed", async () => { + // The SDK's HTTP mapper builds `new NotFoundError(data.resourceType || 'Resource', ...)`, so a + // file 404 carries 'Resource' whenever the server does not name the type. + const deleted = Object.assign(new Error('not found'), { + name: 'NotFoundError', + resourceType: 'Resource', + resourceId: 'unknown', + }) + const box = { fs: { read: () => Promise.reject(deleted) } } + expect(await boxSurfaceReader(box, { attempts: 1 })('CLAUDE.md')).toEqual({ + succeeded: false, + missing: true, + error: 'not found', + }) + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('CLAUDE.md', 'original')], + read: boxSurfaceReader(box, { attempts: 1 }), + }) + expect(diffs.map((d) => d.status)).toEqual(['removed']) + }) + + it('stops retrying once the signal aborts instead of spending the remaining attempts', async () => { + const controller = new AbortController() + let calls = 0 + const box = { + fs: { + read: () => { + calls += 1 + controller.abort() + return Promise.reject(new Error('transport down')) + }, + }, + } + const outcome = await boxSurfaceReader(box, { + attempts: 5, + retryDelayMs: 0, + signal: controller.signal, + })('CLAUDE.md') + expect(calls).toBe(1) + expect(outcome).toEqual({ succeeded: false, missing: false, error: 'transport down' }) + }) + it('reports a box-level NotFoundError (resourceType names the sandbox) as unreadable, never missing', async () => { const boxGone = Object.assign(new Error('sandbox sb-1 not found'), { name: 'NotFoundError', diff --git a/src/runtime/surface-diff.ts b/src/runtime/surface-diff.ts index a88fbe59..9dec7a3e 100644 --- a/src/runtime/surface-diff.ts +++ b/src/runtime/surface-diff.ts @@ -200,6 +200,7 @@ export interface SurfaceReadBox { fs: { read(path: string): Promise } } +/** Retry and cancellation controls for {@link boxSurfaceReader}. */ export interface BoxSurfaceReaderOptions { /** Read attempts per path before settling on a failed outcome. The data plane can transiently * 404 a just-written file (the same blip `openSandboxRun`'s deliverable read retries for), and a @@ -251,12 +252,15 @@ export function boxSurfaceReader( err && typeof err === 'object' && 'resourceType' in err ? String((err as { resourceType: unknown }).resourceType) : undefined - // A NotFoundError naming a non-file resource (Sandbox / session) means the TRANSPORT is - // gone — reporting it as `missing` would read as "every surface was removed". - const fileNotFound = notFound && (resourceType === undefined || /file|path/i.test(resourceType)) + // A NotFoundError naming the BOX or the session means the TRANSPORT is gone — calling that + // `missing` would read as "the agent deleted every surface". Anything else is the file: the + // SDK's HTTP mapper defaults `resourceType` to `'Resource'` when the server does not name one, + // so matching only file-ish names would classify an ordinary deletion as `unreadable` and the + // harvest would never report `removed`. + const transportGone = resourceType !== undefined && /sandbox|session/i.test(resourceType) return { succeeded: false, - missing: fileNotFound, + missing: notFound && !transportGone, error: err instanceof Error ? err.message : String(err), } } From 0327f7f77589f1c55bfd2a6c290b47bf1eb473ae Mon Sep 17 00:00:00 2001 From: Drew Stone Date: Wed, 12 Aug 2026 18:33:44 -0600 Subject: [PATCH 09/10] fix(kernel): resolve symlinks before trusting the fsSurfaceReader root boundary MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lexical containment let a symlink planted inside the root read host bytes through a contained-looking name: `readFile` follows the link, so the harvest could hash a file outside the worktree — the existence/hash oracle the contract says it prevents. Check containment twice: once lexically, then again on the link-resolved path. The root is link-resolved too, so a worktree or temp dir reached through a link still reads normally. --- docs/api/runtime.md | 6 ++++-- src/runtime/surface-diff.test.ts | 25 ++++++++++++++++++++++- src/runtime/surface-diff.ts | 35 ++++++++++++++++++++------------ 3 files changed, 50 insertions(+), 16 deletions(-) diff --git a/docs/api/runtime.md b/docs/api/runtime.md index 13a799f0..340f8da4 100644 --- a/docs/api/runtime.md +++ b/docs/api/runtime.md @@ -26372,8 +26372,10 @@ A [SurfaceReader](#surfacereader) over the local filesystem, for worktree/local relative or absolute — must resolve INSIDE `root`: a path that escapes it (`../`, an absolute path elsewhere) fails as a contained non-missing outcome rather than reading outside the worktree, so a persisted or mistyped manifest path cannot turn the harvest into an -existence/hash oracle over the host filesystem. Absence maps to `missing: true`; every other -failure carries the error message. +existence/hash oracle over the host filesystem. Containment is checked twice — once on the +lexical path, then again on the symlink-resolved path, because `readFile` follows a link and a +link planted inside the root would otherwise read host bytes through a contained-looking name. +Absence maps to `missing: true`; every other failure carries the error message. #### Parameters diff --git a/src/runtime/surface-diff.test.ts b/src/runtime/surface-diff.test.ts index 586f797b..a3655a55 100644 --- a/src/runtime/surface-diff.test.ts +++ b/src/runtime/surface-diff.test.ts @@ -1,5 +1,5 @@ import { createHash } from 'node:crypto' -import { mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { mkdtempSync, rmSync, symlinkSync, writeFileSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' import { afterAll, describe, expect, it } from 'vitest' @@ -347,6 +347,29 @@ describe('fsSurfaceReader', () => { expect(insideAbsolute.succeeded).toBe(true) }) + it('refuses a symlink planted inside the root that points at a host file', async () => { + const outside = mkdtempSync(join(tmpdir(), 'surface-diff-outside-')) + const secret = join(outside, 'host-secret.txt') + writeFileSync(secret, 'host bytes') + symlinkSync(secret, join(root, 'looks-contained.md')) + const read = fsSurfaceReader(root) + const outcome = await read('looks-contained.md') + expect(outcome).toMatchObject({ succeeded: false, missing: false }) + if (!outcome.succeeded) expect(outcome.error).toContain('outside the reader root') + rmSync(outside, { recursive: true, force: true }) + }) + + it('still reads a file reached through a symlinked root', async () => { + const realRoot = mkdtempSync(join(tmpdir(), 'surface-diff-real-')) + const linkedRoot = join(mkdtempSync(join(tmpdir(), 'surface-diff-link-')), 'root-link') + writeFileSync(join(realRoot, 'inside.md'), 'contained') + symlinkSync(realRoot, linkedRoot) + const outcome = await fsSurfaceReader(linkedRoot)('inside.md') + expect(outcome.succeeded).toBe(true) + if (outcome.succeeded) expect(new TextDecoder().decode(outcome.value)).toBe('contained') + rmSync(realRoot, { recursive: true, force: true }) + }) + it('composes with the harvest over a real worktree edit', async () => { writeFileSync(join(root, 'skill.md'), 'v1') const mounts = [mount('skill.md', 'v1')] diff --git a/src/runtime/surface-diff.ts b/src/runtime/surface-diff.ts index 9dec7a3e..92024bd8 100644 --- a/src/runtime/surface-diff.ts +++ b/src/runtime/surface-diff.ts @@ -24,7 +24,7 @@ */ import { createHash } from 'node:crypto' -import { readFile } from 'node:fs/promises' +import { readFile, realpath } from 'node:fs/promises' import { isAbsolute, resolve, sep } from 'node:path' import { readBoxPathWithRetry } from './box-read-retry' import type { MountManifestEntry } from './types' @@ -271,22 +271,31 @@ export function boxSurfaceReader( * relative or absolute — must resolve INSIDE `root`: a path that escapes it (`../`, an absolute * path elsewhere) fails as a contained non-missing outcome rather than reading outside the * worktree, so a persisted or mistyped manifest path cannot turn the harvest into an - * existence/hash oracle over the host filesystem. Absence maps to `missing: true`; every other - * failure carries the error message. + * existence/hash oracle over the host filesystem. Containment is checked twice — once on the + * lexical path, then again on the symlink-resolved path, because `readFile` follows a link and a + * link planted inside the root would otherwise read host bytes through a contained-looking name. + * Absence maps to `missing: true`; every other failure carries the error message. */ export function fsSurfaceReader(root: string): SurfaceReader { - const boundary = resolve(root) + const lexicalRoot = resolve(root) + let resolvedRoot: string | undefined + const escapes = (candidate: string, boundary: string): boolean => + candidate !== boundary && !candidate.startsWith(boundary + sep) + const outside = (path: string, boundary: string): SurfaceReadOutcome => ({ + succeeded: false, + missing: false, + error: `fsSurfaceReader: path ${JSON.stringify(path)} resolves outside the reader root ${JSON.stringify(boundary)}`, + }) return async (path) => { - const target = isAbsolute(path) ? resolve(path) : resolve(boundary, path) - if (target !== boundary && !target.startsWith(boundary + sep)) { - return { - succeeded: false, - missing: false, - error: `fsSurfaceReader: path ${JSON.stringify(path)} resolves outside the reader root ${JSON.stringify(boundary)}`, - } - } + const target = isAbsolute(path) ? resolve(path) : resolve(lexicalRoot, path) + if (escapes(target, lexicalRoot)) return outside(path, lexicalRoot) try { - return { succeeded: true, value: new Uint8Array(await readFile(target)) } + // The root itself may be reached through a link (a worktree, a temp dir); compare + // link-resolved against link-resolved so a legitimate mount is not read as an escape. + resolvedRoot ??= await realpath(lexicalRoot) + const resolvedTarget = await realpath(target) + if (escapes(resolvedTarget, resolvedRoot)) return outside(path, resolvedRoot) + return { succeeded: true, value: new Uint8Array(await readFile(resolvedTarget)) } } catch (err) { const code = (err as NodeJS.ErrnoException).code return { From 5764424619f2eaa7a577692087c7439401199f43 Mon Sep 17 00:00:00 2001 From: Drew Stone Date: Wed, 12 Aug 2026 18:56:23 -0600 Subject: [PATCH 10/10] fix(kernel): keep a vanished workspace and an aborted run out of the removed evidence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two ways the harvest could invent agent deletions: A missing or unreadable reader root made every mount report `removed` — the filesystem twin of a dead box answering for each path. The root is now classified once: gone means `unreadable`. The retry guard also sat after the read, so an already-cancelled run still spent an attempt and `beforeAttempt` ran twice for the same failed attempt. Move the guard between `beforeAttempt` and the read: one call per attempt, no read on an abandoned run, and an abort landing during the final attempt again returns the turn with its `readError` instead of throwing. --- src/runtime/box-read-retry.test.ts | 40 ++++++++++++++++++++++++++++++ src/runtime/box-read-retry.ts | 15 ++++++----- src/runtime/surface-diff.test.ts | 13 ++++++++++ src/runtime/surface-diff.ts | 10 ++++++++ tests/runtime/sandbox-run.test.ts | 21 ++++++++++++++++ 5 files changed, 93 insertions(+), 6 deletions(-) diff --git a/src/runtime/box-read-retry.test.ts b/src/runtime/box-read-retry.test.ts index 2773f65d..5eed0ed7 100644 --- a/src/runtime/box-read-retry.test.ts +++ b/src/runtime/box-read-retry.test.ts @@ -44,6 +44,46 @@ describe('readBoxPathWithRetry', () => { expect(calls).toBe(1) }) + it('spends no read at all when the signal is already aborted', async () => { + let calls = 0 + const controller = new AbortController() + controller.abort() + const result = await readBoxPathWithRetry( + () => { + calls += 1 + return Promise.resolve('never reached') + }, + 'a.md', + { attempts: 3, delayMs: 0, signal: controller.signal }, + ) + expect(calls).toBe(0) + expect(result.succeeded).toBe(false) + if (!result.succeeded) + expect(boxReadErrorMessage(result.error)).toContain('aborted before reading') + }) + + it('runs beforeAttempt exactly once per attempt, including the aborted one', async () => { + const controller = new AbortController() + const seen: (string | undefined)[] = [] + await readBoxPathWithRetry( + () => { + controller.abort() + return Promise.reject(new Error('transient')) + }, + 'a.md', + { + attempts: 4, + delayMs: 0, + signal: controller.signal, + beforeAttempt: (lastError) => { + seen.push(boxReadErrorMessage(lastError)) + }, + }, + ) + // One call before the read that failed, one before the attempt the abort cancels. + expect(seen).toEqual([undefined, 'transient']) + }) + it('treats a sub-1 attempt count as a single attempt rather than skipping the read', async () => { let calls = 0 const read = () => { diff --git a/src/runtime/box-read-retry.ts b/src/runtime/box-read-retry.ts index 1787d02b..30bab3d0 100644 --- a/src/runtime/box-read-retry.ts +++ b/src/runtime/box-read-retry.ts @@ -49,18 +49,21 @@ export async function readBoxPathWithRetry( let lastError: unknown for (let attempt = 1; attempt <= attempts; attempt += 1) { options.beforeAttempt?.(lastError) + // An abandoned run earns no further reads. The guard sits AFTER `beforeAttempt` so a caller + // that raises its own abort error keeps that authority, and BEFORE the read so an already + // cancelled run spends nothing. + if (options.signal?.aborted) { + lastError ??= new Error( + `readBoxPathWithRetry: aborted before reading ${JSON.stringify(path)}`, + ) + break + } try { return { succeeded: true, text: await read(path) } } catch (err) { lastError = err if (attempt < attempts && options.delayMs > 0) await sleep(options.delayMs * attempt, options.signal) - // An abandoned run earns no further attempts. `beforeAttempt` still runs first, so a caller - // that must raise its own abort error keeps that authority instead of getting a bare result. - if (options.signal?.aborted) { - options.beforeAttempt?.(lastError) - break - } } } return { succeeded: false, error: lastError } diff --git a/src/runtime/surface-diff.test.ts b/src/runtime/surface-diff.test.ts index a3655a55..81462a93 100644 --- a/src/runtime/surface-diff.test.ts +++ b/src/runtime/surface-diff.test.ts @@ -370,6 +370,19 @@ describe('fsSurfaceReader', () => { rmSync(realRoot, { recursive: true, force: true }) }) + it('reports a vanished worktree root as unreadable, never as every mount removed', async () => { + const doomed = mkdtempSync(join(tmpdir(), 'surface-diff-doomed-')) + writeFileSync(join(doomed, 'a.md'), 'v1') + const read = fsSurfaceReader(doomed) + rmSync(doomed, { recursive: true, force: true }) + const diffs = await harvestSurfaceDiffs({ + mounts: [mount('a.md', 'v1'), mount('b.md', 'v1')], + read, + }) + expect(diffs.map((d) => d.status)).toEqual(['unreadable', 'unreadable']) + expect(diffs[0]?.error).toContain('reader root') + }) + it('composes with the harvest over a real worktree edit', async () => { writeFileSync(join(root, 'skill.md'), 'v1') const mounts = [mount('skill.md', 'v1')] diff --git a/src/runtime/surface-diff.ts b/src/runtime/surface-diff.ts index 92024bd8..49a57185 100644 --- a/src/runtime/surface-diff.ts +++ b/src/runtime/surface-diff.ts @@ -293,6 +293,16 @@ export function fsSurfaceReader(root: string): SurfaceReader { // The root itself may be reached through a link (a worktree, a temp dir); compare // link-resolved against link-resolved so a legitimate mount is not read as an escape. resolvedRoot ??= await realpath(lexicalRoot) + } catch (err) { + // The WORKSPACE is gone, not the surface. Calling this `missing` would report every mount as + // a deliberate agent deletion — the filesystem twin of a dead box answering for each path. + return { + succeeded: false, + missing: false, + error: `fsSurfaceReader: reader root ${JSON.stringify(lexicalRoot)} is unreadable (${err instanceof Error ? err.message : String(err)})`, + } + } + try { const resolvedTarget = await realpath(target) if (escapes(resolvedTarget, resolvedRoot)) return outside(path, resolvedRoot) return { succeeded: true, value: new Uint8Array(await readFile(resolvedTarget)) } diff --git a/tests/runtime/sandbox-run.test.ts b/tests/runtime/sandbox-run.test.ts index 0f7d22d6..2fddeefd 100644 --- a/tests/runtime/sandbox-run.test.ts +++ b/tests/runtime/sandbox-run.test.ts @@ -574,6 +574,27 @@ describe('openSandboxRun — abort-aware artifact read preserves the partial tra // One read attempted, then the post-backoff guard short-circuits the retries. expect(reads).toBe(1) }) + + it('returns the turn with its readError when every read attempt fails without an abort', async () => { + let reads = 0 + const { client } = createFakeClient({ + events: [{ type: 'result', data: { ok: true } } as SandboxEvent], + fsRead: () => { + reads += 1 + throw new Error('Resource not found: never flushed') + }, + }) + const run = await openSandboxRun( + client, + { agentRun: spec(), signal: new AbortController().signal, readRetryDelayMs: 0 }, + artifactDeliverable('solution.patch'), + ) + const turn = await run.start('write the patch') + // The retry budget is spent, then the failure is REPORTED rather than thrown, so a caller + // still distinguishes "produced nothing" from a read fault. + expect(reads).toBe(4) + expect(turn.readError).toMatch(/never flushed/) + }) }) describe('openSandboxRun — runtime hooks', () => {