evals: consume agent mounts generically in the claude_code tool adapter

[shriyatheunicorn]

Replaces the per-surface logic inside the adapter with one generic
mount consumer: any surface declaring an agent mount gets its handles
mounted as the single local-MCP run tool, with the surface's own prompt
instructions. Net deletion — the bespoke playwright_code/cdp_code blocks
move behind the contract, and adding a surface no longer means editing
the adapter.

Part of the nondeterministic-evals stack (#2591 → #2611). Requires the
three mount declarations below it.

Verified across the stack: tsc and oxlint per part, evals vitest, live
core runs on Browserbase for --tool stagehand_code, playwright_code, and
cdp_code (3/3 each, prompt exit), one long-horizon row (webvoyager x
claude_code x stagehand_code, 3/3), and the paired webvoyager acceptance
run (stagehand_code 2/3 vs playwright_code 3/3, both cells traced to
Braintrust).

---

Summary by cubic

Consume LLM exposure mounts generically in the claude_code adapter so all code surfaces expose their handles as a single MCP run tool. This aligns with STG-2671 and adds optional evidence capture while removing surface-specific logic.

• New Features

  • Generic agent mount consumption across playwright_code, cdp_code, and stagehand_code; prompt text, run-tool copy/deny messages, and task/console bindings come from the mount.
  • Optional captureEvidence() for grading (timeout EVAL_CAPTURE_EVIDENCE_TIMEOUT_MS); standardized identifiers via AGENT_RUN_TOOL_NAME and AGENT_RUN_TOOL_SERVER.

• Refactors

  • One mount consumer replaces bespoke surface code; snippets run with handle names plus startUrl/task/console. Re-exports waitForCdpEvent.
  • Shared tool/target lifecycle via startAgentToolRuntime with idempotent cleanup (timeout EVAL_AGENT_MOUNT_CLEANUP_TIMEOUT_MS); startup profiles: browse_cli/stagehand_code own the browser, playwright_code/cdp_code expect runner-provided CDP. Improved browse_cli build-artifacts error.

^{Written for commit 5a1a4ea. Summary will update on new commits.}

[changeset-bot]

⚠️ No Changeset found

Latest commit: 5a1a4ea

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[cubic-dev-ai]

All reported issues were addressed across 2 files

Architecture diagram

sequenceDiagram
    participant Test as Eval Test
    participant Adapter as ClaudeCodeToolAdapter
    participant Exposure as LLMExposure (code_handles)
    participant Runner as Eval Runner (CDP/Browser)
    participant MCP as Local MCP Server
    participant Surface as Surface Module (v4/playwright/cdp_code)

    Note over Test,Adapter: SURFACE RESOLUTION
    Test->>Adapter: prepareClaudeCodeToolAdapter(toolSurface, startupProfile)
    alt browse_cli
        Adapter->>Adapter: prepareBrowseCliHarnessAdapter()
        Adapter-->>Test: browse_cli adapter
    else playwright_code/cdp_code/v4_code (NEW generic path)
        Adapter->>Surface: prepareLLMExposure(plan, environment, logger, startupProfile)
        Surface-->>Adapter: exposure (kind: "code_handles", handles, prompt, cleanup)
        Adapter->>Adapter: prepareCodeExposureAdapter(exposure, input)
        Note over Adapter,Exposure: VERIFY kind="code_handles" with handles
        Adapter->>Adapter: Create per-surface temp dir (v4/playwright/cdp)
        Adapter->>Adapter: Build AsyncFunction scope over handle names
        Adapter->>MCP: Mount "run" tool with surface handles + startUrl/task/console
        Adapter-->>Test: Generic exposure adapter
    end

    Note over Test,MCP: STARTUP PROFILE RESOLUTION
    alt v4_code (NEW)
        Test->>Adapter: resolveClaudeCodeStartupProfile("v4_code", env)
        Adapter-->>Test: tool_create_browserbase / tool_launch_local
        Note over Adapter,Runner: v4 SDK owns its browser instance
    else playwright_code/cdp_code
        Test->>Adapter: resolveClaudeCodeStartupProfile("playwright_code", env)
        Adapter-->>Test: runner_provided_local_cdp / runner_provided_browserbase_cdp
        Note over Adapter,Runner: Runner provides CDP endpoint
    end

    Note over Test,MCP: RUNTIME EXECUTION
    Test->>Adapter: runTool(toolName, input)
    Adapter->>MCP: Execute code with surface handles
    MCP->>Surface: Call handle functions (browser/page/CDP/console)
    Surface-->>MCP: Results
    MCP-->>Adapter: Tool output
    Adapter-->>Test: Record<string, unknown>

    Note over Test,Adapter: FINAL STATE CAPTURE (NEW optional)
    alt Surface exposure implements captureFinalState
        Test->>Adapter: captureFinalState()
        Adapter->>Surface: captureFinalState() (bounded by EVAL_FINAL_STATE_TIMEOUT_MS)
        Surface-->>Adapter: TerminalArtifact
        Adapter-->>Test: captureFinalState result
    end

    Note over Adapter,Surface: CLEANUP
    Test->>Adapter: cleanup()
    Adapter->>Surface: exposure.cleanup() (bounded by EVAL_EXPOSURE_CLEANUP_TIMEOUT_MS)
    Adapter->>Adapter: Remove per-surface temp dir
    Adapter-->>Test: Cleanup complete

Loading

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

Review completed against the latest diff

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>