add _experimental_batch

[seanmcguire12]

why

client-server round trips can be expensive on remote browsers. this PR adds an experimental batch API that runs a callback inside the Stagehand extension service worker. commands issued by that callback route directly to the existing worker runtime, avoiding the client-server round trip between each operation

what changed

• added stagehand._experimental_batch() to the TypeScript and Python SDKs and Stagehand.ExperimentalBatch() to Go
• TS accepts an async JavaScript callback. Python and Go accept self-contained JavaScript source, because callbacks cannot be translated between languages
• installed globalThis.__stagehandRunCallbackBatch in the service worker, which selects the active or explicitly requested page, constructs the callback context, applies the overall timeout, invokes the callback, and returns a JSON result/error envelope
• added a transport-independent StagehandCommandClient interface and moved the public Page, Locator, BrowserContext, Response, clipboard, and WebMCP wrappers onto it
  • regular SDK calls use the remote RPC client, while batch callbacks use an in-browser client backed by the existing RPCRouter
• preserved normal Stagehand routing and validation inside batches, including deep locator resolution, response handles, context page registration, WebMCP wrappers, and Zod protocol validation
• exposed page, context, act, observe, extract, and metrics to callbacks. the callback context intentionally excludes context.close() and does not expose Stagehand lifecycle or recursive batch operations
• added one-batch-at-a-time protection and cooperative overall timeouts. an operation already running in the router may still finish after the caller receives a timeout
• added JSON validation for callback input and output, explicit handling for undefined, and reconstructed worker errors in the calling SDK
• kept host-only conveniences out of the worker contract:
  • screenshot path cannot write to the caller's filesystem;
  • local paths cannot be used for file uploads or init scripts;
  • worker screenshot bytes are Uint8Array rather than a Node Buffer
• added matching TS, Python, & Go examples and documented the TS/Python APIs in the v4 Stagehand reference docs
• updated cross-SDK, example, and docs parity checks to treat exported Go ExperimentalBatch as the equivalent of _experimental_batch in TypeScript and Python

TypeScript:

const result = await stagehand._experimental_batch(
  async ({ page }, input) => {
    await page.goto(input.url);
    return {
      title: await page.title(),
      heading: await page.locator("h1").innerText(),
    };
  },
  { url: "https://example.com" },
  { timeout: 30_000 },
);

Python:

result = await stagehand._experimental_batch(
    """
    async ({ page }, input) => {
      await page.goto(input.url);
      return { title: await page.title() };
    }
    """,
    {"url": "https://example.com"},
    timeout=30_000,
)

Go:

var result struct {
    Title string `json:"title"`
}

err := client.ExperimentalBatch(
    ctx,
    `async ({ page }, input) => {
        await page.goto(input.url)
        return { title: await page.title() }
    }`,
    map[string]any{"url": "https://example.com"},
    &result,
    stagehand.ExperimentalBatchOptions{Timeout: 30 * time.Second},
)

test plan

• Verify the service worker runner routes shared Page and Locator operations through the in-browser RPC router
• Verify context.close() is absent from the callback surface
• Verify callback input is serialized independently from callback source and options
• Verify ordinary JSON results and the distinct undefined result envelope
• Verify the generated CDP expression includes the worker capability guard and uses awaitPromise/returnByValue.
• Verify the TypeScript API is async and forwards callback source, input, page, and timeout
• Verify Python CDP evaluation, static typing, and transport compatibility

---

Summary by cubic

Adds an experimental batch API that runs a trusted JavaScript callback inside the Stagehand service worker to speed up multi-step flows by skipping SDK↔browser round trips. The API is stagehand.experimentalBatch() (TypeScript), stagehand.experimental_batch() (Python), and Stagehand.ExperimentalBatch() (Go); page.screenshot() now returns Uint8Array.

• New Features

  • Run batches via JSON‑RPC stagehand.callback_batch; the CDP transport attaches the executable callback with Runtime.evaluate while using the normal pending RPC path.
  • Batch context exposes { page, context, act, observe, extract, metrics }, returns JSON‑serializable results (preserves undefined), and preserves callback names with a bundled __name helper.
  • AI methods in a batch use the SDK’s active‑page default; options.page only sets batch.page.
  • Service worker: enforces one active batch, blocks internal Stagehand/context APIs, routes ops to the right page, propagates cancellation/timeouts, normalizes extract results, and forwards W3C trace context.
  • Refactor: introduced transport‑independent StagehandCommandClient; public wrappers (Page, Locator, BrowserContext, Response, clipboard, WebMCP) use it for in‑worker execution.
  • Docs and examples added for the new API; docs updated to show page.screenshot() returns Uint8Array.

• Bug Fixes

  • Timeouts: standardized validation across SDKs; include stagehand.callback_batch in the RPC timeout policy; cap batch timeouts at 2_147_483_647 - 10_000 ms.
  • JSON/typing: Python generator preserves JSON integers in recursive schemas; fixed StagehandMetrics typing; all SDKs validate inputs, reject empty/null options and non‑string sources; Python checks for a closed RPC client.
  • Binary/FS: canonical base64 decoder; page.screenshot() returns Uint8Array; disallow path in batches; avoid double‑encoding and count bytes before encoding; throw RangeError for uploads >50MB; Node‑only FS reads are dynamically imported with clearer errors (including init scripts).
  • Robustness: fail fast on invalid worker responses; abort the callback controller after it settles.

^{Written for commit e5a1b1c. Summary will update on new commits.}

[changeset-bot]

⚠️ No Changeset found

Latest commit: e5a1b1c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[seanmcguire12]

@cubic-dev-ai

[cubic-dev-ai]

@cubic-dev-ai

@seanmcguire12 I have started the AI code review. It will take a few minutes to complete.

[cubic-dev-ai]

All reported issues were addressed across 37 files

Architecture diagram

sequenceDiagram
    participant SDK as SDK Client (TS/Py/Go)
    participant CDP as CDP Session (Worker)
    participant SW as Service Worker
    participant Router as RPCRouter
    participant InProc as InProcessCommandClient
    participant Page as Page / Locator Wrappers
    participant Browser as Browser Runtime

    Note over SDK,Browser: NEW: Experimental Batch Flow

    SDK->>CDP: Runtime.evaluate (callback batch expression)
    Note over SDK,CDP: Wraps callback source, serialized input, options with pageId & timeout

    CDP->>SW: globalThis.__stagehandRunCallbackBatch(callback, input, options)
    alt Another batch already active
        SW-->>CDP: { ok: false, error: "Another batch running" }
        CDP-->>SDK: Propagate error
    else Valid batch
        SW->>SW: Mark active, create AbortController, set timeout
        SW->>InProc: new InProcessCommandClient(router, signal)
        SW->>Router: context.pages (or context.active_page)
        Router->>InProc: Return page list / active page
        InProc-->>SW: Page reference

        Note over SW,Page: Callback executes with worker-local objects

        SW->>Page: callback({ page, context, act, observe, extract, metrics }, input)
        Page->>InProc: locator.click({ selector: "button" })
        InProc->>Router: handle({ method: "locator.click", params })
        Router->>Browser: Execute CDP click command
        Browser-->>Router: Result
        Router-->>InProc: { clicked: true }
        InProc-->>Page: Return result
        Page->>InProc: page.title()
        InProc->>Router: handle({ method: "page.title", params })
        Router->>Browser: Execute CDP title command
        Browser-->>Router: "Example Page"
        Router-->>InProc: "Example Page"
        InProc-->>Page: Return title
        Page-->>SW: { title: "Example Page" }

        SW->>SW: JSON round-trip result
        alt Result is undefined
            SW-->>CDP: { ok: true, valueIsUndefined: true }
        else JSON-serializable result
            SW-->>CDP: { ok: true, value: { title: "Example Page" } }
        end
        CDP-->>SDK: Decode envelope, return result
    end

    Note over SDK,SW: Key Constraints<br/>- context.close() excluded from callback scope<br/>- No recursive batch calls<br/>- Overall timeout via AbortController + cooperative check<br/>- CDP uses awaitPromise: true, returnByValue: true

    Note over SDK,CDP: Host-only features excluded in worker:<br/>- screenshot path (no Node fs)<br/>- file upload local paths<br/>- Buffer → Uint8Array conversion

Loading

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 40 files

Architecture diagram

sequenceDiagram
    participant SDK as SDK Client (TS/Python/Go)
    participant CDP as CDP Client / WebSocket
    participant SW as Service Worker
    participant CB as callbackBatch Runner
    participant RPCR as RPCRouter (In-Browser)
    participant Page as Page/Locator Wrapper
    participant Context as BrowserContext

    Note over SDK,Context: NEW: Experimental Batch Flow (avoiding SDK↔Server RTT per operation)

    SDK->>CDP: sendCommand("Runtime.evaluate", { expression, awaitPromise, returnByValue })
    Note over SDK,CDP: Expression wraps callback source + serialized input + options

    CDP->>SW: Runtime.evaluate CDP command (target service worker session)

    alt SW supports batches
        SW->>SW: Check globalThis.__stagehandRunCallbackBatch exists
        SW->>CB: invoke __stagehandRunCallbackBatch(callback, input, options)
        CB->>CB: Check no other batch active (one-batch-at-a-time guard)
        CB->>CB: Create InProcessCommandClient backed by RPCRouter
        CB->>Context: Resolve selected page (by pageId or activePage())
        Context->>RPCR: context.pages / context.active_page
        RPCR-->>Context: Page list / active page ref
        Context-->>CB: Page object
        CB->>CB: Build callback context { page, context, act, observe, extract, metrics }
        Note over CB: context.close() is intentionally excluded
        CB->>CB: Invoke callback(stagehand, input)
        loop Inside callback
            Page->>CB: page.goto(url)
            CB->>RPCR: stagehand.act / stagehand.observe / stagehand.extract
            RPCR-->>CB: Result
            Page->>CB: page.locator("h1").innerText()
            CB->>RPCR: locator.innerText
            RPCR-->>CB: Text value
        end
        alt Result is undefined
            CB-->>SW: { ok: true, valueIsUndefined: true }
        else Result is JSON-serializable
            CB->>CB: JSON round-trip validation
            CB-->>SW: { ok: true, value: <JSON> }
        end
    else SW incompatible
        SW-->>CDP: { ok: false, error: "StagehandRuntimeIncompatibleError" }
    end

    SW-->>CDP: Runtime.evaluate result (envelope JSON)
    CDP->>CDP: Parse CallbackBatchEnvelopeSchema
    alt OK envelope
        alt valueIsUndefined
            CDP-->>SDK: undefined/null
        else has value
            CDP-->>SDK: Decoded result
        end
    else Error envelope
        CDP-->>SDK: Reconstructed Error with name & message
    end

    Note over SDK,SW: Timeout handling: cooperative + CDP evaluation grace period
    opt Timeout expires
        CB->>CB: Abort controller triggers
        CB-->>SW: { ok: false, error: "Stagehand callback batch timed out" }
    end

Loading

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 13 files (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[miguelg719]

Two blocking items before merge (correctness, not security — the architecture is sound). Remaining should-fix / nits to follow.

[miguelg719]

rest of the review — should-fix + nits. none of these block on their own, but the page-pinning and extract heuristic ones are easy footguns.

[cubic-dev-ai]

All reported issues were addressed across 29 files (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>