[Docs] Make model routing logic clearer - #2642
Conversation
|
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
cubic analysis
All reported issues were addressed across 1 file
Linked issue analysis
Linked issue: STG-2795: Make sure v3 model routing doc updates are covered in v4
| Status | Acceptance criteria | Notes |
|---|---|---|
| ✅ | Explain server-side selection when no model is configured (selection happens per call and picks up new models) | The diff adds prose clarifying that when no model is configured, Browserbase selects a model server-side on every call and picks up new models as they are added. |
| ✅ | Add a table mapping configuration to where inference runs (No model; model no apiKey; model with apiKey; client-side LLM) | The diff inserts a table that maps each configuration case to where inference runs, covering the four listed cases. |
| ✅ | Document routing semantics: routing is per instance (no ‘auto’ model name) and note about per-call overrides | The diff adds a Note stating routing is per instance, there is no "auto" model name, and references per-call overrides. |
| ✅ | Warn that Model Gateway rejects stopSequences and recommend pinning model with provider apiKey when needed | The diff adds a Warning that Model Gateway rejects stopSequences and suggests pinning a model with provider API key when stopSequences are required. |
| ✅ | Add troubleshooting entries for three gateway-related errors (LLM not configured during initialization; model inference requires provider API key or Browserbase session; Gateway does not support stop sequences) with solutions | The diff inserts three Accordion troubleshooting entries each describing the error and providing solutions (launch with apiKey, pass model with apiKey, or supply client-side LLM / drop stopSequences, etc.). |
Architecture diagram
sequenceDiagram
participant App as Stagehand SDK
participant Router as Model Router
participant Browser as Browser Session
participant Gateway as Model Gateway
participant Provider as Provider API
Note over App,Provider: Model routing decision - set per instance, not per call
Note over App,Provider: "auto" is not a model name - omit "model" to route via Gateway
App->>Router: act()/extract()/observe() -> resolveModel(config)
Router->>Browser: check Browserbase session? (apiKey + session ID)
alt No model configured + Browserbase session
Router->>Gateway: route without model field
Gateway->>Gateway: select model server-side per call (picks up new models)
Gateway-->>App: inference result
else No model configured + local browser
Router-->>App: error "an LLM was not configured" (first call, not create())
else model without apiKey + Browserbase session
Router->>Gateway: route with pinned model
Note over Gateway: NEW: stopSequences rejected by Gateway
Gateway-->>App: inference result
else model without apiKey + local browser
Router-->>App: error "model inference requires a provider API key..."
else model with apiKey
Router->>Provider: route straight to provider (bypass Gateway)
Provider-->>App: inference result (stopSequences supported)
else client-side LLM callback
App->>App: run custom LLM callback locally (bypass Gateway)
App-->>App: inference result
end
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
filip-michalsky
left a comment
filip-michalsky
left a comment
There was a problem hiding this comment.
some small adjustments needed otherwise the errors enum is a nice touch!
| When no model is configured, Stagehand routes inference through Model Gateway without a `model` field and Browserbase selects one automatically. An explicit model without a provider-specific API key still routes through Model Gateway, but pins that model instead. | ||
| When no model is configured, Stagehand routes inference through Model Gateway without a `model` field and Browserbase selects one automatically. Selection happens server-side on every call, so your code never pins a model name and picks up new models as Browserbase adds them. | ||
|
|
||
| What you pass decides where a call goes: |
There was a problem hiding this comment.
the "auto" model name is accepted by the Stagehand V3 api service please see here https://github.com/browserbase/core/blob/main/apps/stagehand-api-v3/lib/auto-mode/auto-model-router.ts#L8
There was a problem hiding this comment.
talked offline, v4 is different from v3 api so this is still valid
| </Note> | ||
|
|
||
| <Note> | ||
| Routing is per instance, not per call. There is no `"auto"` model name: every `modelName` carries a provider prefix, so to route a call you omit `model` rather than naming a router. A [per-call override](#per-call-model-overrides) pins that call to a specific model. |
There was a problem hiding this comment.
hmmm... what does it mean "per instance"? we do currently route per call-
mode: "auto"
stagehand.act -> auto
next step in the trajectory
stagehand.extract -> auto
etc.
currently, each step on trajectory fed into act/extract/observe is auto-routed.
There is a very nuanced interplay with caching as well...
There was a problem hiding this comment.
I updated this a bit for clarity. But this is a stagehand "instance" if that clears things up!
const stagehand = await Stagehand.create({
browser,
model: {
modelName: "openai/gpt-5.4-mini",
apiKey: process.env.OPENAI_API_KEY,
},
});
There was a problem hiding this comment.
All reported issues were addressed across 1 file (changes from recent commits).
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2 participants
why
Making sure the recently added instructions in #2408 are also present in the v4 docs
what changed
test plan
Summary by cubic
Clarifies v4 model routing in
packages/docs/v4/configuration/models.mdx, aligning v4 with STG-2795. Adds a routing table, explains per‑call server-side selection whenmodelis omitted, notes Gateway requires Browserbase-hosted browsers and doesn’t supportstopSequences, and expands troubleshooting for missing LLM config, missing Browserbase session or provider key on pinned models, and stop‑sequence errors.Written for commit db0b5c9. Summary will update on new commits.