You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 42d920a
Browse filesBrowse the repository at this point in the historyBrowse files
feat(Decision): use OpenRouter Decisions API only, document custom decisions
Drop the decision provider option and @ai-sdk/typesafe-ai: all decision
models, including TypeSafe Jev, are served by the OpenRouter Decisions API.
Document how decisions use AI SDK experimental_decide and how to ask
choice/score questions from a custom helper.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: docs/agents.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -186,6 +186,8 @@ I.decide([
186
186
187
187
The agent runs the statement on the live page like any other command and keeps it only if it passes. The test then checks it on every run.
188
188
189
+
To let the agent use decisions, enable the Decision helper and set `OPENROUTER_API_KEY`. See [Decision Assertions](/assertions#decision-assertions) for the config.
190
+
189
191
Decision models like [Jev](https://openrouter.ai/typesafe/jev-1.13) are built for this. They answer in a fraction of a second, cost a fraction of a cent per request, and return a probability instead of free text. That makes them fast and cheap enough to run on every CI build, and predictable enough to keep in a test.
Copy file name to clipboardExpand all lines: docs/ai.md
+43-1Lines changed: 43 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -353,7 +353,49 @@ I.decideVisually('sidebar is shown')
353
353
354
354
It uses a [decision model](https://openrouter.ai/models?output_modalities=decisions) like [Jev](https://openrouter.ai/typesafe/jev-1.13) instead of a chat model. A decision model reads the page and returns the probability that a statement is true. It is fast, costs a fraction of a cent per request, and gives a probability instead of free text, so a step passes or fails on a confidence threshold you set.
355
355
356
-
Configure it in `ai.decisionModel`. Decisions call the decisions API directly, so they don't need `ai.model` or the `--ai` flag. See [Decision Assertions](/assertions#decision-assertions) for setup and usage.
356
+
Decisions are requested through the [OpenRouter Decisions API](https://openrouter.ai/models?output_modalities=decisions) with AI SDK [`experimental_decide`](https://ai-sdk.dev/docs/ai-sdk-core/decisions). Set `OPENROUTER_API_KEY` and configure the model in `ai.decisionModel`. Decisions don't need `ai.model` or the `--ai` flag. See [Decision Assertions](/assertions#decision-assertions) for setup and usage.
357
+
358
+
### Custom Decisions
359
+
360
+
`I.decide` asks yes/no questions. AI SDK decisions also support `choice` and `score` questions. Use [`experimental_decide`](https://ai-sdk.dev/docs/ai-sdk-core/decisions) in a [custom helper](/helpers) to ask them about the current page:
criteria: { login:'sign in form', dashboard:'user dashboard', error:'error message' },
383
+
},
384
+
},
385
+
})
386
+
387
+
returnanswers.page.choice
388
+
}
389
+
}
390
+
391
+
exportdefaultPageDecisions
392
+
```
393
+
394
+
```js
395
+
constkind=awaitI.grabPageKind()
396
+
```
397
+
398
+
`ai` and `@openrouter/ai-sdk-provider` are installed with CodeceptJS. A `score` question returns a fractional position on a list of levels, and a `boolean` question returns a probability. See [AI SDK decisions](https://ai-sdk.dev/docs/ai-sdk-core/decisions) for all question types.
|`apiKey`|| API key, overrides the environment variable |
429
+
|`apiKey`|| OpenRouter API key, overrides `OPENROUTER_API_KEY`|
431
430
|`model`|`typesafe/jev-1.13`| model for `I.decide`|
432
-
|`visualModel`|`cloudflare/clef`| model with image input for `I.decideVisually`, OpenRouter only|
431
+
|`visualModel`|`cloudflare/clef`| model with image input for `I.decideVisually`|
433
432
|`confidence`|`0.7`| minimal probability for a statement to pass |
434
433
|`timeout`|`15000`| request timeout in ms |
435
434
|`maxLength`|`12000`| maximal length of ARIA snapshot or HTML sent to the model |
436
435
437
-
Decisions don't need the `--ai` flag, and `ai.model` is not required.
436
+
Any model from the [OpenRouter decision models](https://openrouter.ai/models?output_modalities=decisions) list can be used. Decisions don't need the `--ai` flag, and `ai.model` is not required.
if(!this.provider)thrownewError(`Unknown decision provider "${provider}" in ai.decisionModel, use one of: ${Object.keys(DECISION_PROVIDERS).join(', ')}`)
289
+
const{ confidence }=this.config
299
290
if(!(confidence>0&&confidence<1))thrownewError(`ai.decisionModel.confidence must be between 0 and 1, got ${confidence}`)
0 commit comments