Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ This integration is enabled by default when <PlatformLink to="/tracing/">tracing

The `typesafeIntegration` adds instrumentation for [`@typesafe-ai/sdk`](https://www.npmjs.com/package/@typesafe-ai/sdk) to capture <PlatformLink to="/agent-tracing/">agent tracing</PlatformLink> `gen_ai.evaluate` spans, recording model, token usage, latency, and (when enabled) evaluation inputs and answers.

If you call Jev through Vercel AI SDK's `experimental_evaluate()`, see the [Vercel AI integration](/platforms/javascript/guides/node/configuration/integrations/vercelai/#trace-jev-evaluations) instead.
If you call Jev through Vercel AI SDK's `experimental_evaluate()`, see the [Vercel AI integration](/platforms/javascript/guides/node/configuration/integrations/vercelai/#trace-jev-evaluations) instead. If you call Jev through Cloudflare Workers AI, see [Workers AI](/platforms/javascript/guides/cloudflare/agent-tracing/workers-ai/#trace-jev-evaluations).

```javascript
Sentry.init({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,39 @@ Only `env.AI.run` is instrumented. Streaming spans end when the stream is fully

You can browse the full list of available models in the [Cloudflare Workers AI models directory](https://developers.cloudflare.com/ai/models/).

## Trace Jev Evaluations

<AvailableSince version="11.3.0" />

Calls to TypeSafe's [Jev model](https://developers.cloudflare.com/ai/models/typesafe/jev/) (`typesafe/jev`) through `env.AI.run()` are traced automatically. No additional integration or setup is needed:

```typescript
export default {
async fetch(request, env) {
// env.AI is automatically instrumented
const result = await env.AI.run("typesafe/jev", {
state: "I was charged twice. Please refund the extra charge.",
questions: {
department: {
type: "choice",
instructions: "Which team should handle this?",
criteria: {
billing: "Charges and refunds",
technical: "Bugs and outages",
},
},
},
});

return new Response(JSON.stringify(result));
},
} satisfies ExportedHandler<Env>;
```

Each call creates a `gen_ai.evaluate` span named `evaluate <model>`, with the response model (for example, `jev-1.13.0`), latency, and token usage. When recording is enabled, Sentry captures `state` and `questions` as input messages and `answers` as output messages, including probabilities and confidence values returned by Jev. These spans have the same attributes as spans from the [TypeSafe integration](/platforms/javascript/guides/node/configuration/integrations/typesafe/) on other runtimes.

Evaluation state and answers can contain sensitive data. To stop recording them, set `genAI: { inputs: false, outputs: false }` in the <PlatformLink to="/configuration/options/#dataCollection">`dataCollection` option</PlatformLink>.

## Next Steps

- Group multi-turn chats with <PlatformLink to="/agent-tracing/#tracking-conversations">`setConversationId()`</PlatformLink>
Expand Down
Loading