Skip to content

feat(server-utils): Record AI SDK v7 runtime and tool context on Vercel AI spans - #25774

Open
isaacs wants to merge 3 commits into
developfrom
isaacs/js-3788
Open

isaacs wants to merge 3 commits into
developfrom
isaacs/js-3788

Conversation

@isaacs

@isaacs isaacs commented Oct 27, 2026

Copy link
Copy Markdown
Member

On ai 7 the telemetry.metadata option is gone. Per-call metadata goes in runtimeContext, and the app opts keys into telemetry with telemetry.includeRuntimeContext (and includeToolsContext for tool context). The channel subscriber did not read either field, so there was no way to put request-level data on gen_ai spans.

The subscriber now records each included runtimeContext key as vercel.ai.settings.context.<key> on the operation span and on its generate_content and execute_tool child spans. It records each included toolContext key as vercel.ai.toolCall.context.<key> on the execute_tool span. The runtimeContext names match @ai-sdk/otel (ai.settings.context.*) under the usual ai.* to vercel.ai.* renaming.

ai before 7.0.114 published the full context to the channel and ignored the include lists. A filtered context looks the same as an unfiltered one, so the subscriber only reads context from events that carry aiVersion, which ai added in 7.0.154. Older 7.x versions record no context, so keys the app did not opt in never leave it.

Fixes #24706
Fixes JS-3788

…el AI spans

On `ai` 7 the `telemetry.metadata` option is gone. Per-call metadata
goes in `runtimeContext`, and the app opts keys into telemetry with
`telemetry.includeRuntimeContext` (and `includeToolsContext` for tool
context). The channel subscriber did not read either field, so there
was no way to put request-level data on gen_ai spans.

The subscriber now records each included `runtimeContext` key as
`vercel.ai.settings.context.<key>` on the operation span and on its
`generate_content` and `execute_tool` child spans. It records each
included `toolContext` key as `vercel.ai.toolCall.context.<key>` on
the `execute_tool` span. The `runtimeContext` names match
`@ai-sdk/otel` (`ai.settings.context.*`) under the usual `ai.*` to
`vercel.ai.*` renaming.

`ai` before 7.0.114 published the full context to the channel and
ignored the include lists. A filtered context looks the same as an
unfiltered one, so the subscriber only reads context from events that
carry `aiVersion`, which `ai` added in 7.0.154. Older 7.x versions
record no context, so keys the app did not opt in never leave it.

Fixes #24706
Fixes JS-3788

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@isaacs
isaacs requested a review from a team as a code owner October 27, 2026 17:13
@isaacs
isaacs requested review from andreiborza and mydea and removed request for a team October 27, 2026 17:13
@linear-code

linear-code Bot commented Oct 27, 2026

Copy link
Copy Markdown

JS-3788

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

size-limit report 📦

Compared: 8594b78 (PR test merge using develop at b722d3c)

Against: b722d3c (develop) · base run #64026

✅ Baseline matches: this diff contains only this PR's changes.

Path Size % Change Change
@sentry/browser 30.39 kB - -
@sentry/browser - with treeshaking flags 28.5 kB - -
@sentry/browser - with treeshaking flags tracing without tracing 28.4 kB - -
@sentry/browser (incl. Tracing) 52.66 kB - -
@sentry/browser (incl. Tracing + Span Streaming) 52.66 kB - -
@sentry/browser (incl. Tracing, Profiling) 55.59 kB - -
@sentry/browser (incl. Tracing, Replay) 92.89 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 81.5 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 97.63 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 110.6 kB - -
@sentry/browser (incl. Feedback) 47.95 kB - -
@sentry/browser (incl. sendFeedback) 35.47 kB - -
@sentry/browser (incl. FeedbackAsync) 40.57 kB - -
@sentry/browser (incl. Metrics) 31.4 kB - -
@sentry/browser (incl. Logs) 31.68 kB - -
@sentry/browser (incl. Metrics & Logs) 32.36 kB - -
@sentry/react 32.2 kB - -
@sentry/react (incl. Tracing) 54.94 kB - -
@sentry/vue 38.67 kB - -
@sentry/vue (incl. Tracing) 55.55 kB - -
@sentry/svelte 30.42 kB - -
@sentry/remix (Remix 3 client bundle) 57.71 kB - -
CDN Bundle 32.11 kB - -
CDN Bundle (incl. Tracing) 53.14 kB - -
CDN Bundle (incl. Logs, Metrics) 34.31 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) 55.09 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) 75.76 kB - -
CDN Bundle (incl. Tracing, Replay) 91.3 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) 93.26 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 97.44 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) 99.44 kB - -
CDN Bundle - uncompressed 94.47 kB - -
CDN Bundle (incl. Tracing) - uncompressed 157.46 kB - -
CDN Bundle (incl. Logs, Metrics) - uncompressed 101.02 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed 163.43 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed 232.57 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 279.15 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed 285.1 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 292.85 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed 298.79 kB - -
@sentry/nextjs (client) 57.79 kB - -
@sentry/sveltekit (client) 53.11 kB - -
@sentry/core/server 44.34 kB - -
@sentry/core/browser 13.95 kB - -
@sentry/node 155.25 kB +0.11% +165 B 🔺
@sentry/node/import (ESM hook with diagnostics-channel injection) 81.67 kB - -
@sentry/node - without tracing 95.84 kB +0.03% +23 B 🔺
@sentry/node - without channel injection 135.86 kB +0.11% +148 B 🔺
@sentry/aws-serverless 104.01 kB +0.03% +25 B 🔺
@sentry/cloudflare (withSentry) - minified 198.09 kB - -
@sentry/cloudflare (withSentry) 492.11 kB - -
@sentry/nextjs/cloudflare (withSentry) - minified 218.68 kB - -

View base workflow run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7dcc279. Configure here.

Comment thread packages/server-utils/test/integrations/vercel-ai/runtime-context.test.ts Outdated
Comment thread dev-packages/node-integration-tests/suites/tracing/vercelai/v6_v7/test.ts Outdated
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Optional E2E tests

Some optional checks were not successful. They do not block the merge. 1 not successful, 66 successful. See the run.

Check Status
❌ CI: Optional E2E Tests / E2E node-anthropic-send-to-sentry (latest) Test (optional) Failing after 3m Details
Successful checks
Check Status
✅ CI: Optional E2E Tests / E2E cloudflare-agent Test (optional) Successful in 26s Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (canary, turbopack) Test (optional) Successful in 4m Details
✅ CI: Optional E2E Tests / E2E nextjs-15 (latest) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E cloudflare-pi-durable Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E cloudflare-mastra Test (optional) Successful in 39s Details
✅ CI: Optional E2E Tests / E2E cloudflare-think Test (optional) Successful in 48s Details
✅ CI: Optional E2E Tests / E2E cloudflare-pi-durable (latest) Test (optional) Successful in 24s Details
✅ CI: Optional E2E Tests / E2E nextjs-15 (turbo) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E debug-id-sourcemaps Test (optional) Successful in 26s Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (deno) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (canary, webpack) Test (optional) Successful in 5m Details
✅ CI: Optional E2E Tests / E2E cloudflare-workers-send-to-sentry Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nextjs-16-tunnel (cloudflare) Test (optional) Successful in 57s Details
✅ CI: Optional E2E Tests / E2E nextjs-14 (canary) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E cloudflare-think (ai v6) Test (optional) Successful in 50s Details
✅ CI: Optional E2E Tests / E2E gen-ai-libraries Test (optional) Successful in 32s Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (vinext) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (cloudflare, latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (vinext, bun) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-14 (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E astro-7-static Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E angular (canary) Test (optional) Successful in 57s Details
✅ CI: Optional E2E Tests / E2E node-flue-send-to-sentry Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E deno (latest) Test (optional) Successful in 54s Details
✅ CI: Optional E2E Tests / E2E node-pi-durable Test (optional) Successful in 26s Details
✅ CI: Optional E2E Tests / E2E cloudflare-flue-send-to-sentry Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E astro-7 Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-mastra Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E angular (canary) Test (optional) Successful in 58s Details
✅ CI: Optional E2E Tests / E2E node-fastify-6 (tracer provider) Test (optional) Successful in 30s Details
✅ CI: Optional E2E Tests / E2E nextjs-16-standalone (canary, turbopack) Test (optional) Successful in 28s Details
✅ CI: Optional E2E Tests / E2E supabase-nextjs Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E gen-ai-libraries (cloudflare) Test (optional) Successful in 24s Details
✅ CI: Optional E2E Tests / E2E node-openai-decisions-send-to-sentry Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nuxt-5 Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-flue Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-flue (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-express-mcp-v2 Test (optional) Successful in 47s Details
✅ CI: Optional E2E Tests / E2E node-express-send-to-sentry Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nuxt-4 (canary) Test (optional) Successful in 3m Details
✅ CI: Optional E2E Tests / E2E tanstackstart-react-cloudflare Test (optional) Successful in 21s Details
✅ CI: Optional E2E Tests / E2E node-openai-send-to-sentry (latest) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-anthropic-send-to-sentry Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-fastify-6 Test (optional) Successful in 24s Details
✅ CI: Optional E2E Tests / E2E nuxt-3 (canary) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-pi-durable (latest) Test (optional) Successful in 26s Details
✅ CI: Optional E2E Tests / E2E node-eve (orchestrion) Test (optional) Successful in 41s Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps (client source maps kept) Test (optional) Successful in 29s Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps Test (optional) Successful in 27s Details
✅ CI: Optional E2E Tests / E2E node-express-send-to-sentry-static Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-express-mcp-v1 Test (optional) Successful in 18s Details
✅ CI: Optional E2E Tests / E2E node-mastra (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-orpc Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E react-router-7-framework-instrumentation Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps (static / nuxt generate) Test (optional) Successful in 26s Details
✅ CI: Optional E2E Tests / E2E node-openai-send-to-sentry Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nuxt-4-static (canary) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E solidstart-2 Test (optional) Successful in 41s Details
✅ CI: Optional E2E Tests / E2E node-eve (latest) Test (optional) Successful in 47s Details
✅ CI: Optional E2E Tests / E2E react-send-to-sentry Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-flue-send-to-sentry (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-eve Test (optional) Successful in 46s Details
✅ CI: Optional E2E Tests / E2E node-openai-decisions-send-to-sentry (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E tanstackstart-react-sourcemaps Test (optional) Successful in 21s Details
✅ CI: Optional E2E Tests / E2E vue-3 (no Options API) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E vue-3 (canary) Test (optional) Successful in 57s Details

andreiborza
andreiborza previously approved these changes Oct 28, 2026

@andreiborza andreiborza left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM just one casing change request and adding these to conventions 🙏

Comment on lines +85 to +90
@@ -86,6 +86,8 @@ const VERCEL_AI_OPERATION_ID_ATTRIBUTE = 'vercel.ai.operationId';
const VERCEL_AI_MODEL_PROVIDER_ATTRIBUTE = 'vercel.ai.model.provider';
const VERCEL_AI_SETTINGS_MAX_RETRIES_ATTRIBUTE = 'vercel.ai.settings.maxRetries';
const VERCEL_AI_TELEMETRY_METADATA_ATTRIBUTE_PREFIX = 'vercel.ai.telemetry.metadata.';
const VERCEL_AI_SETTINGS_CONTEXT_ATTRIBUTE_PREFIX = 'vercel.ai.settings.context.';
const VERCEL_AI_TOOL_CALL_CONTEXT_ATTRIBUTE_PREFIX = 'vercel.ai.toolCall.context.';

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add these all to https://github.com/getsentry/sentry-conventions? We are missing all of them.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, before we merge this PR please, in case we need to change them.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread packages/server-utils/src/integrations/vercel-ai/vercel-ai-dc-subscriber.ts Outdated
@andreiborza
andreiborza dismissed their stale review October 28, 2026 16:33

Removing the approval so we don't accidentally merge this before sentry conventions are in.

Rename `vercel.ai.toolCall.context.<key>` to
`vercel.ai.tool_call.context.<key>`, because fixed segments in Sentry
conventions use snake_case. Record every context value as a string, to
match the `string` type of both attributes in sentry-conventions.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@isaacs

isaacs commented Oct 28, 2026

Copy link
Copy Markdown
Member Author

@andreiborza I added a PR to sentry-conventions for the fields being added here, but there are a few others in that file from 3bfeb64 which are also not tracked in sentry-conventions. I think the move is land this and post a follow-up issue to clean those up? What do you think?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

vercelAIIntegration: record AI SDK v7 runtimeContext (filtered by telemetry.includeRuntimeContext) as span attributes

4 participants