Skip to content

fix(server-utils): Don't capture AI client errors as unhandled at the instrumentation level - #23024

Open
lux-in-tenebris-lucet wants to merge 1 commit into
getsentry:developfrom
lux-in-tenebris-lucet:fix/ai-instrumentation-error-capture
Open

fix(server-utils): Don't capture AI client errors as unhandled at the instrumentation level#23024
lux-in-tenebris-lucet wants to merge 1 commit into
getsentry:developfrom
lux-in-tenebris-lucet:fix/ai-instrumentation-error-capture

Conversation

@lux-in-tenebris-lucet

Copy link
Copy Markdown

The exported instrumentOpenAiClient, instrumentAnthropicAiClient and instrumentGoogleGenAIClient wrappers called captureException(..., { mechanism: { handled: false } }) and then rethrew. The SDK decided the error was an unhandled crash before the application's retry or fallback logic ran. A call that succeeded on retry still reported a crash, and every retry reported another one.

This applies the convention already established for the channel-based OpenAI integration in #21877, letting the error bubble to the boundary that knows the outcome, to the manual client instrumentation. On the edge and serverless runtimes that is the only available path.

Fixes #23023

Why this matters beyond the flag

handled feeds release health, so the effect lands on numbers teams act on rather than on a label.

Crash-free rate stops describing reality. An unhandled error marks the session crashed, so a 429 the app retried successfully costs the same crash-free rate as a real crash. Deploy during a provider blip and release health flags the new version as a regression. Rolling back a good deploy is the expensive part.

On-call gets paged for failures the app already absorbed. Volume-threshold alerts fire on retry storms that never reached a user. Unhandled issues are auto-prioritized, so recovered transient failures outrank real bugs in triage.

Provider outages are when this hurts most. One retried request bills 3 to 5 events instead of 0. During an incident that spike is large, and orgs near their quota start dropping real errors, so an OpenAI outage costs teams visibility into unrelated production bugs.

The same failure is reported twice with conflicting metadata. The wrapper captures and rethrows, so when the error really is fatal the application boundary captures it again. dedupeIntegration is not in the Node defaults, so both land: one handled: false, one with the caller's actual status.

AI apps hit this harder than other integrations because the multipliers are standard practice here. Retry with backoff against providers that rate-limit aggressively. Fallback chains across models and vendors. Agent loops making 10 to 30 calls per user action. Errors that are ordinary control flow rather than faults, like context_length_exceeded handled by truncating and retrying, or a content-policy refusal the app renders as a normal message.

Teams who cannot fix the noise tend to remove its source. Muting the issue is one thing, but dropping the AI integration to stop false crash reports gives up the gen-AI spans, token accounting and latency data with it.

Reasoning

Removing the capture rather than adding an option. An opt-out (captureErrors: false, as @sentry/cloudflare has) would work, but #21877 chose removal for the channel-based path and framed it as matching the DB/cache channel subscribers. An option here would leave the two OpenAI paths behaving differently and add API surface the v11 direction is shedding. Error span status is unchanged, so the failure is still visible in tracing.

Scoped to errors that are rethrown to the caller. The captures that remain are the ones where a provider reports an error as data on an otherwise successful call: Anthropic's error stream events and error-shaped responses (isErrorEvent, handleResponseError), and Google GenAI's blocked-content signal. The caller never observes those as a thrown error, so removing them would silently drop the signal. The distinction is load-bearing rather than a judgement call. Removing the .ignore('event') masks left exactly those three tests emitting events, and they are the three that keep the mask with a comment explaining why.

mechanismType threading removed. It existed only to label the capture in createWithResponseWrapper, so it is dropped from wrapPromiseWithMethods and its four call sites. The .catch() in that helper stays, because it is what keeps a .withResponse() rejection from becoming an unhandled rejection when the instrumented promise rejects first. Its comment is rewritten to say so now that the capture no longer implies it.

Integration tests now assert the absence. The AI suites previously dropped these events with .ignore('event'), which is why the behavior went unnoticed. Those masks are removed so the suites fail if instrumentation-level capture comes back.

Root cause

The capture sat inside the wrapped method, upstream of every application-level decision about the error. Rethrowing after capturing means a fatal failure is recorded twice and a retried one is recorded once per attempt.

… instrumentation level

The exported instrumentOpenAiClient, instrumentAnthropicAiClient and
instrumentGoogleGenAIClient wrappers captured provider errors with
mechanism.handled = false and then rethrew, so the SDK classified the
error as an unhandled crash before the application's retry or fallback
logic ran. A call that succeeded on retry still produced an unhandled
event, and each retry produced another one.

Applies the convention established for the channel-based OpenAI
integration in getsentry#21877 to the manual client instrumentation, which is the
only available path on the edge and serverless runtimes. Error span
status and the original error identity are unchanged.

Captures are kept where a provider reports an error as data on an
otherwise successful call, since the caller never sees those as a thrown
error. The AI integration suites no longer mask these events with
.ignore('event'), so they fail if the capture returns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lux-in-tenebris-lucet
lux-in-tenebris-lucet marked this pull request as ready for review August 4, 2026 14:35
@lux-in-tenebris-lucet
lux-in-tenebris-lucet requested review from a team as code owners August 4, 2026 14:35
@lux-in-tenebris-lucet
lux-in-tenebris-lucet requested review from isaacs and mydea and removed request for a team August 4, 2026 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AI client instrumentation reports caller-handled provider errors as unhandled, once per retry

2 participants