fix(providers): retry transient failures on OpenAI Responses and Gemini by waleedlatif1 · Pull Request #8383 · simstudioai/sim

waleedlatif1 · 2026-09-28T18:50:19Z

Summary

OpenAI Responses (and Azure Responses endpoints) sent each request once, so a single transient 500 failed the block. Every OpenAI-compatible, Anthropic, and Bedrock client already retried through its SDK — this path and Gemini/Vertex were the gaps
New providers/retry.ts: one retry policy for calls no vendor SDK retries, matching what openai, @anthropic-ai/sdk, and the AI SDK agree on
- PROVIDER_MAX_RETRIES (2) replays with jittered exponential backoff
- retries 408/409/429/5xx and dropped connections (Node TypeError, Bun ECONNRESET)
- obeys x-should-retry, then retry-after-ms, then retry-after
- never retries a caller abort; covers only the wait for headers, so a started stream is never replayed
- retries only real network failures, recognised by the socket error code anywhere in the cause chain (Node's fetch failed carries it on cause; Bun sets ECONNRESET / ConnectionRefused / ConnectionClosed directly), never by message text. A TypeError from building the request is a bug and is not retried
- two deliberate divergences from the SDKs, both so a block's fallback model runs instead of waiting on a failure that will not clear: a 429 reporting an exhausted balance (insufficient_quota) is not retried, and neither is a failure whose requested delay exceeds 30s (a daily quota, a provider-wide pause) — the AI SDK likewise ignores long server delays
- 429 bodies are read directly (bounded, under the caller's signal), not via clone(): cancelling one branch of a tee waits on the other, so an oversized body could hang
- retry log lines carry status and x-request-id, and bound SDK error text to 200 chars
OpenAI Responses uses fetchWithProviderRetry; the payload is prepared once outside the retry, since preparing can compact the conversation with its own model call
Gemini/Vertex wrap generateContent/generateContentStream in withProviderRetry, honoring the google.rpc.RetryInfo delay from the error body (the SDK exposes no headers). The SDK's own retryOptions stays off: it replaces every error (4xx included) with a generic statusText message and ignores the abort signal. Deep research interactions.* is left alone — that client already retries, and wrapping would double it
Anthropic, Azure Anthropic, and Azure OpenAI chat pin maxRetries to PROVIDER_MAX_RETRIES (Azure OpenAI via openAICompatTransport()), so an SDK bump cannot silently move their retry budget. No behavior change: both equal the SDK defaults. Anthropic pins only maxRetries; its SDK derives the timeout from max_tokens
TypeSafe's existing loop now reads the shared status and retry-after helpers
Quota-body classifier moved from embeddings to lib/core/errors/provider-quota.ts (now two consumers)
Generation-context coverage guard accepts a const bound to a prepared payload, and follows the postOnce → post rename (the old name silently dropped the OpenAI body from coverage)

Accepted risk

A transport failure after the server finished a long non-streaming generation re-runs it, the same trade-off every SDK-backed provider already makes. /v1/responses takes no idempotency key. Bun's ~300s socket wall surfaces as TimeoutError, which is not retried, so a runaway generation is never replayed

Follow-ups (separate PRs)

Wand and vision call OpenAI with raw fetch and no retry; wrap them in fetchWithProviderRetry
OpenAI Responses and Gemini have no per-attempt time-to-headers bound (pre-existing)

Type of Change

Bug fix

Testing

providers/retry.test.ts: one test per failure mode (transient status, budget exhaustion, permanent 4xx, exhausted quota, x-should-retry both ways, retry-after-ms, dropped connection, unknown error, abort during backoff, pre-aborted request, SDK status errors); each guard mutated and its test watched go red
OpenAI and Gemini wiring tests, both red with the retry unwired; coverage guard verified to still flag an unprepared Gemini payload and OpenAI body
Every guard in providers/retry.ts mutated and its test watched go red (long delay on both paths, TypeError narrowing, Bun codes, uncapped retry-after, 429 tee hang, abort during body read)
Full providers + lib/embeddings suites (1,958 tests)
Independent audit of every provider's transport (18 OpenAI-compatible, Anthropic, Azure, Bedrock, Gemini/Vertex, deep research, TypeSafe): no double retries, abort signal passed everywhere, no replay after a stream starts
bun run type-check, bun run lint, bun run check:audits (51 audits), docs-manifest:check

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing (new tests pass the test-audit authoring gate)
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2026-09-28T18:50:25Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Sep 28, 2026 7:47pm UTC

cubic-dev-ai

No issues found across 12 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

greptile-apps · 2026-09-28T18:57:59Z

Confidence Score: 5/5

[High risk] Adds retry logic to AI provider API calls.

The PR appears safe to merge; no outstanding findings or new actionable issues were identified.

Summary

The PR adds a shared retry policy for OpenAI Responses and Gemini/Vertex requests, pins SDK retry budgets, and updates supporting tests and transport helpers.

The change since the previous review narrows connection-failure detection to socket error codes, excluding database and application error codes.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Provider request] --> B{Response or error}
  B -->|Success| C[Return result]
  B -->|Failure| D{Retryable status or socket error?}
  D -->|No| E[Return or throw failure]
  D -->|Yes| F{Budget, delay, and abort checks pass?}
  F -->|No| E
  F -->|Yes| G[Wait, then retry]
  G --> A

_{Reviews (7) · Last reviewed commit: "fix(providers): limit transport retries ..."}

…, resolve shadowed payloads

waleedlatif1 · 2026-09-28T19:03:07Z

@greptile

waleedlatif1 · 2026-09-28T19:03:09Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-28T19:03:15Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

1 issue found across 14 files

Confidence score: 4/5

In apps/sim/providers/gemini/core.ts, exhausted-quota 429 errors may be retried twice before surfacing, adding unnecessary delay; apply the policy’s insufficient_quota classifier to this retry path.

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/sim/providers/gemini/core.ts">

<violation number="1" location="apps/sim/providers/gemini/core.ts:1221">
P2: This SDK retry path does not apply the policy's `insufficient_quota` exemption. A Gemini/Vertex 429 carrying an exhausted-quota error will be replayed twice before surfacing; apply the quota-body classifier to `withProviderRetry` as well.</violation>
</file>

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

…retries, pin SDK retry budgets

waleedlatif1 · 2026-09-28T19:17:44Z

@greptile

waleedlatif1 · 2026-09-28T19:17:46Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-28T19:17:53Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 17 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

… message

waleedlatif1 · 2026-09-28T19:24:39Z

@greptile

waleedlatif1 · 2026-09-28T19:24:40Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-28T19:24:49Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

waleedlatif1 · 2026-09-28T19:29:31Z

@greptile

cubic-dev-ai

All reported issues were addressed across 17 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

waleedlatif1 · 2026-09-28T19:32:01Z

@greptile

waleedlatif1 · 2026-09-28T19:32:03Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-28T19:32:09Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 17 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

waleedlatif1 · 2026-09-28T19:40:38Z

@greptile

waleedlatif1 · 2026-09-28T19:40:39Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-28T19:40:46Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 17 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

waleedlatif1 · 2026-09-28T19:47:35Z

@greptile

waleedlatif1 · 2026-09-28T19:47:36Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-28T19:47:42Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 18 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

fix(providers): retry transient failures on OpenAI Responses and Gemini

0ef2b03