fix: align Undici headersTimeout with apiRequestTimeout for OpenAI-compatible providers#12245
Draft
roomote-v0[bot] wants to merge 1 commit intomainfrom
Draft
fix: align Undici headersTimeout with apiRequestTimeout for OpenAI-compatible providers#12245roomote-v0[bot] wants to merge 1 commit intomainfrom
roomote-v0[bot] wants to merge 1 commit intomainfrom
Conversation
…mpatible providers Fixes #12244. The OpenAI SDK timeout option only controls the SDK-level request timeout but does not override Undici default headersTimeout of 300s. This causes premature HeadersTimeoutError for slow providers like LM Studio when time-to-first-byte exceeds 5 minutes. Creates a shared createFetchWithUndiciTimeout() utility that builds a custom fetch function backed by an Undici Agent with headersTimeout and bodyTimeout aligned to the configured apiRequestTimeout. Applied to all three provider paths that create OpenAI clients: lm-studio, openai (generic OpenAI-compatible), and base-openai-compatible-provider.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related GitHub Issue
Closes: #12244
Description
This PR attempts to address Issue #12244. Feedback and guidance are welcome.
The OpenAI SDK's
timeoutoption only controls the SDK-level request timeout but does not override Undici's defaultheadersTimeoutof 300 seconds (5 minutes). When time-to-first-byte exceeds 300s (common with large local models via LM Studio, Ollama, or other OpenAI-compatible providers), Undici throws aHeadersTimeoutErrorbefore the SDK timeout fires.How it works:
createFetchWithUndiciTimeout()utility insrc/api/providers/utils/undici-fetch.tsthat builds a customfetchfunction backed by an UndiciAgentwithheadersTimeoutandbodyTimeoutaligned to the configuredapiRequestTimeout.lm-studio.ts(LM Studio provider)openai.ts(generic OpenAI-compatible provider, including Azure variants)base-openai-compatible-provider.ts(base class for providers like DeepSeek, xAI, etc.)undiciis already insrc/package.json.As noted by @yangzetao in the issue, this fix covers not just LM Studio but the generic OpenAI-compatible provider path as well, ensuring consistency across all providers using the OpenAI SDK transport layer.
Test Procedure
createFetchWithUndiciTimeout()utility (undici-fetch.spec.ts)fetchoption is passed to the OpenAI SDK constructorRun tests:
Pre-Submission Checklist
Screenshots / Videos
N/A -- no UI changes.
Documentation Updates
No documentation updates needed. This is an internal fix to the Undici transport layer timeout configuration.
Interactively review PR in Roo Code Cloud