Repository navigation
SMOODEV-4158: Re-ask an LLM call whose stream breaks mid-response - #209
Merged
Merged
Conversation
On the bench, six gpt-6-luna turns died on "stream read error: error decoding
response body". chat_stream retried only the initial send, so a body that
broke in transit failed the whole turn. The error was also formatted with {e},
which dropped the source chain, so the log could not say whether the cause was
a reset, a truncation or a timeout.
Body read errors are now a typed StreamInterrupted that carries the full
chain. run_with_channel re-asks the same request up to MAX_STREAM_RETRIES (2)
times with 500ms/1s backoff, and emits AgentEvent::LlmStreamRetry. It re-asks
only while that cannot be observed or repeat anything: the response has not
completed, no answer text has reached the consumer, and no tool call has been
dispatched. Nothing is pushed to the conversation and nothing is costed until
a response completes. An error after a complete response is still surfaced
and not retried.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On the SMOODEV-4125 bench, six gpt-6-luna turns died on
stream read error: error decoding response body, after 17 to 428s, and were counted INCONCLUSIVE.chat_streamretries only the initial send. A body that breaks in transit failed the whole turn, and the error was formatted with{e}, so the source chain (reset, truncation or timeout) was lost.Solution
StreamInterrupted(re-exported) that carries the fullsource()chain.run_with_channelre-asks the same request up toMAX_STREAM_RETRIES(2) times with 500ms/1s backoff, and emitsAgentEvent::LlmStreamRetry { iteration, attempt, max_retries, error }.Donenot seen);StreamInterrupted, not a refusal or a malformed event.gateway_list_price_for/record_cost_and_check_budget.MockLlmClient::push_stream_interruptedscripts a mid-stream break.Verification
StreamInterruptedqualifies (including through.context());cargo fmt --all -- --check,cargo clippy --workspace --all-targets -- -D warnings,cargo test --workspace: all green (640 core lib tests).For #208 (1.15.0). @stream-usage owns the changeset.
Jira: SMOODEV-4158
🤖 Generated with Claude Code