Repository navigation
ai-providers: one shared SSE frame loop (FrameHandler) and response-to-event-stream helper #855
Copy link
Copy link
Open
Description
Goal
The streaming loop that reads SSE frames with content timeout, keep-alive handling, transport errors and abrupt end of stream exists once. Providers only implement how a frame is parsed and how the stream is finalised.
Context
process_frame_streamexists three times:anthropic.rs:1453,openai_compatible.rs:1380,openai_responses.rs:577. Each is afutures::stream::unfoldwithtokio::time::timeout_at(state.content_deadline, state.frames.next())(anthropic.rs:1488,openai_compatible.rs:1416,openai_responses.rs:615), the same transport-error, timeout and abrupt-EOF branches (68–74 % identical lines).event_stream_from_responseexists three times:anthropic.rs:1433,openai_compatible.rs:1358,openai_responses.rs:229(94 % identical, two differing lines).- Provider state machines with the parts that differ:
AnthropicStreamState::handle_event/finalize(anthropic.rs:995,1196),OpenAiStreamState::handle_chunk/finalize(openai_compatible.rs:1108,1218),handle_eventinopenai_responses.rs:422. - Shared building blocks already exist in
sse.rs(sse_frame_stream,frame_is_content,CONTENT_TIMEOUT,SSE_INACTIVITY_TIMEOUT). - Specs: 0065 / ADR 0056 (a tool call from a truncated or incomplete response is never forwarded; only an allowlist of stop reasons counts as complete), issue AI request hangs forever with only "Automatic mode running": content timeout, total deadline and a visible wait state #694 (content timeout and total deadline).
Requirements
- New module
crates/ai-providers/src/common/frame_loop.rswith a trait, e.g.FrameHandler { fn on_frame(&mut self, frame: &SseFrame) -> Vec<RawEvent>; fn finalize(&mut self, abrupt: bool) -> Vec<RawEvent>; }(plus whatever the error mapping needs), and one genericprocess_frame_stream. - One shared
event_stream_from_response; the two differing lines become a parameter or a trait method. - The three providers implement the trait on their existing state structs; their own loop copies are deleted.
- The timeout semantics stay exactly as today: keep-alive frames do not extend the content deadline, content deltas do, the inactivity timeout yields a network error instead of hanging.
- The withheld-tool-call rule (security invariant 5) stays in the providers'
finalizeand is not weakened.
Acceptance Criteria
-
grep -rn "timeout_at(" crates/ai-providers/src(production code) finds exactly one call site, incommon/frame_loop.rs. -
grep -rn "fn process_frame_stream\|fn event_stream_from_response" crates/ai-providers/srcfinds each once. - Unit tests for the shared loop with a fake
FrameHandler: content timeout, keep-alive does not reset the deadline, transport error, abrupt EOF callsfinalize(true), normal end callsfinalize(false). - The existing timeout tests of all three providers pass (
test_inactivity_timeout_yields_network_error_instead_of_hanging_forever,test_keep_alive_frames_do_not_restart_content_timeout,test_slow_deltas_over_three_minutes_complete_without_error,test_continuous_deltas_for_eleven_minutes_keep_streaming,test_responses_keep_alive_frames_time_out_but_deltas_do_not). - The existing truncated/abrupt tool-call tests pass unchanged (e.g.
test_truncated_but_parseable_tool_call_is_never_forwarded_and_triggers_retry,test_abrupt_disconnect_mid_tool_call_triggers_retry_not_silent_parse). - The number of tests in
cargo test -p ai-providers -- --listdoes not decrease. - Gate green (
cargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warnings,cargo test --workspace,cargo build --workspace).
Technical Notes
- Security-relevant (incomplete tool calls must never be executed, security invariant 5 in
.claude/skills/smart-ssh-coder/references/security-invariants.md); review priority elevated. - Pure restructuring: event order and contents emitted to the app stay identical.
- New code comments in English.
Spec impact: none
Open questions
- Static dispatch (generic
process_frame_stream<H: FrameHandler>) orBox<dyn FrameHandler>. Recommendation: generic; the handler type is known per provider.
Related Work
- Previous steps: ai-providers: one shared formatter for command results and rejected actions (single fencing call site) #845, ai-providers: one shared HTTP send loop with 429/5xx retry for all providers #851; next: ai-providers: shared max-tokens retry, openai_responses without openai_compatible internals, dead budget field removed #860
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
P1 - High
Activity
21 remaining items