Skip to content

ai-providers: one shared SSE frame loop (FrameHandler) and response-to-event-stream helper #855

Open

Description

@smart-ssh-planner

Goal

The streaming loop that reads SSE frames with content timeout, keep-alive handling, transport errors and abrupt end of stream exists once. Providers only implement how a frame is parsed and how the stream is finalised.

Context

  • process_frame_stream exists three times: anthropic.rs:1453, openai_compatible.rs:1380, openai_responses.rs:577. Each is a futures::stream::unfold with tokio::time::timeout_at(state.content_deadline, state.frames.next()) (anthropic.rs:1488, openai_compatible.rs:1416, openai_responses.rs:615), the same transport-error, timeout and abrupt-EOF branches (68–74 % identical lines).
  • event_stream_from_response exists three times: anthropic.rs:1433, openai_compatible.rs:1358, openai_responses.rs:229 (94 % identical, two differing lines).
  • Provider state machines with the parts that differ: AnthropicStreamState::handle_event/finalize (anthropic.rs:995,1196), OpenAiStreamState::handle_chunk/finalize (openai_compatible.rs:1108,1218), handle_event in openai_responses.rs:422.
  • Shared building blocks already exist in sse.rs (sse_frame_stream, frame_is_content, CONTENT_TIMEOUT, SSE_INACTIVITY_TIMEOUT).
  • Specs: 0065 / ADR 0056 (a tool call from a truncated or incomplete response is never forwarded; only an allowlist of stop reasons counts as complete), issue AI request hangs forever with only "Automatic mode running": content timeout, total deadline and a visible wait state #694 (content timeout and total deadline).

Requirements

  • New module crates/ai-providers/src/common/frame_loop.rs with a trait, e.g. FrameHandler { fn on_frame(&mut self, frame: &SseFrame) -> Vec<RawEvent>; fn finalize(&mut self, abrupt: bool) -> Vec<RawEvent>; } (plus whatever the error mapping needs), and one generic process_frame_stream.
  • One shared event_stream_from_response; the two differing lines become a parameter or a trait method.
  • The three providers implement the trait on their existing state structs; their own loop copies are deleted.
  • The timeout semantics stay exactly as today: keep-alive frames do not extend the content deadline, content deltas do, the inactivity timeout yields a network error instead of hanging.
  • The withheld-tool-call rule (security invariant 5) stays in the providers' finalize and is not weakened.

Acceptance Criteria

  • grep -rn "timeout_at(" crates/ai-providers/src (production code) finds exactly one call site, in common/frame_loop.rs.
  • grep -rn "fn process_frame_stream\|fn event_stream_from_response" crates/ai-providers/src finds each once.
  • Unit tests for the shared loop with a fake FrameHandler: content timeout, keep-alive does not reset the deadline, transport error, abrupt EOF calls finalize(true), normal end calls finalize(false).
  • The existing timeout tests of all three providers pass (test_inactivity_timeout_yields_network_error_instead_of_hanging_forever, test_keep_alive_frames_do_not_restart_content_timeout, test_slow_deltas_over_three_minutes_complete_without_error, test_continuous_deltas_for_eleven_minutes_keep_streaming, test_responses_keep_alive_frames_time_out_but_deltas_do_not).
  • The existing truncated/abrupt tool-call tests pass unchanged (e.g. test_truncated_but_parseable_tool_call_is_never_forwarded_and_triggers_retry, test_abrupt_disconnect_mid_tool_call_triggers_retry_not_silent_parse).
  • The number of tests in cargo test -p ai-providers -- --list does not decrease.
  • Gate green (cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace, cargo build --workspace).

Technical Notes

  • Security-relevant (incomplete tool calls must never be executed, security invariant 5 in .claude/skills/smart-ssh-coder/references/security-invariants.md); review priority elevated.
  • Pure restructuring: event order and contents emitted to the app stay identical.
  • New code comments in English.

Spec impact: none

Open questions

  • Static dispatch (generic process_frame_stream<H: FrameHandler>) or Box<dyn FrameHandler>. Recommendation: generic; the handler type is known per provider.

Related Work

Activity

  1. added this to the 0.8.0 milestone on Oct 26, 2026
  2. 21 remaining items

  3. marked this as blocking 15 issues on Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Fields

    Priority

    P1 - High

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions