Skip to content

fix(gateway): classify terminal SSE faults before recording health - #387

Open
fylorn wants to merge 1 commit into
mainfrom
codex/sse-terminal-health
Open

fylorn wants to merge 1 commit into
mainfrom
codex/sse-terminal-health

Conversation

@fylorn

@fylorn fylorn commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

When the last upstream returns HTTP 200 with an SSE error before any content, a clean EOF currently leaves its health record as a success. Balance and quota failures are therefore retried on later requests, and rate-limit errors incorrectly add a success sample. An error frame without a trailing blank line is only recognized after the relay has already decided health.

Read the upstream error before recording an opening success and classify terminal faults with the same helper used by WebSocket turns, including known quota resets. Flush the final error frame before health accounting, and recognize an undelimited final frame while buffering the opening as well. Balance and quota failures pause the provider, server/auth failures use the existing breaker, and rate limits stay out of the health tally. Original SSE errors still reach the client; content already delivered keeps its existing success and is never replayed.

This fixes the pre-existing stream-finalization issue found while reviewing #378 and #379, and is based on the merged #379.

Validation:

  • Reproduced the failures against unmodified main before applying the fix.
  • Six regression tests cover Anthropic, Chat Completions, Responses and Gemini; buffered and immediate forwarding; undelimited final frames; exact health tallies; known/body quota resets; later routing; client/model errors; and no replay after content.
  • cargo fmt --all -- --check: passed.
  • cargo clippy --workspace --all-targets -- -D warnings: passed.
  • Full workspace nextest on the final code: 4,296 passed, 1 failed, 8 skipped. The single failure is the existing macOS tw-watch::fd_budget::with_too_few_fds_it_falls_back_to_fsevents_and_misses_nothing assertion (37 != 5); it fails identically when run independently against unmodified main b9ea83df081a3911e800938af2e0e5d5002d8c99. This PR does not change tw-watch or its dependencies.
  • scripts/smoke.sh: 86 passed, 0 failed.

A terminal SSE error can arrive over HTTP 200 and a clean EOF. Pulse
correctly treats errors as progress for idle timing, but the relay used
that signal to record success before inspecting the error. Its EOF path
also only applied rate limits, so balance and quota failures stayed healthy.

Inspect errors before recording an opening success, reuse the shared
stream-fault classifier and known quota reset, and flush undelimited error
frames before final health accounting. Apply the same EOF tolerance while
buffering a stream opening so the original error is retained there too. Keep rate limits out of the health
tally and retain the existing success after content has started.

Cover the terminal health tally, pause and reset behavior, original error
forwarding, subsequent routing, all four SSE dialects and no replay after
content.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant