[fylorn]

Summary

Fixes to how a stream's speed is judged against its provider's minimum (follow-up to #131).

• Upstream-reported counts are no longer used. Gemini's running output count includes the thoughts it does not show. Once that count got ahead of the text, the text that followed counted for nothing, and a fast stream could be cut. Speed is now judged on visible text only.
• Summarised reasoning is left out. Responses reasoning summaries, Anthropic thinking blocks, Gemini thought parts and Bedrock reasoning content arrive in bursts while the model reasons at full speed.
  • No second is judged while such a phase is open.
  • After the phase ends, its time and its text are left out of the window.
  • Reasoning streamed as it is generated (Chat reasoning_content, Responses response.reasoning_text.delta) still counts.
• Short pauses.
  • A long pause is now a quarter of the window, at most 5 s.
  • No second is judged if the last content arrived more than 1 s ago, so the burst an upstream sends after holding tokens back is waited for.
• End of answer. The end is now also detected from each format's last frame (Anthropic message_stop, Chat [DONE], Responses response.completed), not only from a stop reason. Gemini and Bedrock end with their stop reason.
• Upstream connection. It is dropped at the cut, before the error frame goes to the client.
• Affinity and log row. The affinity is dropped, and the speed is written to the log row, only when the stream's outcome is the cut.
• The CHANGELOG entry, schema comment and console hint are updated to match.

Tests

New unit tests in proxy::min_speed. Run against the previous implementation, these fail:

• hidden reasoning in Gemini's count
• a summarised Responses reasoning phase
• an open Anthropic thinking block
• the quarter-window pause
• waiting for the burst after a short pause
• end detection in every format

reasoning_streamed_as_generated_is_judged guards that raw reasoning still counts.

Also run:

• make precommit and pnpm lint pass.
• The gateway_min_speed, gateway_retry and streaming_and_cache integration tests pass locally, except one streaming_and_cache test that needs ClickHouse, which was not running; CI runs it.

The connection drop and the outcome guard come without a failing test: the pump returns right after the cut, so locally there is no ordering that tells the old code from the new.

🤖 Generated with Claude Code