Skip to content

feat(desktop): live conversation token rate (tok/s) - #416

Open
GISWLH wants to merge 2 commits into
vastsa:mainfrom
GISWLH:feat/conversation-token-rate
Open

GISWLH wants to merge 2 commits into
vastsa:mainfrom
GISWLH:feat/conversation-token-rate

Conversation

@GISWLH

@GISWLH GISWLH commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Fixes #394

Summary

  • Show a live sliding-window output rate (tok/s) on the transcript stream-health strip while a turn is running (working / run-activity indicators, plus a compact chip while answer tokens stream).
  • Prefer provider outputTokens mid-stream; otherwise estimate from visible thinking+answer text (four code points ≈ one token) and label as approximate ().
  • Context inspector Generation speed remains the completed-turn snapshot (D428).

Algorithm

  • Cumulative samples on a stable ~250ms interval (useLiveTokenRate keeps content/thinking/outputTokens in refs so stream deltas do not tear down the timer).
  • Rate = rounded Δtokens / Δt over a 2s sliding window (min duration 200ms).
  • TTFT: until some positive output has been seen, rate stays undefined (chip hidden)—never a false 0 tok/s.
  • Stall: after positive output, a drained window reports 0 so reconnect/stall is visible.
  • Estimate → provider handoff: if the source flips to provider usage and the provider count is lower than the estimate, reset the sample window so monotonic clamping cannot freeze a hard 0 without .

Review fixes (this push)

  • a11y: ticking rate is aria-hidden / decorative (mirrors working-elapsed); streaming-only chip is not a polite live region.
  • Guard mounts with tokensPerSecond !== undefined (not always-truthy {tokenRate ? …}).
  • zh-CN mirrors for component-spec, interaction-patterns §3.2a, E2E plan, decisions-log / D428.
  • FR live copy uses jetons/s.

Tests

  • pnpm typecheck / pnpm lint in apps/desktop — green
  • node --test test/streaming-token-rate.test.mjs test/active-turn-surface.test.mjs — green (TTFT undefined, estimate→provider reset, hook interval deps, a11y/guard contracts)

Surface a sliding-window tok/s reading on the transcript stream-health
strip so users can tell whether the model stream is healthy or stalled.
Prefer provider output usage when available; otherwise estimate from
visible text and label as approximate. Fixes vastsa#394.
Keep the chip hidden until positive output (no false TTFT 0), reset the
sample window on estimate→provider handoff when provider is lower, sample
on a stable 250ms interval via input refs, and treat the ticking rate as
decorative for a11y. Mirror D428 in zh-CN specs and use FR jetons/s.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]对话token速率显示

1 participant