Context
#74 added Telnyx as a TTS provider. Telnyx also runs a streaming STT WebSocket (wss://api.telnyx.com/v2/speech-to-text/transcription) that already fronts Deepgram, Google, and Azure engines, plus a Telnyx in-house engine. I'd like to contribute a Telnyx STT provider as a follow-up, but one design question needs settling first because it touches barge-in.
What the Telnyx engine does (verified against the live API)
- Client sends raw linear16 PCM as binary frames; server sends JSON frames with
transcript, is_final, confidence.
- The Telnyx engine emits exactly ONE final frame after audio stops, then closes the socket (1000). No interims, no word timestamps, no
speech_started / utterance_end. Confidence is null.
interim_results=true suppresses ALL output from the Telnyx engine (adding the param returns zero frames). It cannot be requested.
transcription_engine is case-sensitive: Telnyx works, telnyx closes the socket.
- The Deepgram engine through the same endpoint does emit interims (
is_final: false frames with confidence), so the finals-only behavior is specific to the Telnyx engine.
The problem: barge-in is a hard dependency on partials
In internal/pipeline/inbound.go (line numbers at current main):
hasPartialText is set only by non-final results (~101-108, gate at ~76).
- The barge-in trigger requires
bargeInVAD.IsSpeaking() && agentSpeaking && hasPartialText (~196).
- Backchannel classification ("mm-hm" filtering) reads
latestPartial (~168-193), also fed only by partials.
- Live transcript captions stream to the client from the partial path (~110-113).
With a finals-only provider, hasPartialText is never set: barge-in never fires (dead, not degraded), backchannel filtering never classifies, and the UI gets no live captions.
Options
- Provider + docs note (the finals-only OpenAI whisper precedent,
docs/providers.md:15): ship the Telnyx-engine provider and document that barge-in and live captions do not work with it.
- Provider that exposes Telnyx's engine param:
transcription_engine = "Telnyx" (in-house, finals-only, no barge-in) or "Deepgram" (proxied through the same endpoint, emits interims, barge-in works). Default engine becomes a product choice rather than a hard limitation.
- Pipeline capability flag: expose whether the active STT produces partials and degrade gracefully. More pipeline surface, so I don't want to assume that's welcome without asking.
Which way would you like it? Happy to implement any of the three.
Context
#74 added Telnyx as a TTS provider. Telnyx also runs a streaming STT WebSocket (
wss://api.telnyx.com/v2/speech-to-text/transcription) that already fronts Deepgram, Google, and Azure engines, plus a Telnyx in-house engine. I'd like to contribute a Telnyx STT provider as a follow-up, but one design question needs settling first because it touches barge-in.What the Telnyx engine does (verified against the live API)
transcript,is_final,confidence.speech_started/utterance_end. Confidence isnull.interim_results=truesuppresses ALL output from the Telnyx engine (adding the param returns zero frames). It cannot be requested.transcription_engineis case-sensitive:Telnyxworks,telnyxcloses the socket.is_final: falseframes with confidence), so the finals-only behavior is specific to the Telnyx engine.The problem: barge-in is a hard dependency on partials
In
internal/pipeline/inbound.go(line numbers at current main):hasPartialTextis set only by non-final results (~101-108, gate at ~76).bargeInVAD.IsSpeaking() && agentSpeaking && hasPartialText(~196).latestPartial(~168-193), also fed only by partials.With a finals-only provider,
hasPartialTextis never set: barge-in never fires (dead, not degraded), backchannel filtering never classifies, and the UI gets no live captions.Options
docs/providers.md:15): ship the Telnyx-engine provider and document that barge-in and live captions do not work with it.transcription_engine = "Telnyx"(in-house, finals-only, no barge-in) or"Deepgram"(proxied through the same endpoint, emits interims, barge-in works). Default engine becomes a product choice rather than a hard limitation.Which way would you like it? Happy to implement any of the three.