Blaze Livekit plugin integration by HoangPN711 · Pull Request #5050 · livekit/agents

HoangPN711 · 2026-03-09T15:01:57Z

Summary

Add Blaze plugin support for LiveKit Agents.

Changes

add Blaze STT plugin
add Blaze TTS plugin
add Blaze LLM plugin
add Blaze plugin packaging/config helpers
add Blaze plugin README

Motivation

Enable Blaze voice AI services to be used through the existing LiveKit Agents plugin architecture.

Notes

The implementation is isolated under livekit-plugins/livekit-plugins-blaze and follows the existing provider plugin pattern.

CLAassistant · 2026-03-09T15:02:08Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ HoangPN711
✅ fpt-hoangpn2
❌ trunghlt
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

HoangPN711 · 2026-05-05T23:09:30Z

Hi @tinalenguyen,

This PR adds the Blaze provider integration for LiveKit Agents and keeps the implementation scoped under livekit-plugins/livekit-plugins-blaze.

It follows the existing STT/TTS/LLM plugin structure used by other provider integrations. I’ve also addressed the outstanding review comments on the PR.

Would appreciate a review when you have time. Thanks!

tinalenguyen · 2026-05-13T00:43:59Z

hi @HoangPN711! thank you for the contribution, could you bump the version and add the plugin to this pyproject file

HoangPN711 · 2026-05-13T23:17:54Z

Hi @tinalenguyen , thanks for reviewing this PR!

Updated as requested:

Bumped livekit-plugins-blaze to 1.5.9 (aligned with the current livekit-agents release)
Added blaze = ["livekit-plugins-blaze>=1.5.9"] to livekit-agents/pyproject.toml

Please let me know if any further changes are needed.

…le-session turns

…and utils

- Rebase onto main; keep optional-deps at 1.6.6 and add blaze extra - Bump livekit-plugins-blaze to 1.6.6 to match monorepo - Fix ruff I001 import order in test_plugin_blaze_utils.py - Remove explicit output_emitter.flush() at end of streaming TTS _run so base end_input() can set is_final on the last audio frame

- TTS default model: 2.0-realtime (required by /v1/tts/realtime aliases) - STT batch model: stt-async-1.5 on POST /v1/stt/transcribe - STT stream model: stt-stream-1.5 on native WS /v1/stt/realtime - Enable streaming STT capabilities with interim results

CI ruff format --check failed after the streaming STT changes.

ChunkedStream early-return on blank text left the emitter unstarted, so base end_input() raised RuntimeError. Always initialize (and flush) so empty synthesis completes cleanly.

The transcribe endpoint only accepts ModelVersionEnum v1.0|v2.0. stt-async-1.5 belongs to /v1/stt/execute (async jobs). Keep stt-stream-1.5 as the realtime WebSocket default.

AgentSession stt turn-detection opens the user turn on START_OF_SPEECH. Emit it on the first non-empty partial/final and re-arm after END_OF_SPEECH.

WebSocket audio chunks are not sample-aligned. Carry trailing odd bytes and only fade/push even-length PCM so cast('h') never raises.

result.get("confidence", 1.0) still yields None when the key is present with a null value, which crashes %.3f logging. Match the streaming path.

POST /v1/tts/realtime returns 404 on the public gateway — realtime TTS is WebSocket-only. ChunkedStream now follows the same handshake as streaming TTS (auth → speech-start → query → speech-end) so synthesize() works for the plugin tester and any non-streaming callers.

- Point plugin docs at https://blaze.vn - Set authors to LiveKit <hello@livekit.io> - Empty py.typed PEP 561 marker to match sibling plugins - Convert PCM→WAV via rtc.AudioFrame.to_wav_bytes() - Drop custom TTS sentence batching; use framework SentenceTokenizer - Merge all blaze unit tests into tests/test_plugin_blaze.py

trunghlt · 2026-07-24T06:33:13Z

@tinalenguyen Thanks for the review — all points from the 2026-07-24 pass are addressed in 714ea1e7a343752b5681e46bf29a7c81bd386db5:

README / URLs — site docs use https://blaze.vn (no blaze.ai leftovers in the plugin)
pyproject authors — LiveKit <hello@livekit.io>
py.typed — empty PEP 561 marker (matches siblings)
PCM→WAV — rtc.AudioFrame(...).to_wav_bytes() instead of hand-rolled RIFF headers
TTS chunking — custom batch helpers removed; uses framework SentenceTokenizer (pass tokenizer= to configure); flush/stream protocol retained
Tests — merged into a single tests/test_plugin_blaze.py (pytestmark = pytest.mark.unit); 52 unit tests passing locally

Please take another look when you have a chance.

BlazeConfig accepts api_token, not auth_token (which is the per-plugin constructor param). Align the package docstring example.

Match llm.py: POST /v1/voicebot-call/{bot_id}/chat-conversion-stream.

Do not early-return when chat context only has system/developer messages. Blaze loads the voicebot prompt server-side; match agents-js by still POSTing messages=[] so generate_reply can speak first.

This comment was marked as resolved.

Sign in to view

HoangPN711 changed the title ~~feat: add blaze voicebot plugin~~ Blaze Livekit plugin integration Apr 3, 2026

This comment was marked as resolved.

Sign in to view

HoangPN711 force-pushed the feat/blaze-voicebot-plugin branch from 4653f88 to 77090e4 Compare April 18, 2026 15:06

This comment was marked as resolved.

Sign in to view

HoangPN711 force-pushed the feat/blaze-voicebot-plugin branch 3 times, most recently from 2b94037 to e9052a6 Compare May 19, 2026 22:52

This comment was marked as resolved.

Sign in to view

HoangPN711 force-pushed the feat/blaze-voicebot-plugin branch 3 times, most recently from 1cea05c to 2c82c75 Compare June 11, 2026 22:13

fpt-hoangpn2 and others added 22 commits July 24, 2026 13:32

blaze: fix CI and reset between segment

f4b025d

blaze: fix timout for llm and tts

ed25d7c

bump version 1.6.3

3d527fd

fix(blaze-tts): idle timeout, WS reconnect, and stream guard for sing…

13be3ce

…le-session turns

fix(blaze-tts): resolve ruff and mypy recv return type for CI

bded624

fix(blaze): cap TTS WS fast reconnects and expose LLM bot_id as model

b449db5

test(blaze): add comprehensive unit tests for STT, TTS, LLM, config, …

ba8074c

…and utils

bump version 1.6.4

fdcb6ea

fix(blaze): sync TTS segment count and honor plugin timeouts

9903cbc

style(blaze): ruff format stt.py

0ca2edd

CI ruff format --check failed after the streaming STT changes.

fix(blaze): initialize TTS AudioEmitter before empty-text return

e1cb3f6

ChunkedStream early-return on blank text left the emitter unstarted, so base end_input() raised RuntimeError. Always initialize (and flush) so empty synthesis completes cleanly.

fix(blaze): use v2.0 for batch STT /v1/stt/transcribe

6d4c4f0

The transcribe endpoint only accepts ModelVersionEnum v1.0|v2.0. stt-async-1.5 belongs to /v1/stt/execute (async jobs). Keep stt-stream-1.5 as the realtime WebSocket default.

fix(blaze): emit START_OF_SPEECH in native STT stream

9144447

AgentSession stt turn-detection opens the user turn on START_OF_SPEECH. Emit it on the first non-empty partial/final and re-arm after END_OF_SPEECH.

fix(blaze): tolerate odd-length PCM WS frames in TTS fade

9c1f4e4

WebSocket audio chunks are not sample-aligned. Carry trailing odd bytes and only fade/push even-length PCM so cast('h') never raises.

fix(blaze): coerce null STT confidence to 1.0

7e8831a

result.get("confidence", 1.0) still yields None when the key is present with a null value, which crashes %.3f logging. Match the streaming path.

fix(blaze): preserve 0.0 STT confidence; only default null

94339dd

fix(blaze): honor explicit connect timeout equal to framework default

454400a

fix(blaze): report STT ready-wait timeout as APITimeoutError

7458be0

trunghlt force-pushed the feat/blaze-voicebot-plugin branch from b663e95 to 714ea1e Compare July 24, 2026 06:32

This comment was marked as resolved.

Sign in to view

fix(blaze): use api_token in BlazeConfig docstring example

aee7f51

BlazeConfig accepts api_token, not auth_token (which is the per-plugin constructor param). Align the package docstring example.

This comment was marked as resolved.

Sign in to view

docs(blaze): fix README LLM endpoint path

2bf3a45

Match llm.py: POST /v1/voicebot-call/{bot_id}/chat-conversion-stream.

This comment was marked as resolved.

Sign in to view

fix(blaze): POST empty messages for agent-first LLM turns

e042630

Do not early-return when chat context only has system/developer messages. Blaze loads the voicebot prompt server-side; match agents-js by still POSTing messages=[] so generate_reply can speak first.

Uh oh!

Conversation

HoangPN711 commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Motivation

Notes

Uh oh!

CLAassistant commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

HoangPN711 commented May 5, 2026

Uh oh!

tinalenguyen commented May 13, 2026

Uh oh!

HoangPN711 commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

trunghlt commented Jul 24, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

HoangPN711 commented Mar 9, 2026 •

edited

Loading

CLAassistant commented Mar 9, 2026 •

edited

Loading

HoangPN711 commented May 13, 2026 •

edited

Loading