Add Fish Audio as a third voice engine (cloned voices, streams PCM, no ffmpeg) - #9
Open
BigBeedle wants to merge 2 commits into
Open
Add Fish Audio as a third voice engine (cloned voices, streams PCM, no ffmpeg)#9BigBeedle wants to merge 2 commits into
BigBeedle wants to merge 2 commits into
Conversation
Fish Audio (fish.audio) lets anyone clone their own voice and drive it by API, which makes it a natural fit for an agent that should sound like its person wants. This wires it in as a third engine beside Kokoro and ElevenLabs, following the existing doctrine end to end: - Engine order: fish -> elevenlabs -> Kokoro. Degrade, never mute — any cloud failure falls back exactly like the ElevenLabs path does. - Streams format "pcm" (16-bit mono at the configured sample_rate), so bytes go from the wire straight into the existing playback path: no ffmpeg dependency, unlike the ElevenLabs mp3 route. - Key handling mirrors _get_elevenlabs_key: macOS Keychain (item backtalk-fish, renameable via fish.key_slot), Linux secret-tool, FISH_AUDIO_API_KEY env var as the Windows last resort. Never in a file. - Config: a documented "fish" block in DEFAULTS. Defaults to the s2.1-pro-free model, which works on an unfunded account; the paid tiers (s1, s2-pro, s2.1-pro) return 402 until credits are added, and the comment says so, so nobody debugs a 402 blind. - mouth reads the block with .get, so configs written before this engine existed keep working unchanged. Tested on Windows 11 against a real cloned voice (reference_id from GET /model?self=true): parses, streams, plays through the long-lived output stream, and falls back to Kokoro cleanly when the model tier is unfunded (402) or the key is absent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A fresh httpx client per sentence pays a TLS handshake inside the gap between spoken sentences. One process-lifetime client shaves 0.2-0.4s off each subsequent chunk (measured: 0.36s cold vs 0.22s reused to first byte). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
Fish Audio (fish.audio) as a third voice engine, beside Kokoro and ElevenLabs. The draw: Fish Audio lets anyone clone their own voice (or build a character voice) in their account and drive it by API — so an agent can literally speak in the voice its person made for it. Engine order when enabled: fish → elevenlabs → Kokoro.
Why it fits this codebase
format: "pcm"(16-bit mono at the configuredsample_rate), so chunks go fromhttpx.streamstraight into the existing playback path. Unlike the ElevenLabs route there is no ffmpeg involved. Uses onlyhttpx+numpy, both already in the tree._get_elevenlabs_key— macOS Keychain (itembacktalk-fish, renameable viafish.key_slot), Linux secret-tool,FISH_AUDIO_API_KEYenv var as the Windows last resort. Never in a file.mouthreads the block with.get, so abacktalk.jsonwritten before this engine existed keeps working unchanged; thefishblock ships disabled in DEFAULTS.Config
The default model is
s2.1-pro-freedeliberately: it works on an unfunded account. The paid tiers (s1,s2-pro,s2.1-pro) return 402 Payment Required until the account holds credits — the DEFAULTS comment says so explicitly, so nobody debugs a 402 blind. The person'sreference_idis listable viaGET https://api.fish.audio/model?self=true.Tested
Windows 11, real account, real cloned voice: config parses, sentences stream and play through the long-lived output stream (audio laws respected — the engine only feeds PCM into the existing
_play_streampath), interrupt behavior unchanged, and the Kokoro fallback fires correctly on both a missing key and an unfunded model tier (402). macOS/Linux keychain lookups follow the ElevenLabs code path verbatim but I could only exercise the env-var branch — worth one audition on a Mac before merge.🤖 Generated with Claude Code