Skip to content

Own keys: ElevenLabs and Volcano Engine keys for spoken replies - #187

Merged
longzhi merged 2 commits into
mainfrom
claude/own-keys
Oct 25, 2026
Merged

longzhi merged 2 commits into
mainfrom
claude/own-keys

Conversation

@longzhi

@longzhi longzhi commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator

Closes #182 (parent spec second-state/vibebuddy-cloud#16, "Own keys"; decision ADR-0020 in #179).

What changes

Daemon

  • New voice_keys.rs: config_dir()/voice-keys.json, {"elevenlabs"?, "volc"?}, written 0600 with write_private. It is a file of its own, as speech.json is, so PUT /v1/config can't wipe it.
  • PUT /v1/voice-keys sets or clears each key. A string sets it; null or "" clears it; a provider left out keeps its key; an unknown name or value gets a 400. The answer has the same shape as the status field.
  • /v1/status gets voice_keys: {"elevenlabs": {"set", "refused"}, "volc": {"set", "refused"}}. It never contains a key.
  • synthesis.rs: a new Synthesis dispatcher. If the voice's provider has an own key, the reply goes straight to that provider and never to vibebuddy-api. Otherwise it goes to the service as before.
    • ElevenLabs: /v1/text-to-speech/<voice>/stream?output_format=mp3_24000_48 with eleven_flash_v2_5, the same request the service makes.
    • Volcano Engine: the V3 one-way stream with seed-tts-1.0, asking for MP3 at 24 kHz. The daemon reads its back-to-back JSON objects and drops the leading ID3v2 tag, wherever the pieces split it.
  • A 401 or 403 marks the key refused in the status and ends that reply's audio with "error":"synthesis". There is no fallback to the service. Taking the key again, or changing it, clears the flag. The status stream is told through a watch channel.
  • A release with no voice service still speaks replies in a voice that has an own key. Synthesizer::speaks keeps every other reply text-only, sending nothing, as before.

Apps (Mac and Linux, in step)

  • The Account tab gets "Your own keys": a field each for ElevenLabs and Volcano Engine, and "Get a key" linking to ElevenLabs' API keys page or the Volcano Engine speech console. A saved key shows only as "Set", with "Clear". A refused key gets a red line. Strings are in zh-Hans as 自己的密钥 and 火山引擎; the UI never says "token".
  • Docs: the local API and the Account tab in docs/app.md, and own-key synthesis in docs/resident-session.md.

Volcano Engine MP3 at 64 kbps

vibebuddy-cloud found that Doubao's MP3 comes at 64 kbps whatever bit_rate asks, behind an ID3 tag, so the service asks for PCM and encodes it itself. For an own key I checked the box's player (firmware-rs/core/src/spoken_reply.rs) instead of adding an encoder:

  • nanomp3 decodes any MPEG-2 Layer III bit rate (LOOKAHEAD covers frames up to 160 kbps).
  • The only things tied to 48 kbps are the 300 ms start threshold (it fills a little sooner at 64 kbps) and MAX_REPLY_BYTES. At 64 kbps that 256 KB holds about 32 s ahead instead of 43 s.
  • So the daemon drops the ID3 tag and forwards the frames unchanged, with no new dependency.
  • The box drops a stream that isn't 24 kHz. That keeps the reply silent rather than garbled, if Doubao ever ignored sample_rate too.

Verification

  • cargo test (workspace): all pass. New tests run against fake provider HTTP servers:
    • an own key is used for its provider only, and the other provider's voice still goes to the service;
    • Volcano Engine MP3 arrives without its ID3 tag, with the request's headers and params checked;
    • a refused key is reported and does not fall back to the service;
    • the ID3 tag is dropped however the pieces cut it;
    • the key file is 0600, and neither the status nor the PUT answer contains the key;
    • PUT /v1/voice-keys sets, clears and rejects bad input.
  • Linux decode test for voice_keys, including an older daemon without it.
  • cargo clippy --all-targets: no new warnings. The three that remain were there before, in activity.rs and session_titles.rs.
  • swift build --package-path app: clean. swift run --package-path app SelfTest: passed, with a new decode check for own keys.
  • python3 tools/check-localization.py: ok.
  • Not done by hand: a real ElevenLabs or Volcano Engine key on a box (the issue's manual acceptance check).

🤖 Generated with Claude Code

The user's own key for a speech provider (ADR-0020) is kept by the daemon in
config_dir()/voice-keys.json (mode 0600) and set with PUT /v1/voice-keys. A
reply whose voice is from a provider with an own key goes straight to that
provider and never to vibebuddy-api: ElevenLabs as mp3_24000_48, Volcano
Engine's V3 stream as 24 kHz MP3 with its leading ID3 tag dropped. A 401 or
403 marks the key refused in /v1/status and leaves that reply silent, with
no fallback to the service. The status says only which keys are set.

The Mac and Linux Account tabs get "Your own keys": a field for each, a link
to each provider's key page, "Set" in place of the value, and Clear.

Closes #182

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 24, 2026 09:37

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@longzhi
longzhi merged commit 178d52f into main Oct 25, 2026
2 checks passed
@longzhi
longzhi deleted the claude/own-keys branch October 25, 2026 08:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Own keys: ElevenLabs and Volcano Engine keys for spoken replies

2 participants