Skip to content

Making a Character: the daemon makes the line audio - #189

Open
longzhi wants to merge 2 commits into
claude/own-keysfrom
claude/character-lines
Open

longzhi wants to merge 2 commits into
claude/own-keysfrom
claude/character-lines

Conversation

@longzhi

@longzhi longzhi commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Builds on #187 (own keys).

Changes

  • characters/voices.tsv: id, provider, voice, language, gender, display name. It holds the presets' voices (robot is skipped) and Jessica, and it is compiled into the daemon. GET /v1/voices lists it.
  • POST /v1/characters {id?, name, base, voice, lines:[{occasion,text}]} starts a background job, one at a time (409 while one is running). It answers 202. making in /v1/status reports {id, name, state, done, total, reason, error}. voice must be a voices.tsv id, base a preset with lines, and there can be at most 110 lines (what a pack with a look holds).
  • Each line goes to the provider when the user has their own key for it: ElevenLabs pcm_16000 with eleven_flash_v2_5, or the Volcano Engine V3 stream with format:"pcm", sample_rate:16000. Without a key, it goes to /v1/voice/speak with "for":"character" and the session. The reasons are plan, monthly, global_cap, signed_out (401 forgets the session), and own_key when a provider refuses the user's key.
  • {address} lines, naming and layout follow tools/make-character.sh:
    • The audio is stored as audio/<occasion>/<prefix>-<sha1[:10]>.pcm (+ .txt) and audio-address/<form>/....
    • Address lines get the prefix 0-<row>, all other lines get 1.
    • The comma rules of without_address are ported.
    • Lines that are deleted or reworded are pruned.
  • Processing is done in Rust: trim at -50 dBFS, peak at -1 dBFS, then IMA ADPCM with firmware-rs/core's Encoder. That crate moves from dev- to normal dependencies. It is already a workspace member, so the license bundle is unchanged.
  • Storage is under config_dir()/characters/<id>/:
    • pack.bin and address/<form>.bin are v1 packs with no look, byte for byte what character_pack.py builds. The App takes them as it takes a preset's (withAddress/withLook, Pack::parse/with_address/with_look).
    • manifest.json holds {name, base, voice, lines}.
  • The id is a slug of the name (at most 22 characters) plus 8 random hex digits, 31 at most, so it is a valid pack id.
  • Resume: a line whose PCM file exists is never made again. Files are written atomically. Making an existing id again resumes it; if its voice changed, the old audio is dropped.
  • synthesis.rs: the Volcano Engine JSON stream reader is split out (read_volc) so PCM can share it, and a few constants are now pub(crate).
  • Docs: docs/app.md (the local API), docs/characters.md ("Making a Character"), characters/README.md.

Verification

  • cargo test --workspace --locked: all pass (daemon 356).
  • cargo clippy --workspace --all-targets: no warnings in the touched files.
  • New tests use fake provider and service HTTP servers. They cover:
    • address forms expanded, with make-character names (the SHA-1 checked against shasum) and its comma rules;
    • ADPCM decoding back to the input with SNR > 20 dB;
    • trimming and the -1 dBFS peak;
    • a refusal halfway (monthly) stopping the job, then a resume that makes each line exactly once and the packs parsing with the firmware's parser;
    • plan, global_cap and 401 reasons;
    • the own-key path for both providers;
    • only voices.tsv voices accepted, and every preset's voice_for voice present;
    • one job at a time, and no path traversal through id;
    • the HTTP routes.
  • For the comparison with character_pack.py, daemon/src/fixtures/character/ holds three PCM lines (odd lengths included) and the pack.bin that tools/character_pack.py built from them. The daemon's pack must match it byte for byte. A checked-in fixture keeps cargo test independent of Python, and it pins the format: a drift on either side shows up as a diff.
  • Each new test was checked to fail with its feature broken: cache check off, own key ignored, forms dropped, nibble order swapped, wrong reason, no name sort.

Follow-ups in the second commit:

  • GET /v1/characters lists made Characters: id, name, base, voice, lines, and complete. Starting a job removes the old packs, so complete always matches the lines the Character has now.
  • voice_for knows made Characters. When the box reports a made Character's id, the reply uses the voice from its manifest. This needs the App to keep the made Character's id as the pack id.
  • A panicking job ends as failed with reason internal, through catch_unwind. A lock left poisoned by the panic doesn't stop the next job.
  • Both have tests, and both tests were checked to fail with their feature turned off.

Not done here: the App UI (#184).

Closes #183

🤖 Generated with Claude Code

POST /v1/characters makes a Character's lines in a voice from the new
characters/voices.tsv (GET /v1/voices): with the user's own key for the
voice's provider, 16 kHz PCM straight from ElevenLabs or Volcano Engine;
otherwise from the voice service with "for":"character". Lines with
{address} are made once per form of address, named and laid out as
tools/make-character.sh does. Each line is trimmed, normalized to -1 dBFS
and packed with the box's ADPCM encoder into pack.bin and
address/<form>.bin under config_dir()/characters/<id>/, byte for byte as
tools/character_pack.py packs them. A refusal stops the job and keeps the
lines made; making the same id again resumes. Progress is `making` in
/v1/status.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 24, 2026 09:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

…ve a panic

GET /v1/characters lists the Characters made here from their manifests,
with whether their packs are made. A box wearing a made Character's pack
gets spoken replies in the voice it was made with, not Jessica. A job
that panics ends as failed with reason "internal" instead of making
forever.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants