Speech Service

HTTP services for speech-to-text (STT) and text-to-speech (TTS), bundled in a single repo and venv.

Component	Engine	Port	Endpoint
STT	faster-whisper (CTranslate2)	8741	/transcribe
TTS	OmniVoice (k2-fsa, 600+ languages)	8742	/synthesize

Quick Start

TTS is optional — installing OmniVoice pulls torch+cu128 (~5 GB extra). Pick one:

chmod +x install.sh

# STT only (default — faster-whisper)
./install.sh

# STT + TTS
./install.sh --with-tts

Or use uv directly:

uv sync                      # STT only
uv sync --extra tts          # STT + TTS

Then start either / both servers:

.venv/bin/python whisper_api.py        # STT  on :8741
.venv/bin/python omnivoice_api.py      # TTS  on :8742  (requires --with-tts)

STT API (faster-whisper)

curl -F "file=@audio.mp3" http://localhost:8741/transcribe        # JSON
curl -F "file=@audio.mp3" http://localhost:8741/transcribe/text   # plain text
curl http://localhost:8741/health                                  # health check

CLI wrapper

.venv/bin/whisper-transcribe audio.mp3            # plain text
.venv/bin/whisper-transcribe --json audio.mp3    # JSON with segments

STT endpoints

Endpoint	Method	Description
/health	GET	Service health check
/transcribe	POST	Transcribe audio, returns JSON with language, segments
/transcribe/text	POST	Transcribe audio, returns plain text only

STT configuration

Environment variables (see faster-whisper.service):

Variable	Default	Description
WHISPER_MODEL	base	tiny, base, small, medium, large-v3, large-v3-turbo, distil-large-v3
WHISPER_DEVICE	cpu	cpu or cuda
WHISPER_COMPUTE_TYPE	int8	int8 (CPU), float16 (GPU)
WHISPER_HOST	0.0.0.0	Bind address
WHISPER_PORT	8741	Listen port

Recommended STT models by VRAM

VRAM	Model	Compute Type
4 GB	small	int8
6 GB	large-v3-turbo	float16
8 GB+	large-v3	float16

TTS API (OmniVoice)

OmniVoice is a zero-shot TTS model supporting 600+ languages, voice cloning, and voice design.

# Auto voice (model picks one)
curl -X POST http://localhost:8742/synthesize \
  -F "text=Hello, world." \
  -o out.wav

# Voice cloning (from a reference clip)
curl -X POST http://localhost:8742/synthesize \
  -F "text=Hello, this is a clone." \
  -F "ref_audio=@reference.wav" \
  -F "ref_text=Transcript of the reference audio." \
  -o out.wav

# Voice design (describe the voice)
curl -X POST http://localhost:8742/synthesize \
  -F "text=Hello, this is a designed voice." \
  -F "instruct=female, british accent" \
  -o out.wav

TTS endpoints

Endpoint	Method	Description
/health	GET	Service health, model, device, sample rate
/synthesize	POST	Generate speech, returns audio/wav (24 kHz, mono, PCM16)
/synthesize/metadata	POST	Same inputs as /synthesize, returns JSON diagnostics only (no audio)
/synthesize/attributes	GET	List supported voice-design attributes (gender, age, pitch, accents, dialects)

TTS request fields (multipart form-data)

Field	Required	Description
text	yes	Text to synthesize
mode	no	auto | clone | design (auto-inferred from other fields if omitted)
ref_audio	no	Reference audio file for cloning (3–10 s recommended)
ref_text	no	Transcript of ref_audio. If omitted, Whisper ASR auto-transcribes it
instruct	no	Voice design instruction, e.g. "male, british accent, low pitch"
language	no	Force language (e.g. English, en). Default: auto-detect
speed	no	Speed factor, default 1.0. Ignored if target_duration_sec is set
target_duration_sec	no	Fix output to a specific duration (overrides speed)
num_step	no	Diffusion steps (default 32, lower = faster)
guidance_scale	no	CFG scale (default 2.0)
denoise	no	Apply denoising (default true)
postprocess_output	no	Trim long silences (default true)

TTS response headers (on /synthesize)

X-Mode-Used, X-Duration-Ms, X-Generation-Ms — diagnostics without a separate metadata call.

TTS configuration

Environment variables (see omnivoice.service):

Variable	Default	Description
OMNIVOICE_MODEL	k2-fsa/OmniVoice	Model checkpoint or HF repo id
OMNIVOICE_DEVICE	auto (cuda/cpu)	Inference device
OMNIVOICE_HOST	0.0.0.0	Bind address
OMNIVOICE_PORT	8742	Listen port
OMNIVOICE_LOAD_ASR	1	Load Whisper ASR for auto-transcribing ref_audio (set 0 to disable)
OMNIVOICE_ASR_MODEL	openai/whisper-large-v3-turbo	ASR model used when ref_text is omitted

GPU Setup

CUDA libs are bundled via nvidia-cublas-cu12 / nvidia-cudnn-cu12 and torch is installed from the cu128 PyTorch index.

For STT on GPU, set: WHISPER_DEVICE=cuda WHISPER_COMPUTE_TYPE=float16. For TTS, GPU is auto-detected (override with OMNIVOICE_DEVICE).

If libcudnn cannot be found, prepend it to LD_LIBRARY_PATH:

VENV=$(pwd)/.venv/lib/python3.12/site-packages/nvidia
export LD_LIBRARY_PATH="$VENV/cudnn/lib:$VENV/cublas/lib:${LD_LIBRARY_PATH:-}"

The systemd units (faster-whisper.service, omnivoice.service) handle this automatically.

Running as systemd user services

cp faster-whisper.service ~/.config/systemd/user/
cp omnivoice.service       ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now faster-whisper omnivoice

Integration with OpenClaw

Add to OpenClaw's openclaw.json:

{
  "tools": {
    "media": {
      "audio": {
        "enabled": true,
        "models": [
          {
            "type": "cli",
            "command": "curl",
            "args": ["-s", "-X", "POST", "http://<HOST>:8741/transcribe/text", "-F", "file=@{{MediaPath}}"],
            "timeoutSeconds": 60
          }
        ]
      }
    }
  }
}