Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions api-reference/server/services/llm/together.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ from pipecat.services.together import TogetherLLMService

llm = TogetherLLMService(
api_key=os.getenv("TOGETHER_API_KEY"),
model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
model="zai-org/GLM-5.1",
)
```

Expand All @@ -107,7 +107,7 @@ from pipecat.services.together import TogetherLLMService
llm = TogetherLLMService(
api_key=os.getenv("TOGETHER_API_KEY"),
settings=TogetherLLMService.Settings(
model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
model="zai-org/GLM-5.1",
temperature=0.7,
top_p=0.9,
max_completion_tokens=1024,
Expand Down
148 changes: 148 additions & 0 deletions api-reference/server/services/stt/together.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
---
title: "Together AI"
description: "Speech-to-text service using Together AI's real-time transcription API"
---

## Overview

`TogetherSTTService` provides real-time speech recognition using Together AI's WebSocket API with OpenAI-compatible speech-to-text endpoints. It supports streaming transcription with interim results and automatic reconnection.

<CardGroup cols={2}>
<Card
title="Together AI STT API Reference"
icon="code"
href="https://reference-server.pipecat.ai/en/latest/api/pipecat.services.together.stt.html"
>
Pipecat's API methods for Together AI STT
</Card>
<Card
title="Example Implementation"
icon="play"
href="https://github.com/pipecat-ai/pipecat/blob/main/examples/transcription/transcription-together.py"
>
Complete transcription example
</Card>
<Card
title="Together AI Documentation"
icon="book"
href="https://docs.together.ai/reference/audio-transcriptions-realtime"
>
Official Together AI Realtime API documentation
</Card>
<Card
title="Together AI Platform"
icon="microphone"
href="https://together.ai/"
>
Access models and manage API keys
</Card>
</CardGroup>

## Installation

To use Together AI STT services, install the required dependencies:

```bash
uv add "pipecat-ai[together]"
```

## Prerequisites

### Together AI Account Setup

Before using Together AI STT services, you need:

1. **Together AI Account**: Sign up at [Together AI](https://together.ai/)
2. **API Key**: Generate an API key from your account dashboard
3. **Model Selection**: Choose from available transcription models

### Required Environment Variables

- `TOGETHER_API_KEY`: Your Together AI API key for authentication

## Configuration

<ParamField path="api_key" type="str" required>
Together AI API key for authentication.
</ParamField>

<ParamField path="sample_rate" type="int" default="None">
Audio sample rate in Hz. When `None`, uses the pipeline's configured sample
rate.
</ParamField>

<ParamField path="base_url" type="str" default="wss://api.together.ai/v1">
WebSocket base URL for Together AI API.
</ParamField>

<ParamField path="settings" type="TogetherSTTService.Settings" default="None">
Runtime-configurable settings. See [Settings](#settings) below.
</ParamField>

<ParamField path="ttfs_p99_latency" type="float" default="1.00">
P99 latency from speech end to final transcript in seconds. Override for your
deployment. See
[https://github.com/pipecat-ai/stt-benchmark](https://github.com/pipecat-ai/stt-benchmark).
</ParamField>

### Settings

Runtime-configurable settings passed via the `settings` constructor argument using `TogetherSTTService.Settings(...)`. These can be updated mid-conversation with `STTUpdateSettingsFrame`. See [Service Settings](/pipecat/fundamentals/service-settings) for details.

| Parameter | Type | Default | Description |
| ---------- | ----------------- | --------------------------- | ----------------------------------------- |
| `model` | `str` | `"openai/whisper-large-v3"` | Model identifier. _(Inherited.)_ |
| `language` | `Language \| str` | `Language.EN` | Language for transcription. _(Inherited)_ |

## Usage

### Basic Setup

```python
import os
from pipecat.services.together import TogetherSTTService

stt = TogetherSTTService(
api_key=os.getenv("TOGETHER_API_KEY"),
)
```

### With Custom Settings

```python
from pipecat.services.together import TogetherSTTService
from pipecat.transcriptions.language import Language

stt = TogetherSTTService(
api_key=os.getenv("TOGETHER_API_KEY"),
settings=TogetherSTTService.Settings(
model="openai/whisper-large-v3",
language=Language.EN,
),
)
```

### In a Voice Pipeline

```python
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.processors.audio.vad_processor import VADProcessor
from pipecat.services.together import TogetherSTTService

stt = TogetherSTTService(api_key=os.getenv("TOGETHER_API_KEY"))
vad_processor = VADProcessor(vad_analyzer=SileroVADAnalyzer())

pipeline = Pipeline([
transport.input(),
vad_processor,
stt,
# ... rest of pipeline
])
```

## Notes

- Together AI's STT service uses an OpenAI-compatible WebSocket protocol for real-time transcription.
- The service automatically handles reconnection on connection errors.
- Transcription is committed when `VADUserStoppedSpeakingFrame` is received.
2 changes: 2 additions & 0 deletions api-reference/server/services/supported-services.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ Speech-to-Text services receive and audio input and output transcriptions.
| [Smallest](/api-reference/server/services/stt/smallest) | `uv add "pipecat-ai[smallest]"` | Pipecat |
| [Soniox](/api-reference/server/services/stt/soniox) | `uv add "pipecat-ai[soniox]"` | Pipecat |
| [Speechmatics](/api-reference/server/services/stt/speechmatics) | `uv add "pipecat-ai[speechmatics]"` | Pipecat |
| [Together AI](/api-reference/server/services/stt/together) | `uv add "pipecat-ai[together]"` | Pipecat |
| [Uplift AI](/api-reference/server/services/stt/upliftai) | `uv pip install git+https://github.com/havkerboi123/pipecat-upliftai-stt.git` | Community |
| [Whisper](/api-reference/server/services/stt/whisper) | `uv add "pipecat-ai[whisper]"` | Pipecat |
| [xAI](/api-reference/server/services/stt/xai) | `uv add "pipecat-ai[xai]"` | Pipecat |
Expand Down Expand Up @@ -201,6 +202,7 @@ Text-to-Speech services receive text input and output audio streams or chunks.
| [Soniox](/api-reference/server/services/tts/soniox) | `uv add "pipecat-ai[soniox]"` | Pipecat |
| [Speechmatics](/api-reference/server/services/tts/speechmatics) | `uv add "pipecat-ai[speechmatics]"` | Pipecat |
| [Supertonic](/api-reference/server/services/tts/supertonic) | `uv add pipecat-supertonic` | Community |
| [Together AI](/api-reference/server/services/tts/together) | `uv add "pipecat-ai[together]"` | Pipecat |
| [Typecast](/api-reference/server/services/tts/typecast) | `uv add pipecat-ai-typecast` | Community |
| [Uplift AI](/api-reference/server/services/tts/upliftai) | `uv pip install git+https://github.com/havkerboi123/pipecat-upliftai-tts.git` | Community |
| [Voice.ai](/api-reference/server/services/tts/voiceai) | `uv pip install git+https://github.com/voice-ai/voice-ai-pipecat-tts.git` | Community |
Expand Down
152 changes: 152 additions & 0 deletions api-reference/server/services/tts/together.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
---
title: "Together AI"
description: "Text-to-speech service using Together AI's real-time WebSocket API"
---

## Overview

`TogetherTTSService` provides real-time text-to-speech using Together AI's WebSocket API. It supports streaming synthesis with configurable voice and model options, interruption handling, and automatic reconnection.

<CardGroup cols={2}>
<Card
title="Together AI TTS API Reference"
icon="code"
href="https://reference-server.pipecat.ai/en/latest/api/pipecat.services.together.tts.html"
>
Pipecat's API methods for Together AI TTS
</Card>
<Card
title="Example Implementation"
icon="play"
href="https://github.com/pipecat-ai/pipecat/blob/main/examples/voice/voice-together.py"
>
Complete voice bot example
</Card>
<Card
title="Together AI Documentation"
icon="book"
href="https://docs.together.ai/reference/audio-speech-websocket"
>
Official Together AI TTS WebSocket API documentation
</Card>
<Card
title="Together AI Platform"
icon="microphone"
href="https://together.ai/"
>
Access models and manage API keys
</Card>
</CardGroup>

## Installation

To use Together AI TTS services, install the required dependencies:

```bash
uv add "pipecat-ai[together]"
```

## Prerequisites

### Together AI Account Setup

Before using Together AI TTS services, you need:

1. **Together AI Account**: Sign up at [Together AI](https://together.ai/)
2. **API Key**: Generate an API key from your account dashboard
3. **Model Selection**: Choose from available TTS models and voices

### Required Environment Variables

- `TOGETHER_API_KEY`: Your Together AI API key for authentication

## Configuration

<ParamField path="api_key" type="str" required>
Together AI API key for authentication.
</ParamField>

<ParamField
path="url"
type="str"
default="wss://api.together.ai/v1/audio/speech/websocket"
>
WebSocket URL for Together AI TTS API.
</ParamField>

<ParamField path="sample_rate" type="int" default="24000">
Output sample rate for emitted PCM frames. Together AI streams at 24 kHz and
does not support other rates.
</ParamField>

<ParamField path="settings" type="TogetherTTSService.Settings" default="None">
Runtime-configurable settings. See [Settings](#settings) below.
</ParamField>

### Settings

Runtime-configurable settings passed via the `settings` constructor argument using `TogetherTTSService.Settings(...)`. These can be updated mid-conversation with `TTSUpdateSettingsFrame`. See [Service Settings](/pipecat/fundamentals/service-settings) for details.

| Parameter | Type | Default | Description |
| -------------------- | ----------------- | ---------------------- | ------------------------------------------------------------- |
| `model` | `str` | `"hexgrad/Kokoro-82M"` | Model identifier. _(Inherited.)_ |
| `voice` | `str` | `"af_heart"` | Voice identifier. _(Inherited.)_ |
| `language` | `Language \| str` | `Language.EN` | Language for synthesis. _(Inherited.)_ |
| `max_partial_length` | `int \| None` | `None` | Maximum partial text length for streaming. `None` for no cap. |

## Usage

### Basic Setup

```python
import os
from pipecat.services.together import TogetherTTSService

tts = TogetherTTSService(
api_key=os.getenv("TOGETHER_API_KEY"),
)
```

### With Custom Settings

```python
from pipecat.services.together import TogetherTTSService
from pipecat.transcriptions.language import Language

tts = TogetherTTSService(
api_key=os.getenv("TOGETHER_API_KEY"),
settings=TogetherTTSService.Settings(
model="hexgrad/Kokoro-82M",
voice="af_heart",
language=Language.EN,
),
)
```

### In a Voice Pipeline

```python
from pipecat.pipeline.pipeline import Pipeline
from pipecat.services.together import TogetherTTSService

tts = TogetherTTSService(
api_key=os.getenv("TOGETHER_API_KEY"),
settings=TogetherTTSService.Settings(
voice="af_heart",
model="hexgrad/Kokoro-82M",
),
)

pipeline = Pipeline([
# ... upstream processors
llm,
tts,
transport.output(),
])
```

## Notes

- Together AI TTS streams audio at 24 kHz. The service outputs 24 kHz signed 16-bit mono PCM; the transport layer resamples to the pipeline's configured rate if needed.
- The service supports interruption handling and automatically clears the text buffer when interrupted.
- Audio is streamed incrementally via WebSocket deltas for low-latency synthesis.
2 changes: 2 additions & 0 deletions docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -374,6 +374,7 @@
"api-reference/server/services/stt/smallest",
"api-reference/server/services/stt/soniox",
"api-reference/server/services/stt/speechmatics",
"api-reference/server/services/stt/together",
"api-reference/server/services/stt/upliftai",
"api-reference/server/services/stt/whisper",
"api-reference/server/services/stt/xai"
Expand Down Expand Up @@ -448,6 +449,7 @@
"api-reference/server/services/tts/soniox",
"api-reference/server/services/tts/speechmatics",
"api-reference/server/services/tts/supertonic",
"api-reference/server/services/tts/together",
"api-reference/server/services/tts/tts-cache",
"api-reference/server/services/tts/typecast",
"api-reference/server/services/tts/upliftai",
Expand Down
Loading