Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,7 @@
"server/services/tts/resembleai",
"server/services/tts/rime",
"server/services/tts/sarvam",
"server/services/tts/smallest",
"server/services/tts/speechmatics",
"server/services/tts/xtts"
]
Expand Down
1 change: 1 addition & 0 deletions server/services/supported-services.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ Text-to-Speech services receive text input and output audio streams or chunks.
| [ResembleAI](/server/services/tts/resembleai) | `pip install "pipecat-ai[resembleai]"` |
| [Rime](/server/services/tts/rime) | `pip install "pipecat-ai[rime]"` |
| [Sarvam](/server/services/tts/sarvam) | No dependencies required |
| [Smallest AI](/server/services/tts/smallest) | `pip install "pipecat-ai[smallest]"` |
| [Speechmatics](/server/services/tts/speechmatics) | `pip install "pipecat-ai[speechmatics]"` |
| [XTTS](/server/services/tts/xtts) | `pip install "pipecat-ai[xtts]"` |

Expand Down
175 changes: 175 additions & 0 deletions server/services/tts/smallest.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
---
title: "Smallest AI"
description: "Text-to-speech service using Smallest AI's WebSocket streaming API"
---

## Overview

Smallest AI provides real-time text-to-speech synthesis through a WebSocket-based integration with their Waves API. The service supports configurable voice parameters, multiple languages, and handles interruptions by reconnecting the WebSocket.

<CardGroup cols={2}>
<Card
title="Smallest AI TTS API Reference"
icon="code"
href="https://reference-server.pipecat.ai/en/latest/api/pipecat.services.smallest.tts.html"
>
Complete API reference for all parameters and methods
</Card>
<Card
title="Example Implementation"
icon="play"
href="https://github.com/pipecat-ai/pipecat/blob/main/examples/foundational/07zl-interruptible-smallest.py"
>
Complete example with WebSocket streaming
</Card>
</CardGroup>

## Installation

```bash
pip install "pipecat-ai[smallest]"
```

## Prerequisites

1. **Smallest AI Account**: Sign up at [Smallest AI](https://www.smallest.ai/)
2. **API Key**: Generate an API key from your account dashboard

Set the following environment variable:

```bash
export SMALLEST_API_KEY=your_api_key
```

## Configuration

<ParamField path="api_key" type="str" required>
Smallest AI API key for authentication.
</ParamField>

<ParamField path="base_url" type="str" default="wss://waves-api.smallest.ai">
Base WebSocket URL for the Smallest API. Override for custom or proxied
deployments.
</ParamField>

<ParamField path="sample_rate" type="int" default="None">
Output audio sample rate in Hz. When `None`, uses the pipeline's configured
sample rate.
</ParamField>

<ParamField path="settings" type="SmallestTTSService.Settings" default="None">
Runtime-configurable settings. See [Settings](#settings) below.
</ParamField>

### Settings

Runtime-configurable settings passed via the `settings` constructor argument using `SmallestTTSService.Settings(...)`. These can be updated mid-conversation with `TTSUpdateSettingsFrame`. See [Service Settings](/guides/fundamentals/service-settings) for details.

| Parameter | Type | Default | Description |
| ------------- | ----------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------- |
| `model` | `str` | `lightning-v3.1` | Model identifier: `lightning-v2` or `lightning-v3.1`. Model changes require WebSocket reconnection. _(Init-only setting)_ |
| `voice` | `str` | `sophia` | Voice identifier. |
| `language` | `Language \| str` | `Language.EN` | Language code for synthesis. |
| `speed` | `float` | `None` | Speech speed multiplier. When `None`, uses API defaults. |
| `consistency` | `float` | `None` | Consistency level for voice generation (0-1). Only supported by `lightning-v2`. When `None`, uses API defaults. |
| `similarity` | `float` | `None` | Similarity level for voice generation (0-1). Only supported by `lightning-v2`. When `None`, uses API defaults. |
| `enhancement` | `int` | `None` | Enhancement level for voice generation (0-2). Only supported by `lightning-v2`. When `None`, uses API defaults. |

<Note>
`None` values use the Smallest AI API defaults. The `consistency`,
`similarity`, and `enhancement` parameters are only supported by the
`lightning-v2` model.
</Note>

## Usage

### Basic Setup

```python
from pipecat.services.smallest import SmallestTTSService

tts = SmallestTTSService(
api_key=os.getenv("SMALLEST_API_KEY"),
settings=SmallestTTSService.Settings(
voice="sophia",
),
)
```

### With Voice Customization

```python
from pipecat.transcriptions.language import Language

tts = SmallestTTSService(
api_key=os.getenv("SMALLEST_API_KEY"),
settings=SmallestTTSService.Settings(
voice="sophia",
language=Language.ES,
speed=1.2,
),
)
```

### Using Lightning V2 Model

```python
from pipecat.services.smallest.tts import SmallestTTSModel

tts = SmallestTTSService(
api_key=os.getenv("SMALLEST_API_KEY"),
settings=SmallestTTSService.Settings(
model=SmallestTTSModel.LIGHTNING_V2,
voice="sophia",
consistency=0.7,
similarity=0.8,
enhancement=1,
),
)
```

### Updating Settings at Runtime

Voice settings can be changed mid-conversation using `TTSUpdateSettingsFrame`:

```python
from pipecat.frames.frames import TTSUpdateSettingsFrame
from pipecat.services.smallest.tts import SmallestTTSSettings

await task.queue_frame(
TTSUpdateSettingsFrame(
delta=SmallestTTSSettings(
voice="new_voice",
speed=1.1,
)
)
)
```

<Tip>
Changing the `model` setting will trigger a WebSocket reconnection, which may
cause a brief interruption in service.
</Tip>

## Notes

- **WebSocket streaming**: The service uses WebSocket connections for real-time streaming. The connection is automatically managed and will reconnect if interrupted.
- **Keepalive**: The service sends periodic keepalive messages (every 30 seconds) to prevent idle timeouts on the WebSocket connection.
- **Model-specific parameters**: The `consistency`, `similarity`, and `enhancement` parameters are only effective when using the `lightning-v2` model. They are ignored by `lightning-v3.1`.
- **Language support**: Supports multiple languages including Arabic, Bengali, German, English, Spanish, French, Gujarati, Hebrew, Hindi, Italian, Kannada, Marathi, Dutch, Polish, Russian, and Tamil.

## Event Handlers

Smallest AI TTS supports the standard [service connection events](/server/utilities/service-events):

| Event | Description |
| --------------------- | ----------------------------------- |
| `on_connected` | Connected to Smallest AI WebSocket |
| `on_disconnected` | Disconnected from Smallest WebSocket |
| `on_connection_error` | WebSocket connection error occurred |

```python
@tts.event_handler("on_connected")
async def on_connected(service):
print("Connected to Smallest AI")
```
Loading