# CLI and Server

**Production URLs:**
- `https://bolovoice.apps.minascode.com/` (preferred)
- `https://omnivoice.apps.minascode.com/`

> These are public endpoints backed by the production server at `192.168.1.109:8880`.

The Bolo Voice Service ships two primary user-facing binaries:

- `bolo-tts-cli`
- `bolo-tts-server`

## CLI Overview

The CLI supports:

- full end-to-end inference
- prompt preparation and inspection
- stage0 token generation
- stage1 decoding
- batch inference

Main commands:

- `infer`
- `infer-batch`
- `prepare-prompt`
- `stage1-prepare`
- `stage1-decode`
- `stage0-generate`
- `stage0-debug`
- `artifacts validate`

## Common Inference Modes

### 1. Auto Voice

No reference audio and no voice-design instruction. The model chooses a voice automatically.

```powershell
cargo run -p bolo-tts-cli -- infer `
  --model model `
  --text "Hello, this is an auto voice example." `
  --language en `
  --output out\auto.wav `
  --device cpu `
  --dtype f32
```

### 2. Voice Clone

Reference audio plus reference text, or reference audio with ASR fallback when `--ref-text` is omitted.

```powershell
cargo run -p bolo-tts-cli -- infer `
  --model model `
  --text "This sample should preserve the reference speaking style." `
  --language en `
  --ref-audio ref.wav `
  --ref-text (Get-Content ref_text.txt -Raw) `
  --output out\clone.wav `
  --device cpu `
  --dtype f32
```

### 3. Voice Design

No reference audio. The speaker is described with `--instruct`.

```powershell
cargo run -p bolo-tts-cli -- infer `
  --model model `
  --text "This sample uses a designed voice." `
  --language en `
  --instruct "female, low pitch, british accent" `
  --output out\design.wav `
  --device cpu `
  --dtype f32
```

### 4. Combined Clone + Design

Reference audio **and** voice-design instruction can be provided at the same time. The model will attempt to clone the voice from the reference audio while applying the style modifications described in the instruction (e.g., changing gender, pitch, or accent).

This works on the `infer` command by combining flags, and on the server via `/v1/audio/speech` or `/clone` when an `instruct` field is also present.

```powershell
cargo run -p bolo-tts-cli -- infer `
  --model model `
  --text "This sample clones the voice but shifts it." `
  --language en `
  --ref-audio ref.wav `
  --ref-text "Reference transcript text" `
  --instruct "female, low pitch, british accent" `
  --output out\clone_design.wav `
  --device cpu `
  --dtype f32
```

On the server you can use a profile (which provides `ref_audio`) together with `instruct`:

```bash
curl -X POST http://localhost:8880/v1/audio/speech \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Hello world",
    "voice": "vp_1",
    "instruct": "female, low pitch, british accent"
  }' --output clone_design.wav
```

> **Note:** `/clone` and `/design` endpoints are identical to `/v1/audio/speech` except they enforce that `ref_audio` or `instruct` is present, respectively. You can still send the other field in the same request; the pipeline handles both.

## Device and DType Selection

Accepted device values:

- `auto`
- `cpu`
- `cuda`
- `cuda:N`
- `mps`
- `metal`

Accepted dtype values:

- `auto`
- `f32`
- `f16`
- `bf16`

Current implementation notes:

- `auto` device prefers `CUDA -> Metal -> CPU`
- `auto` dtype currently resolves to `f32`
- CPU-only inference is supported
- GPU acceleration is optional but preferred when available

## Important CPU Note About `--seed`

CPU inference works, but `--seed` is currently not a reliable CPU option in this workspace.

Observed local behavior:

- CPU inference without `--seed`: works
- CPU inference with `--seed`: can fail with `Candle(cannot seed the CPU rng with set_seed)`

Until that is fixed in the port or upstream backend behavior changes, omit `--seed` for CPU runs.

## Batch Inference

`infer-batch` reads a JSONL test list and writes generated WAV files into a result directory.

Useful controls:

- `--batch-size`
- `--batch-duration`
- `--nj-per-gpu`
- `--warmup`

Use this when you want to process multiple samples under one runtime configuration.

## OpenAI-Compatible Server

`bolo-tts-server` exposes:

- `GET /`
- `GET /health`
- `GET /v1/models`
- `POST /v1/audio/speech`
- `POST /v1/audio/denoise` — Remove background noise from audio (DeepFilterNet)
- `POST /v1/synthesize` — Vapi-compatible TTS webhook
- `GET  /v1/ws/transcribe` — Vapi-compatible WebSocket transcriber

Operational behavior:

- `GET /health` returns `200` only after the runtime is ready to serve synthesis requests
- during startup failures or model loading, `GET /health` returns `503`
- all routes can be mounted behind `--base-path`
- browser preflight requests are supported via CORS and `OPTIONS`

Authentication:

- required via `Authorization: Bearer <token>`
- token source: `--api-key` or `OMNIVOICE_API_KEY`

Start the server:

```powershell
$env:OMNIVOICE_API_KEY = "local-dev-token"
cargo run -p bolo-tts-server -- `
  --model model `
  --host 127.0.0.1 `
  --port 8000 `
  --base-path /edge `
  --request-timeout-secs 300 `
  --device auto `
  --dtype auto
```

## `POST /v1/audio/speech`

Base request fields:

- `model`
- `input`
- `voice`
- `response_format`

Transport:

- `application/json`
- `multipart/form-data`

For multipart requests, `ref_audio` can be uploaded as a real file instead of a base64 data URI.

Supported `response_format` values:

- `wav`
- `pcm`
- `mp3`

Optional request extensions accepted by this server:

- `language`
- `duration`
- `speed`
- `ref_text`
- `ref_audio` as a base64 data URI
- `instruct`
- `asr_model`
- `seed`
- `num_step`
- `guidance_scale`
- `t_shift`
- `layer_penalty_factor`
- `position_temperature`
- `class_temperature`
- `preprocess_prompt`
- `postprocess_output`
- `denoise`
- `audio_chunk_duration`
- `audio_chunk_threshold`

> **Warning:** The API documentation in `server.rs` previously showed example values of `1.0` and `0.5` for these fields. Those are **not** the actual defaults. The real defaults are `15.0` and `30.0` seconds. Setting these too low (e.g. `1.0` and `0.5`) will cause excessive chunking, audible cross-fade gaps, and fast-playback artifacts for long text.
- `stream_format`

Multipart notes:

- `ref_audio` is accepted as an uploaded audio file and decoded through the existing OmniVoice audio loader
- JSON requests remain backward-compatible and still accept `ref_audio` as a base64 data URI
- `voice` is accepted for OpenAI-shape compatibility but does not currently select a server-side voice catalog

Example request:

```json
{
  "model": "default",
  "input": "hello",
  "voice": "alloy",
  "response_format": "wav",
  "language": "en",
  "speed": 1.0,
  "instruct": "female, low pitch, british accent",
  "num_step": 16
}
```

Example multipart request:

```bash
curl http://127.0.0.1:8000/v1/audio/speech \
  -H "Authorization: Bearer local-dev-token" \
  -F model=default \
  -F input="hello" \
  -F voice=alloy \
  -F language=en \
  -F ref_text="reference text" \
  -F ref_audio=@ref.wav \
  -F response_format=wav
```

## Voice Profiles

The server supports persistent voice profiles for voice cloning. A profile stores reference audio with an optional `ref_text` transcript and optional `language` tag. When you synthesize with a profile, the server automatically injects the stored audio, transcript, and language into the generation request (unless explicitly overridden by the request).

### Why `ref_text` Matters

Without `ref_text`, the server runs Whisper ASR on the profile audio every time it is used to obtain the reference transcript. If the ASR makes mistakes (e.g. on noisy audio, uncommon names, or non-English speech), the cloned voice quality degrades.

By storing `ref_text` at profile creation time, you bypass ASR entirely for that profile. The stored transcript is used directly.

### Profile Endpoints

| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/v1/voices/profiles` | Create a new profile |
| `GET` | `/v1/voices/profiles` | List all custom profiles |
| `GET` | `/v1/voices/profiles/{id}` | Get a single profile |
| `PUT` | `/v1/voices/profiles/{id}` | Update name/description/ref_text/language |
| `DELETE` | `/v1/voices/profiles/{id}` | Delete a profile |
| `GET` | `/v1/voices` | List built-in + custom voices |

OpenAI-compatible aliases:

| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/v1/audio/voices` | Create profile (OpenAI shape) |
| `GET` | `/v1/audio/voices` | List profiles (OpenAI shape) |
| `GET` | `/v1/audio/voices/{id}` | Get profile (OpenAI shape) |
| `DELETE` | `/v1/audio/voices/{id}` | Delete profile (OpenAI shape) |

### Creating a Profile

Multipart request. Accepted audio: WAV, MP3, OGG, OPUS, FLAC, AAC. Auto-converted to 24kHz mono 16-bit WAV internally. Long audio (>30s) is trimmed via silence-gap detection.

Fields:

- `file` (required): audio file
- `name` (optional): profile name (default: `"unnamed"`)
- `description` (optional): profile description
- `ref_text` (optional): exact transcript of the spoken audio
- `language` (optional): language code (e.g. `"en"`, `"ur"`, `"zh"`). When set, this language is used as the default for synthesis with this profile. The request-level language takes precedence if both are specified.
- `password` (optional): password-protect the profile
  - Only the creator knows the plaintext password.
  - The server stores only a **bcrypt hash** — never the plaintext.
  - Anyone using this profile for synthesis must provide `voice_password`.
- `denoise` (optional): `"true"` or `"false"` — remove background noise using DeepFilterNet before storing the profile audio. Defaults to `false`.

```bash
curl -X POST http://localhost:8080/v1/voices/profiles \
  -H "Authorization: Bearer $BOLO_TTS_API_KEY" \
  -F 'file=@sample.wav' \
  -F 'name=My Voice' \
  -F 'description=Warm male voice' \
  -F 'ref_text=This is the exact spoken text from the sample' \
  -F 'language=en' \
  -F 'denoise=true'
```

Create a password-protected profile:

```bash
curl -X POST http://localhost:8080/v1/voices/profiles \
  -H "Authorization: Bearer $BOLO_TTS_API_KEY" \
  -F 'file=@sample.wav' \
  -F 'name=Premium Voice' \
  -F 'password=mysecret'
```

Response:

```json
{
  "id": "vp_1",
  "name": "My Voice",
  "description": "Warm male voice",
  "ref_text": "This is the exact spoken text from the sample",
  "language": "en",
  "created_at": "2025-01-01T00:00:00.000000Z",
  "updated_at": "2025-01-01T00:00:00.000000Z",
  "audio_path": "/path/to/profiles/vp_1/ref_audio.wav",
  "format": "wav",
  "sample_rate": 24000,
  "duration_seconds": 3.5
}
```

### Updating `ref_text` and `language`

```bash
curl -X PUT http://localhost:8080/v1/voices/profiles/vp_1 \
  -H "Authorization: Bearer $BOLO_TTS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"ref_text": "Corrected transcript", "language": "en"}'
```

To clear the stored `ref_text`, send an empty string:

```bash
curl -X PUT http://localhost:8080/v1/voices/profiles/vp_1 \
  -H "Authorization: Bearer $BOLO_TTS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"ref_text": ""}'
```

### Updating Password

You can password-protect an existing profile at any time, even if it was created without a password.

```bash
curl -X PUT http://localhost:8080/v1/voices/profiles/vp_1 \
  -H "Authorization: Bearer $BOLO_TTS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"password": "newsecret"}'
```

To remove the password, send an empty string:

```bash
curl -X PUT http://localhost:8080/v1/voices/profiles/vp_1 \
  -H "Authorization: Bearer $BOLO_TTS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"password": ""}'
```

### Using a Profile for Synthesis

Pass the profile ID as `voice`. If the profile has a stored language, it is used automatically. You can override it with the `language` field:

```bash
curl -X POST http://localhost:8080/v1/audio/speech \
  -H "Authorization: Bearer $BOLO_TTS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "default",
    "input": "Hello from my stored voice.",
    "voice": "vp_1",
    "language": "en"
  }'
```

For a **password-protected** profile, add `voice_password`:

```bash
curl -X POST http://localhost:8080/v1/audio/speech \
  -H "Authorization: Bearer $BOLO_TTS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "default",
    "input": "Hello from my premium voice.",
    "voice": "vp_2",
    "voice_password": "mysecret"
  }'
```

Behavior:

- If the profile has `ref_text`, it is injected automatically. Whisper ASR is skipped.
- If the profile has no `ref_text`, the server falls back to Whisper ASR on the stored audio.
- If you also pass an explicit `ref_text` in the speech request, it overrides the profile one for that request only.
- If the profile has a stored `language`, it is used as the default language for synthesis. The request-level `language` field overrides it if provided.

### Denoise Audio

Remove background noise from any audio file using DeepFilterNet. Returns a WAV file.

```bash
curl -X POST http://localhost:8080/v1/audio/denoise \
  -H "Authorization: Bearer $BOLO_TTS_API_KEY" \
  -F 'file=@noisy_recording.wav' \
  --output denoised.wav
```

Supported input formats: WAV, MP3, OGG, FLAC. Output is always 48kHz mono WAV (DeepFilterNet native format), resampled to 24kHz mono 16-bit WAV internally if used for profiles.

This endpoint is also available as a tool in the WebSocket demo UI (`/websocket`).
- You cannot pass both `voice: "<profile_id>"` and `ref_audio` in the same request — this returns `422 Conflict`.
- If a profile is password-protected, synthesis without `voice_password` returns `401 Unauthorized`.
- Wrong `voice_password` also returns `401 Unauthorized`.

## `POST /v1/synthesize` (Vapi-compatible TTS webhook)

This endpoint accepts Vapi's custom TTS webhook format and returns raw PCM s16le mono audio.

**Authentication** (in priority order):
1. `X-VAPI-SECRET` header
2. `?x_vapi_secret=<secret>` query parameter
3. `Authorization: Bearer <api-key>` header
4. `?api_key=<key>` query parameter

**Request body** (JSON):

```json
{
  "message": {
    "type": "voice-request",
    "text": "Text to synthesize",
    "sampleRate": 24000,
    "voice": "vp_1",
    "instruct": "female, low pitch",
    "language": "en",
    "speed": 1.0,
    "duration": 5.0,
    "guidanceScale": 2.0,
    "numStep": 32,
    "denoise": true,
    "voicePassword": "secret123"
  }
}
```

**Query parameters** — same keys as message body. Body values override query.

```bash
curl -X POST "http://localhost:8080/v1/synthesize?voice=vp_1&instruct=female%20low%20pitch&x_vapi_secret=my-vapi-secret" \
  -H "Content-Type: application/json" \
  -d '{"message":{"type":"voice-request","text":"Hello!","sampleRate":24000}}' \
  --output speech.pcm
```

**Supported sample rates:** 8000, 16000, 22050, 24000, 44100 Hz

**Response:** `Content-Type: application/octet-stream` — raw PCM s16le mono at requested rate.

### Built-in Voices

The server has 14 built-in voice presets that do not require reference audio:

`alloy`, `ash`, `ballad`, `coral`, `echo`, `fable`, `onyx`, `nova`, `sage`, `shimmer`, `verse`, `marin`, `cedar`, `auto`

Using a built-in voice does not trigger profile resolution and does not conflict with an explicit `ref_audio`.

Server behavior is tested against the OpenAI-compatible surface in [crates/bolo-tts-server/tests/openai_server.rs](../crates/bolo-tts-server/tests/openai_server.rs).