> ## Documentation Index
> Fetch the complete documentation index at: https://docs.brainstormer.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Realtime Audio Service

> Realtime Audio service: voice cloning via ElevenLabs, STT via Whisper, LiveKit room management, and voice conversation pipeline.

## Overview

**Port:** 4003

The Realtime Audio Service handles voice cloning (ElevenLabs), speech-to-text (Whisper), LiveKit room management, and the voice conversation pipeline.

<Warning>
  **Voice calls are currently disabled on the frontend ("Coming Soon").** The backend pipeline exists but the browser-side audio capture and HTTP turn processing is not yet implemented. See `docs/architecture/VOICE_CALL_IMPLEMENTATION_PLAN.md` for the full plan.
</Warning>

## Endpoints

### Voice Conversations

| Route                                           | Method | Purpose                                |
| ----------------------------------------------- | ------ | -------------------------------------- |
| `/api/v1/voice-conversations/start`             | POST   | Create LiveKit room + session          |
| `/api/v1/voice-conversations/:sessionId/turn`   | POST   | Process audio turn (STT -> LLM -> TTS) |
| `/api/v1/voice-conversations/:sessionId/stream` | GET/WS | WebSocket streaming turn               |
| `/api/v1/voice-conversations/:sessionId/end`    | POST   | End session                            |
| `/api/v1/voice-conversations/:sessionId/stats`  | GET    | Session metrics                        |
| `/api/v1/voice-conversations/health`            | GET    | Infrastructure health                  |

### Voice Management

| Route                              | Method | Purpose                                |
| ---------------------------------- | ------ | -------------------------------------- |
| `/api/v1/agents/:agentId/voice`    | POST   | Create custom voice (multipart upload) |
| `/api/v1/agents/:agentId/voice`    | GET    | Get agent's custom voice               |
| `/api/v1/voices`                   | GET    | List voices                            |
| `/api/v1/voices/:voiceId/test`     | POST   | Test voice synthesis                   |
| `/api/v1/voices/:voiceId/settings` | PATCH  | Update voice settings                  |

### Service Info

| Route   | Method | Purpose                      |
| ------- | ------ | ---------------------------- |
| `/info` | GET    | Service info + feature flags |

## Config DB Fallback

API keys are loaded from the `platform_config` table at startup (encrypted, decrypted with JWT\_SECRET). Falls back to `process.env` if DB value not set.

**Keys loaded from config:**

* `ELEVENLABS_API_KEY`
* `LIVEKIT_API_KEY`
* `LIVEKIT_API_SECRET`
* `LIVEKIT_URL`
* `OPENAI_API_KEY`

## External Integrations

| Integration    | Config                                                 |
| -------------- | ------------------------------------------------------ |
| LiveKit        | `LIVEKIT_API_KEY`, `LIVEKIT_API_SECRET`, `LIVEKIT_URL` |
| ElevenLabs     | `ELEVENLABS_API_KEY`                                   |
| OpenAI Whisper | `OPENAI_API_KEY`                                       |
| Bot Service    | `BOT_SERVICE_URL` ([http://bot:4002](http://bot:4002)) |

## Voice Conversation Pipeline

```mermaid theme={null}
graph LR
    A[User Audio] --> B[STT<br/>Whisper]
    B --> C[Text Message]
    C --> D[Bot Service<br/>LLM Chat]
    D --> E[Response Text]
    E --> F[TTS<br/>ElevenLabs]
    F --> G[Audio Response]
```

## Billing Hooks

The service records provider costs through `record_external_cost_event()` using `RealtimeAudioBillingRepository`:

| Operation                  | Provider        |
| -------------------------- | --------------- |
| `voice_clone`              | elevenlabs      |
| `voice_tts`                | elevenlabs      |
| `voice_tts_stream_session` | elevenlabs      |
| `voice_tts_stream_usage`   | elevenlabs      |
| `voice_stt_transcription`  | openai\_whisper |

<Note>
  Billing writes are currently **best-effort** for realtime-audio paths. Errors are logged and request flow continues.
</Note>

## Known Issues

<Warning>
  These issues need to be fixed when implementing the full voice call feature:
</Warning>

* `sendMessageToBot()` calls wrong bot endpoint -- should be `POST /bots/:id/chat`
* `/turn` returns mock S3 URL instead of inline base64 audio
* Turn number hardcoded to 1
* LiveKit `RoomServiceClient` needs HTTPS URL (not WSS) -- handled with auto-conversion