# Add Additional Source Source: https://docs.deepdub.ai/api-reference/add-additional-source /managed-dub.openapi.json post /dubbing/job/{request_id}/sources Attach an additional source asset (subtitles, audio hints, etc.) to an existing job. The new source is ingested asynchronously. # Cancel Dubbing Job Source: https://docs.deepdub.ai/api-reference/cancel-dubbing-job /managed-dub.openapi.json delete /dubbing/job/{request_id} Mark a job as cancelled. Cancellation is permanent — a cancelled job cannot be revived; submit a new job instead. # Classify speaker gender Source: https://docs.deepdub.ai/api-reference/gender-detection/classify-speaker-gender post /gender-detection/classify Classify the gender of the speaker in an audio file. Accepts either an audio URL (S3 or HTTP) or base64-encoded audio data. Supports WAV, MP3, FLAC, OGG, and other common audio formats. # Classify speaker gender (file upload) Source: https://docs.deepdub.ai/api-reference/gender-detection/classify-speaker-gender-file-upload post /gender-detection/classify/upload Classify the gender of the speaker from an uploaded audio file. Accepts audio files in WAV, MP3, FLAC, OGG, and other common formats. # Get Dubbing Job Status Source: https://docs.deepdub.ai/api-reference/get-dubbing-job-status /managed-dub.openapi.json get /dubbing/job/{request_id} Return the current status, progress trace, export path, and additional products for a single dubbing job. # Get connections for a model Source: https://docs.deepdub.ai/api-reference/infrastructure/get-connections-for-a-model get /concurrent_connections/{model_id} Get the current number of machines and connections available for a specific model's autoscaling group. # Get connections for multiple models Source: https://docs.deepdub.ai/api-reference/infrastructure/get-connections-for-multiple-models get /concurrent_connections/batch Get autoscaling group information for multiple models at once. # Update model autoscaling group Source: https://docs.deepdub.ai/api-reference/infrastructure/update-model-autoscaling-group put /concurrent_connections/{model_id} Update the min, max, and desired capacity of the autoscaling group for a specific model. # Create a new issue Source: https://docs.deepdub.ai/api-reference/issues/create-issue post /issues Report a problem with a TTS generation by creating a new issue. Issues are tracked internally by the Deepdub team and used to improve voice quality. The `problemAudioFile` field can optionally include base64-encoded audio attached to the issue, and `problemSeconds` indicates the timestamp in the audio where the problem appears. ## Reporting generation problems Use this endpoint to flag a TTS generation that didn't sound right — a mispronunciation, an unwanted artifact, or any other quality issue. Reports are reviewed by the Deepdub team and feed into model improvements. To make a report actionable, include: * `generationId` — the ID of the affected generation * `generatedText` — the exact text passed to TTS * `problemWord` — the specific word or phrase that was generated incorrectly * `voicePromptId` — the voice prompt that was used * `problemSeconds` — timestamp (in seconds) where the problem occurs * `type` *(optional)* — `hallucinations` (the default) or `glossary` * `phoneticHeard` *(optional)* — how the word actually sounded, written phonetically * `phoneticExpected` *(optional)* — how the word should have sounded, written phonetically * `problemAudioFile` *(optional)* — base64-encoded clip of the problem audio * `additionalComments` *(optional)* — anything else that helps reproduce or explain the issue Use `type` to say what kind of problem you're reporting: `hallucinations` for words the model got wrong or invented, `glossary` for terms that need a permanent pronunciation entry. Omitting it defaults to `hallucinations`; any other value returns `400`. The `phoneticHeard` and `phoneticExpected` pair is the fastest way to make a mispronunciation actionable — it tells the team exactly what the model produced and what it should have produced, without anyone having to listen to the audio. The response includes an `id` you can use with the [GET](/api-reference/issues/get-issue), [PUT](/api-reference/issues/update-issue), and [DELETE](/api-reference/issues/delete-issue) endpoints. # Delete an issue Source: https://docs.deepdub.ai/api-reference/issues/delete-issue delete /issues/{id} Archive an issue. Archived issues are no longer visible via `GET /issues/{id}`. Deletes (archives) the issue. After deletion, [`GET /issues/{id}`](/api-reference/issues/get-issue) for the same ID returns `404`. # Get an issue by ID Source: https://docs.deepdub.ai/api-reference/issues/get-issue get /issues/{id} Retrieve an issue by its ID, including its current state (e.g., open, in progress, resolved). ## Issue states The `state` field tracks where a report sits in the review process: | State | Meaning | | ------------------------- | --------------------------------------- | | `open` | Received and waiting for review. | | `uploaded_for_correction` | Accepted and queued for a correction. | | `in_progress` | Actively being worked on. | | `resolved` | Fixed. | | `rejected` | Not actionable — see `rejectionReason`. | `rejectionReason` is returned only when `state` is `rejected`. A report closed as a repeat of an existing one comes back as `duplicate report`. # List issues Source: https://docs.deepdub.ai/api-reference/issues/list-issues get /issues List the issues reported by your account, newest first. Pagination is cursor-based: pass the `nextCursor` from a response as the next request's `cursor`, and stop once a response omits it. Returns the issues reported by your account, newest first. Only your own reports are visible — an API key never sees another customer's issues. ## Pagination Paging is cursor-based. Make the first request without a `cursor`, then pass the `nextCursor` from each response as the `cursor` of the next request: ```bash theme={null} curl "https://restapi.deepdub.ai/api/v1/issues?limit=50" \ -H "x-api-key: $DEEPDUB_API_KEY" ``` ```json theme={null} { "issues": [{ "id": "iss_12345abcde", "state": "open" }], "nextCursor": "eyJvZmZzZXQiOjUwfQ==" } ``` `nextCursor` is only returned when another page actually exists, so you can loop until it's absent without ever fetching an empty page: ```python theme={null} cursor, all_issues = None, [] while True: params = {"limit": 100, **({"cursor": cursor} if cursor else {})} page = requests.get(f"{BASE_URL}/issues", params=params, headers=headers).json() all_issues.extend(page["issues"]) cursor = page.get("nextCursor") if not cursor: break ``` `limit` defaults to 50 and is capped at 100 — asking for more returns 100 rather than an error. ## Filtering by date `from` and `to` bound the report date inclusively, and each accepts either `YYYY-MM-DD` or a full RFC3339 timestamp. A bare `to` date covers that entire UTC day, so `to=2026-08-31` includes issues reported at 23:59 that evening. ```bash theme={null} curl "https://restapi.deepdub.ai/api/v1/issues?from=2026-08-01&to=2026-08-31" \ -H "x-api-key: $DEEPDUB_API_KEY" ``` A date that parses as neither format returns `400`, as does a `from` later than the `to`. # Update an issue Source: https://docs.deepdub.ai/api-reference/issues/update-issue put /issues/{id} Update an existing issue. All fields are optional — only fields included in the body are updated. All fields in the request body are optional — only fields you include will be updated. # Update an issue's state Source: https://docs.deepdub.ai/api-reference/issues/update-issue-state patch /issues/{id}/state Move an issue through the review workflow. You can only change the state of issues your own API key created. Moves an issue through the review workflow. This is separate from [`PUT /issues/{id}`](/api-reference/issues/update-issue), which edits the contents of a report — this endpoint only changes its `state`. You can only change the state of issues created by your own API key; anything else returns `404`. ## States | State | Meaning | | ------------------------- | ------------------------------------------- | | `open` | Received and waiting for review. | | `uploaded_for_correction` | Accepted and queued for a correction. | | `in_progress` | Actively being worked on. | | `resolved` | Fixed. | | `rejected` | Not actionable. Requires `rejectionReason`. | Setting `state` to `rejected` without a `rejectionReason` returns `400`, as does any value outside the table above. ```json theme={null} { "state": "rejected", "rejectionReason": "not reproducible" } ``` # List Dubbing Jobs Source: https://docs.deepdub.ai/api-reference/list-dubbing-jobs /managed-dub.openapi.json get /dubbing/jobs Return a paginated list of dubbing jobs belonging to the authenticated customer. Cancelled jobs are excluded. # Live Streaming API Source: https://docs.deepdub.ai/api-reference/live/overview Live captions and voice dubbing on top of HLS, SRT, RTMP, and CMAF broadcast inputs The Live Streaming API is in early access. Endpoints, request shapes, and stream-type identifiers may change. Contact `support@deepdub.ai` to enable it on your account. ## Overview The Live Streaming API turns a broadcast video feed into live translated captions and dubbed audio in one or more target languages. You point Deepdub at your source stream (HLS, SRT, RTMP, or CMAF), pick target languages and voices, and receive back: * **Captions** in the target languages, delivered as an HLS WebVTT subtitle track and available in real time via WebSocket. * **Dubbed audio** rendered by Deepdub's TTS engine and packaged back into a broadcast output (HLS, SRT, or RTMP). Under the hood the API drives a live-broadcast orchestration provider on your behalf. You never authenticate to that provider directly — you use your standard Deepdub `x-api-key`. ## Base URL | Region | URL | | ------------ | ------------------------------------ | | US (default) | `https://restapi.deepdub.ai/live` | | EU | `https://restapi.eu.deepdub.ai/live` | Use the host for the region your account is provisioned in. The examples below use the US host; substitute the EU one throughout, including the captions WebSocket. Authentication is the same `x-api-key` header used everywhere else in the Deepdub API. See [Authentication](/authentication). ## Concepts A running or startable pipeline that ingests one source and produces one or more output streams. Services are long-lived resources that you `create` once and `start` / `stop` as needed. Accounts have a limit on the number of defined services — reuse and update them rather than creating a new one per broadcast. The upstream broadcast feed. Selected by an input stream type (e.g. `HlsPullSource`, `SrtPushSource`). Where translated captions and/or dubbed audio are delivered. Each service can have one or more outputs of types compatible with the chosen input. A `(language, TTS engine, voice)` triple. Multiple translations can run on the same service to produce several target languages simultaneously. ## Supported stream types This matrix is dynamic — always call the enum endpoints below for the authoritative list. The values shown reflect what is enabled at the time of writing. ### Input stream types Returned by `GET /live/enums/input-stream-types`: | Identifier | Protocol | Notes | | ------------------ | ----------- | ------------------------------------------------------------------------- | | `HlsPullSource` | HLS (pull) | Deepdub pulls your `.m3u8` playlist. | | `HlsPushSource` | HLS (push) | You push an HLS stream to a URL provided by Deepdub. | | `SrtPushSource` | SRT (push) | You push a Secure Reliable Transport stream to a URL provided by Deepdub. | | `SrtPullSource` | SRT (pull) | Deepdub pulls from your SRT listener. | | `RtmpPushSource` | RTMP (push) | You push an RTMP(S) stream to a URL provided by Deepdub. | | `CmafIngestSource` | CMAF ingest | Standards-based CMAF push ingest. | ### Output stream types Returned by `GET /live/enums/output-stream-types/{inputStreamType}`. The available outputs depend on the input. As of writing: | Input | Compatible outputs | | -------------------------------- | --------------------------------------------------------------------------- | | `HlsPullSource`, `HlsPushSource` | `HlsPushOutput`, `608` | | `SrtPushSource`, `SrtPullSource` | `SrtPushOutput`, `SrtPullOutput`, `RtmpPullOutput`, `RtmpPushOutput`, `608` | | `RtmpPushSource` | `SrtPushOutput`, `SrtPullOutput`, `RtmpPullOutput`, `RtmpPushOutput`, `608` | | `CmafIngestSource` | `HlsPushOutput`, `608` | `608` refers to CEA-608 line-21 captions embedded into the video output. ## Endpoints ### Services CRUD and lifecycle for live services. | Method | Path | Purpose | | -------- | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `GET` | `/live/services` | List services on your account. Query params: `page`, `perPage`, `search`, `scope`. | | `GET` | `/live/services/{serviceId}` | Fetch a single service and its full `config`. | | `POST` | `/live/services` | Create a new service. Returns `201` with the created service. | | `PUT` | `/live/services/{serviceId}` | Replace a service's `config` in place — use this to update the HLS/SRT URL, translations, or outputs without burning a slot on a new service. Same body shape as `POST`. | | `DELETE` | `/live/services/{serviceId}` | Delete a service. Returns `204`. | | `GET` | `/live/services/{serviceId}/status` | Runtime status (active, input connected, etc.). | | `POST` | `/live/services/{serviceId}/start` | Start a stopped service. | | `POST` | `/live/services/{serviceId}/stop` | Stop a running service. | The service `config` object is deep and evolves — build it by fetching a known-good service with `GET /live/services/{id}` and modifying the fields you need, then send it back with `PUT`. Deepdub Support can share a starter template if you don't have one yet. ### Enums Read-only endpoints for the discriminated unions that appear inside `config`. Call these when constructing a service to get the currently supported identifiers. | Method | Path | Purpose | | ------ | ----------------------------------------------------------- | --------------------------------------------------------------------------------------- | | `GET` | `/live/enums/build-versions` | Available live pipeline build versions. | | `GET` | `/live/enums/input-stream-types` | Input stream identifiers. | | `GET` | `/live/enums/output-stream-types/{inputStreamType}` | Output identifiers compatible with an input. | | `GET` | `/live/enums/translation-languages` | Target languages available for live translation. | | `GET` | `/live/enums/translation-tts-engines` | TTS engines you can pick for dubbing (`Deepdub`, `Amazon`, `Azure`, `ElevenLabs`, ...). | | `GET` | `/live/enums/translation-tts-voices/{language}/{ttsEngine}` | Voices available for a `(language, engine)` pair. Response shape varies per engine. | ### Captions WebSocket Real-time caption fan-out for a running service. ``` wss://restapi.deepdub.ai/live/services/{serviceId}/captions/ws?lang=es ``` Query parameters: BCP-47 language tag of the caption track to subscribe to (for example `es`, `fr-FR`, `de`). Optional explicit HLS WebVTT subtitle playlist URL. Only needed if the service's caption output URL can't be discovered automatically from its config. On connect the server first sends a `hello` frame with the resolved playlist URL, then a stream of caption frames as new cues appear. A `heartbeat` frame is emitted every few seconds when no cues are pending. ```json theme={null} { "type": "hello", "serviceId": 41823, "lang": "es", "playlistUrl": "https://.../subs/es.m3u8" } ``` ```json theme={null} { "type": "cue", "seq": 42, "start": 128.500, "end": 131.100, "text": "Hola, bienvenidos." } ``` ```json theme={null} { "type": "heartbeat" } ``` ## Quick start ```bash theme={null} curl -H "x-api-key: $DEEPDUB_API_KEY" \ https://restapi.deepdub.ai/live/enums/input-stream-types curl -H "x-api-key: $DEEPDUB_API_KEY" \ https://restapi.deepdub.ai/live/enums/output-stream-types/SrtPushSource ``` ```bash theme={null} curl -H "x-api-key: $DEEPDUB_API_KEY" \ "https://restapi.deepdub.ai/live/services?page=1&perPage=50" ``` Pick one with a working `config` and use it as your starting point: ```bash theme={null} curl -H "x-api-key: $DEEPDUB_API_KEY" \ https://restapi.deepdub.ai/live/services/41823 ``` To adjust the input URL on an existing service (no new slot): ```bash theme={null} curl -X PUT -H "x-api-key: $DEEPDUB_API_KEY" -H "Content-Type: application/json" \ --data @service.json \ https://restapi.deepdub.ai/live/services/41823 ``` Or create a new one: ```bash theme={null} curl -X POST -H "x-api-key: $DEEPDUB_API_KEY" -H "Content-Type: application/json" \ --data @service.json \ https://restapi.deepdub.ai/live/services ``` ```bash theme={null} curl -X POST -H "x-api-key: $DEEPDUB_API_KEY" \ https://restapi.deepdub.ai/live/services/41823/start curl -H "x-api-key: $DEEPDUB_API_KEY" \ https://restapi.deepdub.ai/live/services/41823/status ``` Accounts are capped on the number of defined services (active or not). Delete or archive unused ones to free slots: ```bash theme={null} curl -X DELETE -H "x-api-key: $DEEPDUB_API_KEY" \ https://restapi.deepdub.ai/live/services/41823 ``` ## Errors Errors are proxied from the underlying orchestration provider and normalized to standard HTTP codes: | Code | Meaning | | ---- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | 400 | Invalid request body. | | 401 | Missing or invalid `x-api-key`. | | 403 | Feature not enabled for your account. | | 404 | Service or resource not found. | | 422 | The `config` failed validation upstream. The response body includes an `upstream` field with the provider's original error for troubleshooting. | | 429 | Rate limit exceeded. | | 5xx | Deepdub or upstream provider error. | ## Notes and limits * Accounts have a **soft cap on the number of defined services** (typically 10) regardless of whether they are running. If you hit the cap, `PUT` an existing service to change its source instead of creating a new one, or `DELETE` unused ones. * The API is regional — see [Base URL](#base-url). Your API key works only against the region your account is provisioned in. * Live pipelines are billed separately from batch TTS. Contact `support@deepdub.ai` for pricing and to enable additional TTS engines beyond `Deepdub`. # Submit Dubbing Job Source: https://docs.deepdub.ai/api-reference/submit-dubbing-job /managed-dub.openapi.json post /dubbing/job Create a new dubbing job. The source video is validated, locales are resolved, and the job is queued for processing. Returns a requestId used to track the job. # Submit Redubbing Feedback Source: https://docs.deepdub.ai/api-reference/submit-redubbing-feedback /managed-dub.openapi.json patch /dubbing/job/{request_id} Submit quality feedback and issue reports for a completed dubbing job, triggering a resynthesis workflow. # Generate and stream TTS audio Source: https://docs.deepdub.ai/api-reference/tts/generate-and-stream-tts-audio post /tts Generate and stream TTS audio based on the provided text. Returns an audio stream in the specified format (default MP3). Supported formats: `mp3`, `opus`, `mulaw`. For `wav` or `s16le` output, use the WebSocket API. ## Regions The REST API runs in two regions. Use the host for the region your account is provisioned in — the playground above lets you switch between them. | Region | Base URL | | ------------ | -------------------------------------- | | US (default) | `https://restapi.deepdub.ai/api/v1` | | EU | `https://restapi.eu.deepdub.ai/api/v1` | Both regions serve every REST endpoint and take the same request bodies. See [Authentication](/authentication) for the WebSocket hosts. ## Supported languages | Language | Locale code | | ------------------------ | ----------- | | Arabic (Lebanon) | `ar-LB` | | Arabic (Qatar) | `ar-QA` | | Arabic (Saudi) | `ar-SA` | | Arabic (Standard) | `ar-SA` | | Arabic (Syrian) | `ar-SY` | | Czech (Standard) | `cs-CZ` | | Danish (Standard) | `da-DK` | | Dutch (Netherlands) | `nl-NL` | | English (Generic) | `en-GB` | | English (Standard) | `en-AU` | | English (United States) | `en-US` | | Estonian (Standard) | `et-EE` | | Finnish (Standard) | `fi-FI` | | French (Standard) | `fr-FR` | | German (Standard) | `de-DE` | | Greek (Standard) | `el-GR` | | Hebrew (Standard) | `he-IL` | | Hindi (Standard) | `hi-IN` | | Hungarian (Standard) | `hu-HU` | | Indonesian (Standard) | `id-ID` | | Italian (Standard) | `it-IT` | | Japanese (Standard) | `ja-JP` | | Korean (Standard) | `ko-KR` | | Macedonian (Standard) | `mk-MK` | | Norwegian (Standard) | `nb-NO` | | Polish (Standard) | `pl-PL` | | Portuguese (Brazil) | `pt-BR` | | Romanian (Standard) | `ro-RO` | | Russian (Standard) | `ru-RU` | | Spanish (Latam) | `es-419` | | Spanish (Latam — Mexico) | `es-MX` | | Spanish (Standard) | `es-ES` | | Swedish (Standard) | `sv-SE` | | Tamil (Standard) | `ta-IN` | | Thai (Standard) | `th-TH` | | Turkish (Standard) | `tr-TR` | ## Model-specific parameters `seed` applies to `dd-etts-1.1` only. Newer models — including the default `dd-etts-3.0` — do not use it, and setting it has no effect on their output. Do not rely on it to reproduce a generation on any model other than `dd-etts-1.1`. ## Supported output formats The REST API streams audio as raw bytes in the HTTP response body. Supported formats: | Format | Description | | ------- | ----------------------------------------------------------------------------------------------------- | | `mp3` | Compressed audio, smallest file size. **Default.** | | `opus` | High-quality compressed audio, efficient for streaming. | | `mulaw` | 8-bit µ-law encoding, commonly used in telephony. Defaults to 8000 Hz if no sample rate is specified. | The REST API supports `mp3`, `opus`, and `mulaw` only. For `wav` or `s16le` output, use the [Streaming Out API](/api-reference/websocket/overview). ## Sample rates Valid values are `8000`, `16000`, `22050`, `24000`, `32000`, `36000`, `44100`, and `48000` Hz; any other value is rejected with a 400. The internal generation runs at 48 kHz and is resampled to the requested rate. If no sample rate is specified, `mulaw` defaults to 8000 Hz. ## Generation ID Every successful response carries an `x-generation-id` header identifying the generation. Keep it — it is what you quote when [reporting a problem](/api-reference/issues/create-issue) with the audio. ### REST vs WebSocket comparison | Feature | REST API | Streaming Out API | | ----------------------------- | --------------------------------------------- | ----------------------------------------------------------------------------- | | **Delivery** | Streaming HTTP response (chunked audio bytes) | Chunked audio delivered incrementally as base64-encoded JSON messages | | **Formats** | `mp3`, `opus`, `mulaw` | `wav` (default), `mp3`, `opus`, `mulaw`, `s16le` | | **Text streamed in** | No | No — use [Streaming In and Streaming Out](/api-reference/websocket/streaming) | | **Default format** | `mp3` | `wav` | | **Default mulaw sample rate** | 8000 Hz | 8000 Hz | | **Best for** | Simple integrations, file generation | Real-time playback, low-latency applications | # Update Dubbing Job Source: https://docs.deepdub.ai/api-reference/update-dubbing-job /managed-dub.openapi.json put /dubbing/job/{request_id} Update mutable fields of an existing job. Only the fields included in the request body are changed. # Get daily concurrent usage Source: https://docs.deepdub.ai/api-reference/usage/get-daily-concurrent-usage get /usage/concurrent Get daily concurrent-capacity usage for the authenticated customer. Returns usage rows from the billing system, broken down by date, model, and region. # Get usage aggregated by model Source: https://docs.deepdub.ai/api-reference/usage/get-usage-aggregated-by-model get /usage/concurrent/by-model Get concurrent-capacity usage rolled up per model and region over a date range. Returns aggregated totals instead of daily rows. # Delete a voice prompt Source: https://docs.deepdub.ai/api-reference/voice/delete-a-voice-prompt delete /voice/{prompt_id} Permanently delete a voice prompt by its unique identifier. # Get a voice prompt by ID Source: https://docs.deepdub.ai/api-reference/voice/get-a-voice-prompt-by-id get /voice/{prompt_id} Retrieve a specific voice prompt by its unique identifier. # Get voice prompts Source: https://docs.deepdub.ai/api-reference/voice/get-voice-prompts get /voice Retrieve all private voice prompts associated with the authenticated customer's account. Results can be limited using the `limit` query parameter. # Update a voice prompt Source: https://docs.deepdub.ai/api-reference/voice/update-a-voice-prompt put /voice Update metadata for an existing voice prompt. Only non-empty fields are updated — omit fields you don't want to change. # Upload a voice sample Source: https://docs.deepdub.ai/api-reference/voice/upload-a-voice-sample post /voice Upload a voice sample to create a new voice prompt in the voice bank. The audio data must be base64-encoded. Maximum file size is 20 MB. # Streaming Out API Source: https://docs.deepdub.ai/api-reference/websocket/overview Send one complete text and stream the generated audio back over a WebSocket ## Overview You send one complete text, and the audio **streams out** to you as it is generated — delivered incrementally as base64-encoded chunks, so playback can begin before the full generation finishes. This endpoint takes the same generation parameters as the [REST TTS endpoint](/api-reference/tts/generate-and-stream-tts-audio), but delivers audio as a stream of chunks rather than a single response. If your text is not complete up front — for example it arrives token-by-token from an LLM — you want text to stream **in** as well. Use the [Streaming In and Streaming Out API](/api-reference/websocket/streaming) (`/ws`) for that. This page documents the single-request `text-to-speech` protocol (`/open`). ## Connection Connect to the `/open` endpoint with your API key: | Region | URL | | ------ | -------------------------------- | | US | `wss://wsapi.deepdub.ai/open` | | EU | `wss://wsapi.eu.deepdub.ai/open` | Authentication is handled during the WebSocket handshake via the `x-api-key` header or query parameter. ## Request format Send a JSON message on the WebSocket connection: The type of generation request. Model ID to use for generation (e.g., `dd-etts-3.0`). Text to convert to speech. Language locale code (e.g., `en-US`, `fr-FR`). ID of the voice prompt to use. Supports `asset:` prefix for built-in voices. Optional client-provided ID. Auto-generated if not provided. Target audio duration in seconds. Playback speed multiplier, between 0 and 2. Mutually exclusive with `targetDuration` — sending both is rejected. Voice variation level (0.0-1.0). Random seed for deterministic generation. Applies to `dd-etts-1.1` only — newer models do not use it, and setting it has no effect on their output. Generation temperature (0.0-1.0). Output sample rate in Hz. One of `8000`, `16000`, `22050`, `24000`, `32000`, `36000`, `44100`, or `48000`; any other value is rejected. Internal generation is 48 kHz, resampled to the requested rate. Defaults to 8000 Hz for `mulaw` if not specified. Output audio format: `wav` (default), `mp3`, `opus`, `mulaw`, or `s16le`. Enhance voice prompt characteristics. Enable super stretch mode for longer audio. Enable real-time priority processing. Apply audio cleanup processing. Automatically adjust audio gain levels. Accent blending parameters. See [AccentControl](#accent-control) below. ID of a performance reference prompt to guide delivery style. Target speaker gender, `male` or `female`. Used for language-specific handling such as Hebrew diacritics. Other values are ignored rather than rejected. Return the diacritized (menukad) form of the Hebrew `targetText` alongside the audio. When enabled, the first audio chunk carries a [`diacritized`](#audio-chunks) field. Whether Deepdub may record this request's text in its server-side logs. Set to `false` for confidential scripts to keep the text out of the logs. `outputDiacritized` is available on request rather than enabled for every account. Contact [support@deepdub.ai](mailto:support@deepdub.ai) to have it turned on before integrating against it. ### Example request ```json theme={null} { "action": "text-to-speech", "model": "dd-etts-3.0", "targetText": "Welcome to Deepdub's real-time text to speech API.", "locale": "en-US", "voicePromptId": "bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773", "format": "wav", "sampleRate": 16000 } ``` ## Response format ### Audio chunks Audio is delivered as a series of JSON messages. Each chunk contains a portion of the audio data: Sequential chunk index starting from 0. The generation ID for this request. Use this to correlate chunks with requests when running multiple generations on the same connection. Base64-encoded audio data for this chunk. `true` when this is the final chunk of the generation. Diacritized (menukad) Hebrew text for the request. Sent only when `outputDiacritized` was enabled, and only on the first audio chunk — later chunks omit the field entirely. ### Example response stream **Initial acknowledgement:** ```json theme={null} { "data": "", "generationId": "4da9902b-9141-4fb7-9efb-d616ce266ed9", "isFinished": false } ``` **Audio chunks:** ```json theme={null} { "index": 0, "generationId": "4da9902b-9141-4fb7-9efb-d616ce266ed9", "data": "//uQxAAAAAANIAAAAAExBTUUzLjEwMFVVVVVVVVVV...", "isFinished": false } ``` ```json theme={null} { "index": 1, "generationId": "4da9902b-9141-4fb7-9efb-d616ce266ed9", "data": "HAAYABgAGAAgACAA...", "isFinished": false } ``` **Final chunk:** ```json theme={null} { "index": 2, "generationId": "4da9902b-9141-4fb7-9efb-d616ce266ed9", "data": "AAAAAAAAAA==", "isFinished": true } ``` **First chunk when `outputDiacritized` is enabled:** ```json theme={null} { "index": 0, "generationId": "4da9902b-9141-4fb7-9efb-d616ce266ed9", "data": "//uQxAAAAAANIAAAAAExBTUUzLjEwMFVVVVVV...", "isFinished": false, "diacritized": "שָׁלוֹם עוֹלָם" } ``` ## Error responses When an error occurs, the WebSocket sends a JSON error message: Human-readable error description. Error category. One of: `RateLimit`, `MaxExceeded`, `InsufficientCredits`, `InvalidInput`. Present on requests rejected up front; errors raised later, once generation is already under way, carry only `error` and `generationId`. The generation ID, if available. ```json theme={null} { "error": "Rate limit exceeded", "errorType": "RateLimit", "generationId": "4da9902b-9141-4fb7-9efb-d616ce266ed9" } ``` | Error type | Description | | --------------------- | ------------------------------------------------------- | | `RateLimit` | Too many concurrent requests. Reduce request frequency. | | `MaxExceeded` | Maximum generation minutes reached for your plan. | | `InsufficientCredits` | Account has insufficient credits. Top up your balance. | | `InvalidInput` | Invalid request parameters. Check your request body. | The free trial key is additionally capped at 10 generations per IP per day. Once that is used up, requests fail with `errorType: "RateLimit"` and a message of the form `Free tier quota exceeded (used: 10). Please get an API key to continue.` Get your own API key to lift the cap. ## Accent control Blend accents between two locales using the `accentControl` object: ```json theme={null} { "accentControl": { "accentBaseLocale": "en-US", "accentLocale": "fr-FR", "accentRatio": 0.75 } } ``` | Field | Type | Description | | ------------------ | ------ | ----------------------------------------------------- | | `accentBaseLocale` | string | Base accent locale (e.g., `en-US`) | | `accentLocale` | string | Target accent to blend (e.g., `fr-FR`) | | `accentRatio` | number | Blend ratio from 0.0 (base only) to 1.0 (target only) | ## Supported output formats Audio chunks are delivered as base64-encoded data in JSON messages. | Format | Description | | ------- | ------------------------------------------------- | | `wav` | Uncompressed PCM in a WAV container (**default**) | | `mp3` | MP3 | | `opus` | Opus | | `mulaw` | 8-bit μ-law, common in telephony | | `s16le` | Raw signed 16-bit little-endian PCM, no container | ## Sample rates Valid values are `8000`, `16000`, `22050`, `24000`, `32000`, `36000`, `44100`, and `48000` Hz. The internal generation runs at 48 kHz and is resampled to the requested rate. If no sample rate is specified, `mulaw` defaults to 8000 Hz. ### REST vs Streaming Out | Feature | REST API | Streaming Out API | | ----------------------------- | --------------------------------------------- | ----------------------------------------------------------------------------- | | **Delivery** | Streaming HTTP response (chunked audio bytes) | Chunked audio delivered incrementally as base64-encoded JSON messages | | **Formats** | `mp3`, `opus`, `mulaw` | `wav` (default), `mp3`, `opus`, `mulaw`, `s16le` | | **Text streamed in** | No | No — use [Streaming In and Streaming Out](/api-reference/websocket/streaming) | | **Default format** | `mp3` | `wav` | | **Default mulaw sample rate** | 8000 Hz | 8000 Hz | | **Best for** | Simple integrations, file generation | Real-time playback, low-latency applications | ## Code examples ### Python ```python theme={null} import asyncio from deepdub import DeepdubClient client = DeepdubClient(api_key="dd-00000000000000000000000065c9cbfe") async def streaming_tts(): audio_data = bytearray() async with client.async_connect() as conn: async for chunk in conn.async_tts( text="Hello, this is streamed text input.", voice_prompt_id="bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773", model="dd-etts-3.0", locale="en-US", format="wav", sample_rate=16000, ): audio_data.extend(chunk) print(f"Received chunk: {len(chunk)} bytes") with open("output.wav", "wb") as f: f.write(audio_data) print(f"Total audio: {len(audio_data)} bytes") asyncio.run(streaming_tts()) ``` ### JavaScript ```javascript theme={null} const { DeepdubClient } = require("@deepdub/node"); const fs = require("fs"); async function streamingTts() { const deepdub = new DeepdubClient("dd-00000000000000000000000065c9cbfe"); await deepdub.connect(); const chunks = []; for await (const chunk of deepdub.streamTts("Hello, this is streamed text input.", { locale: "en-US", voicePromptId: "bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773", model: "dd-etts-3.0", format: "wav", sampleRate: 16000, })) { chunks.push(chunk); console.log(`Received chunk: ${chunk.length} bytes`); } const audio = Buffer.concat(chunks); fs.writeFileSync("output.wav", audio); console.log(`Total audio: ${audio.length} bytes`); deepdub.disconnect(); } streamingTts(); ``` # Streaming In and Streaming Out API Source: https://docs.deepdub.ai/api-reference/websocket/streaming Stream text in as it is produced and stream the generated audio back, over one persistent WebSocket session ## Overview Text **streams in** and audio **streams out**, over one persistent session. This is built for real-time, agent-driven applications where the text does not exist all at once — for example, arriving token-by-token from an LLM. You configure the voice once, then push text as it becomes available and receive audio chunks back as they are generated. The server buffers and batches incoming text intelligently before dispatching it to TTS workers, so you can send anything from a full sentence to a single token per message without worrying about segmentation. This is a distinct endpoint (`/ws`) from the [Streaming Out API](/api-reference/websocket/overview) (`/open`). If your text is already complete when you call, that endpoint is simpler — use this one only when text has to stream in. ## Connection Connect to the `/ws` endpoint with your API key: | Region | URL | | ------ | ---------------------------- | | US | `wss://wss.deepdub.ai/ws` | | EU | `wss://wss.eu.deepdub.ai/ws` | Authentication is handled during the WebSocket handshake via the `x-api-key` header or an `x-api-key` query parameter. On a successful connection, the server sends a welcome message before you send anything: ```json theme={null} { "action": "status", "connectionId": "b6f1c2d0-1e2f-4a3b-8c9d-0e1f2a3b4c5d", "message": "connected" } ``` ## Protocol flow A typical session follows this sequence: Open the WebSocket with your `x-api-key` and wait for the `status` / `connected` welcome message. Send a `stream-config` message with at least a `model`. This must be sent before any text. Send one or more `stream-text` messages as text becomes available. Audio chunks stream back as JSON messages. Send `end-stream` to flush and finish the current turn, or `cancel` to abort in-flight generation and clear buffers. ## Message format Every message is a JSON object with an `action` field. Request payloads can be placed in either a `data` or a `config` object — both are accepted interchangeably. ```json theme={null} { "action": "...", "data": { ... } } ``` Most client messages do not receive a direct reply; audio and status messages arrive asynchronously. ## Client actions | Action | Description | | --------------- | -------------------------------------------------------------------- | | `stream-config` | Configure the session. **Required before streaming text.** | | `stream-text` | Send text to be synthesized. | | `end-stream` | Signal the end of the text stream and flush remaining buffered text. | | `cancel` | Cancel the current generation and clear all buffers. | | `ping` | Keepalive. Server responds with `{"action": "pong"}`. | ## `stream-config` Sets the session-level TTS configuration. Must be sent before any `stream-text` message, otherwise the server responds with an error: ```json theme={null} { "action": "error", "message": "please send 'stream-config' action with model before streaming text" } ``` It can be sent again later in the same session to reconfigure it. The new settings apply to text flushed after that point; generations already in flight keep the configuration they started with. A successful `stream-config` produces no reply — only failures are reported. ### Example ```json theme={null} { "action": "stream-config", "data": { "model": "dd-etts-3.0", "voicePromptId": "59da0f21-63de-4aef-9ade-e5cabfe639ab", "locale": "en-US", "format": "s16le", "sampleRate": 16000, "realtime": true } } ``` ### Parameters Model ID to use for generation (e.g., `dd-etts-3.0`). Only required field. UUID of the voice/emotion prompt that controls which voice the engine uses. Can be changed mid-stream via [inline config tags](#inline-config-changes). Base64-encoded WAV to clone a voice inline, as an alternative to `voicePromptId`. Expects mono 48 kHz audio, up to 1 MB. Language/locale code for the generated speech (e.g., `en-US`, `he-IL`, `es-MX`). Determines the language model and pronunciation rules. Can be changed mid-stream via [inline config tags](#inline-config-changes). Output audio format: `wav` (default), `mp3`, `opus`, `mulaw`, or `s16le`. Unlike the [Streaming Out API](/api-reference/websocket/overview), this endpoint does not reject unknown values — an unrecognized format silently produces `wav`, so check your spelling. For low-latency playback, `s16le` and `mulaw` avoid the container and decoder overhead of `wav` and `mp3`. Output sample rate in Hz — `8000`, `16000`, `22050`, `24000`, `32000`, `36000`, `44100`, or `48000`. Internal generation is 48 kHz and is resampled to the requested rate. `mulaw` defaults to 8000 Hz; other formats default to 48 kHz. Prioritize low latency. `true` gives the session real-time processing priority; `false` uses standard priority. Apply audio cleanup processing. Target speaker gender, `male` or `female`. Used for language-specific handling such as Hebrew diacritics. Return the diacritized (menukad) form of the synthesized Hebrew text alongside the audio. When enabled, the first audio chunk of each generation carries a [`diacritized`](#audio-chunks) field. Only an explicit `true` opts in — sending `false` behaves the same as omitting the field. Whether Deepdub may record this session's text in its server-side logs. Set to `false` for confidential scripts to keep the text out of the logs. Applies to the whole connection. Maximum time in milliseconds to wait for more text before flushing the first buffered segment to a worker. Lower values reduce time-to-first-audio. Interpret a set of emoji in the streamed text as inline locale and emotion changes instead of speaking them. See [Emoji tags](#emoji-tags). `outputDiacritized` is available on request rather than enabled for every account. Contact [support@deepdub.ai](mailto:support@deepdub.ai) to have it turned on before integrating against it. ## `stream-text` Sends text to be synthesized. The server buffers and batches text before dispatching it to workers, so messages can be as small as a single token. ```json theme={null} { "action": "stream-text", "data": { "text": "Hello, how are you today?" } } ``` A shorthand form with a top-level `text` field is also accepted: ```json theme={null} { "action": "stream-text", "text": "Hello, how are you today?" } ``` The text (or text fragment) to synthesize. Optional opaque blob forwarded to the TTS worker alongside the next flush. It is not echoed back on any response frame — use `generationId` to correlate audio with a generation. ### Inline config changes `locale` and `voicePromptId` can be changed mid-stream by embedding a `` XML tag in the text: ```json theme={null} { "action": "stream-text", "data": { "text": "Hello! ¡Hola, mundo!" } } ``` ```json theme={null} { "action": "stream-text", "data": { "text": "Cheerful tone. Now with a different voice." } } ``` Multiple attributes can be set in one tag: ```xml theme={null} ``` **Behavior:** * Text before the tag is synthesized with the previous config. * The new config applies to all text after the tag. * Tags may be split across multiple `stream-text` messages — incomplete tags are buffered until the closing `>` arrives. * Closing tags (``) are stripped and ignored. ### Emotion tags If your voice has emotion variants configured, you can switch between them mid-stream by writing the emotion name in square brackets: ```json theme={null} { "action": "stream-text", "data": { "text": "That's wonderful! [excited] I can't believe it! [default] Anyway, back to business." } } ``` **Behavior:** * `[default]` and `[normal]` revert to the **base voice** — the one set by `stream-config`, or by the most recent `` tag. * Any other name is looked up in the emotion set configured for your model and base voice. On a match, all text after the tag uses that emotion's voice. Switching emotions does not change the base voice, so `[default]` always returns to it. * Names are matched case-insensitively, and surrounding whitespace is ignored (`[Excited]` and `[ excited ]` both work). * An unrecognized tag is left in place and spoken as literal text, so bracketed text that isn't an emotion passes through unchanged. * Tags may be split across `stream-text` messages; an incomplete fragment is buffered until the `]` arrives. Emotion sets are provisioned per voice. Contact [support@deepdub.ai](mailto:support@deepdub.ai) to find out which emotions are available for your voices. ### Emoji tags When the session is configured with `acceptEmojis: true`, a small set of emoji are also interpreted as inline configuration rather than spoken: | Emoji | Effect | | ---------------------------------------------- | --------------------------------------------- | | A flag, e.g. 🇺🇸 🇪🇸 🇩🇪 | Switch locale to that country's locale | | A globe, e.g. 🌎 | Switch to the associated regional locale | | An emoji bound to one of your voice's emotions | Switch to that emotion's voice | | 😐 | Revert to the base voice, same as `[default]` | Any other emoji is left in the text. With `acceptEmojis: false` (the default), all emoji are treated as ordinary text. ## `end-stream` `end-stream` marks the end of a **turn** — a logical unit of text that should be spoken as one continuous utterance (for example, a single assistant reply). Because the server buffers and batches incoming text to optimize prosody and latency, it does not know when you have finished sending text unless you tell it. `end-stream` does two things: 1. **Flushes the tail as the final segment.** Any remaining buffered text is synthesized and marked as the last segment of the turn. This gives the tail clean sentence-final intonation instead of the "more is coming" prosody used for mid-stream segments. 2. **Closes the turn**, so the final audio chunk is tagged with `"isFinal": true`. This is the signal your client should wait for to know the whole turn is done. ```json theme={null} { "action": "end-stream" } ``` Even without `end-stream`, buffered text is eventually flushed and synthesized on its own — the tail is not lost. What you lose by omitting it is the explicit turn-end signal: no chunk is tagged `"isFinal": true`, so your client can't tell a turn boundary from an ordinary segment boundary, and the tail is spoken as a continuation rather than a clean ending. Always send `end-stream` to close a turn cleanly. ### Multi-turn conversations The connection is persistent and can be reused across many turns. After you receive the `isFinal` chunk for one turn, simply start sending `stream-text` again for the next turn — the session configuration from your initial `stream-config` is retained. Send `end-stream` again to close each subsequent turn. This makes a single connection ideal for a back-and-forth conversation: keep it open for the whole session, configure once, and bracket each assistant reply with `stream-text` messages followed by an `end-stream`. ## `cancel` and `ping` Aborts in-flight generation and clears all buffers. Useful for barge-in — when the user interrupts, cancel the current turn so you stop generating audio the user will never hear. The server emits a synthetic finish message with `"isCancelled": true`, after which you can start a new turn. Keepalive. The server replies with `{"action": "pong"}`. Use it to hold a conversation connection open between turns. ```json theme={null} { "action": "cancel" } ``` ## Server responses | Action | Description | | --------- | ----------------------------------------------------------- | | `status` | Connection status updates (e.g., `connected`, `cancelled`). | | `pong` | Response to `ping`. | | `error` | Error with a `message` field. | | *(audio)* | Audio chunk from a TTS worker (see below). | ### Audio chunks Audio arrives as JSON messages. Decode the base64 `data` field and append the bytes in order. Identifier of the generation this chunk belongs to. Use it to correlate chunks across concurrent generations. Sequential chunk index within a generation, starting from 0. Base64-encoded audio data for this chunk. `true` when this is the final chunk of a generation. `true` on the final chunk of the whole turn. Only produced after you send `end-stream`. `true` when the generation was aborted via `cancel`. Diacritized (menukad) Hebrew text for this generation's segment. Sent only when the session opted in via `outputDiacritized`, and only on the first chunk of each generation — later chunks omit the field entirely. In a multi-segment turn, each segment's first chunk carries its own diacritized text. If a generation produces no audio at all, the field rides its `isFinished` chunk instead. **In-progress chunk:** ```json theme={null} { "generationId": "4da9902b-9141-4fb7-9efb-d616ce266ed9", "index": 0, "isFinished": false, "data": "//uQxAAAAAANIAAAAAExBTUUzLjEwMFVVVVVV..." } ``` **First chunk when `outputDiacritized` is enabled:** ```json theme={null} { "generationId": "4da9902b-9141-4fb7-9efb-d616ce266ed9", "index": 0, "isFinished": false, "data": "//uQxAAAAAANIAAAAAExBTUUzLjEwMFVVVVVV...", "diacritized": "שָׁלוֹם עוֹלָם" } ``` **Final chunk of a generation:** ```json theme={null} { "generationId": "4da9902b-9141-4fb7-9efb-d616ce266ed9", "index": 5, "isFinished": true } ``` **Final chunk of the whole turn** (after `end-stream`): ```json theme={null} { "generationId": "4da9902b-9141-4fb7-9efb-d616ce266ed9", "index": 5, "isFinished": true, "isFinal": true } ``` **Cancelled generation:** ```json theme={null} { "generationId": "4da9902b-9141-4fb7-9efb-d616ce266ed9", "isFinished": true, "isCancelled": true } ``` ### Errors ```json theme={null} { "action": "error", "message": "please send 'stream-config' action with model before streaming text", "time": 1753512960 } ``` Session-level errors always carry `action: "error"`, a human-readable `message`, and a Unix-seconds `time`. Common messages: | Message | Cause | | --------------------------------------------------------------------- | ------------------------------------------------------- | | `Invalid message format` | The message was not valid JSON | | `model is required in configuration` | `stream-config` omitted `model` | | `queue route not found for model: ` | `model` is not a model your account can use | | `please send 'stream-config' action with model before streaming text` | `stream-text` arrived before the session was configured | | `voicePrompt exceeds maximum size of 1MB` | Inline `voicePrompt` is too large | | `voicePrompt must be base64-encoded: ...` | Inline `voicePrompt` is not valid base64 | | `voicePrompt must be mono (1 channel), got N channels` | Inline `voicePrompt` is not mono | | `voicePrompt must be 48000Hz sample rate, got NHz` | Inline `voicePrompt` is not 48 kHz | | `voicePrompt is not a WAV file: ...` | Inline `voicePrompt` is not a RIFF/WAVE file | The session stays open after a session-level error, so you can correct the configuration and continue. Worker-level errors are reported per generation and are shaped differently — no `action`, but a `generationId` identifying the generation that failed: ```json theme={null} { "generationId": "4da9902b-9141-4fb7-9efb-d616ce266ed9", "error": "error message" } ``` A generation that fails this way sends no further chunks. Other generations in the turn are unaffected. ## Code example The following raw Python client connects, configures a session, streams text token-by-token, and collects audio chunks until the turn is final. ```python theme={null} import asyncio import base64 import json import os import websockets WS_URL = "wss://wss.deepdub.ai/ws" API_KEY = os.environ["DEEPDUB_API_KEY"] TOKENS = ["Hello, ", "this ", "is ", "a ", "streaming ", "test."] async def main(): async with websockets.connect( WS_URL, additional_headers={"x-api-key": API_KEY}, ping_timeout=90, ) as ws: # 1) Welcome message hello = json.loads(await ws.recv()) print(hello) # {"action": "status", "connectionId": "...", "message": "connected"} # 2) Configure the session await ws.send(json.dumps({ "action": "stream-config", "data": { "model": "dd-etts-3.0", "voicePromptId": "59da0f21-63de-4aef-9ade-e5cabfe639ab", "locale": "en-US", "format": "s16le", "sampleRate": 16000, "realtime": True, }, })) # 3) Stream text as it becomes available for token in TOKENS: await ws.send(json.dumps({"action": "stream-text", "data": {"text": token}})) # 4) Signal end of the turn await ws.send(json.dumps({"action": "end-stream"})) # 5) Collect audio chunks pcm = bytearray() while True: msg = json.loads(await ws.recv()) if msg.get("action") == "error": raise RuntimeError(msg["message"]) if msg.get("data"): pcm += base64.b64decode(msg["data"]) if msg.get("isFinished") and msg.get("isFinal"): break # whole turn is done with open("output.pcm", "wb") as f: f.write(pcm) print(f"Received {len(pcm)} bytes of audio") asyncio.run(main()) ``` ## Streaming from an LLM The streaming API is built for exactly this pattern: forward tokens from a streaming LLM response into `stream-text` as they arrive, then send `end-stream` once the LLM finishes, and play the audio chunks as they come back. Because sending text and receiving audio happen concurrently, run a **producer** (LLM → `stream-text`) and a **consumer** (audio chunks → playback) at the same time. The key rule is: **call `end-stream` as soon as the LLM has produced its last token**, and treat the `isFinal` chunk as "the assistant has finished speaking." ```python theme={null} import asyncio import base64 import json import os import websockets from openai import AsyncOpenAI WS_URL = "wss://wss.deepdub.ai/ws" API_KEY = os.environ["DEEPDUB_API_KEY"] openai = AsyncOpenAI() async def produce(ws, prompt): """Forward LLM tokens into the TTS stream, then close the turn.""" stream = await openai.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": prompt}], stream=True, ) async for chunk in stream: token = chunk.choices[0].delta.content if token: # Send each token as it arrives; the server batches them for you. await ws.send(json.dumps({"action": "stream-text", "text": token})) # The LLM is done — flush the buffer and close the turn. await ws.send(json.dumps({"action": "end-stream"})) async def consume(ws): """Receive audio chunks until the turn is final.""" pcm = bytearray() while True: msg = json.loads(await ws.recv()) if msg.get("action") == "error": raise RuntimeError(msg["message"]) if msg.get("data"): audio = base64.b64decode(msg["data"]) pcm += audio # play(audio) # feed to your audio output here for real-time playback if msg.get("isFinished") and msg.get("isFinal"): break # the assistant has finished speaking this turn return bytes(pcm) async def main(): async with websockets.connect( WS_URL, additional_headers={"x-api-key": API_KEY}, ping_timeout=90, ) as ws: # Wait for the welcome message, then configure the session once. await ws.recv() await ws.send(json.dumps({ "action": "stream-config", "data": { "model": "dd-etts-3.0", "voicePromptId": "59da0f21-63de-4aef-9ade-e5cabfe639ab", "locale": "en-US", "format": "s16le", "sampleRate": 16000, "realtime": True, }, })) # Run the LLM producer and the audio consumer concurrently. producer = asyncio.create_task(produce(ws, "Tell me a fun fact about the ocean.")) audio = await consume(ws) await producer print(f"Received {len(audio)} bytes of audio") asyncio.run(main()) ``` For a multi-turn conversation, keep the connection open and repeat the producer/consumer cycle per turn: stream the next LLM reply with `stream-text`, send `end-stream`, and wait for the next `isFinal`. If the user interrupts mid-reply, send `cancel` before starting the next turn. # Authentication Source: https://docs.deepdub.ai/authentication Authenticate your requests to the Deepdub API ## API keys All Deepdub API requests require authentication via an API key passed in the `x-api-key` header. ```bash theme={null} curl -X POST https://restapi.deepdub.ai/api/v1/tts \ -H "Content-Type: application/json" \ -H "x-api-key: YOUR_API_KEY" \ -d '{ ... }' ``` ### Key format API keys follow the format `dd-{random_characters}{checksum}`, where: * The key always starts with the `dd-` prefix * The last 8 characters are a checksum for validation ### Obtaining an API key Contact [support@deepdub.ai](mailto:support@deepdub.ai) or visit your Deepdub dashboard to generate an API key. ### Security best practices Never expose your API key in client-side code or public repositories. Always store keys in environment variables or a secrets manager. * Store API keys in a secrets manager (e.g., AWS Secrets Manager, HashiCorp Vault, GCP Secret Manager) * Rotate keys periodically * Use different keys for development and production environments ## Rate limits API requests are subject to rate limiting based on your subscription plan: | Limit type | Default | | -------------------------------- | ------- | | Concurrent requests per customer | 5 | | Concurrent requests per IP | 3 | Rate limit errors return a `429` status code or a WebSocket error with `errorType: "RateLimit"`. ## Credits TTS generation consumes credits based on the duration of generated audio. If your account runs out of credits, requests will return an `InsufficientCredits` error. Check your credit balance in the Deepdub dashboard, or contact support to upgrade your plan. ## Regions Deepdub operates in two regions. Every API is available in both — the EU host is the US host with `.eu` inserted before `.deepdub.ai`. | API | US (default) | EU | | -------------------------------------------------------------------- | ----------------------------------- | -------------------------------------- | | REST | `https://restapi.deepdub.ai/api/v1` | `https://restapi.eu.deepdub.ai/api/v1` | | [Streaming Out](/api-reference/websocket/overview) | `wss://wsapi.deepdub.ai/open` | `wss://wsapi.eu.deepdub.ai/open` | | [Streaming In and Streaming Out](/api-reference/websocket/streaming) | `wss://wss.deepdub.ai/ws` | `wss://wss.eu.deepdub.ai/ws` | Your API key is bound to a region: use the host for the region your account was provisioned in. Pick the region closest to your users — crossing the Atlantic adds latency that matters most for real-time streaming. # Claude Code (AGENTS.md) Source: https://docs.deepdub.ai/claude-code Integrate Deepdub API knowledge into Claude Code's AI agent ## Overview [Claude Code](https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/overview) is Anthropic's agentic coding tool. It automatically reads `AGENTS.md` files from your repository to understand project context. Adding a Deepdub `AGENTS.md` gives Claude Code full knowledge of the API, SDKs, and conventions. ## Installation Download the `AGENTS.md` file to the root of your repository: ```bash theme={null} curl -o AGENTS.md \ https://raw.githubusercontent.com/deepdub-ai/deepdub-api/main/docs/skills/AGENTS.md ``` View, copy, or download the full Claude Code AGENTS.md file. ## AGENTS.md file The `AGENTS.md` file contains the full Deepdub API reference that Claude Code reads automatically. It includes: * **Project structure** — overview of the monorepo layout * **Base URLs** — US, EU, and WebSocket endpoints * **Authentication** — API key format and free trial key * **REST endpoints** — TTS generation, voice management, gender detection * **Python SDK** — `pip install deepdub` with sync/async examples * **JavaScript SDK** — `npm install @deepdub/node` with WebSocket and HTTP examples * **Error codes** — 400, 401, 402, 403, 404, 429, 500 with meanings * **Voice presets** — common preset IDs for quick testing * **Coding conventions** — project-specific patterns and best practices ### Key sections **TTS request body:** ```json theme={null} { "model": "dd-etts-3.0", "targetText": "Hello world", "locale": "en-US", "voicePromptId": "bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773" } ``` **Python SDK:** ```python theme={null} from deepdub import DeepdubClient client = DeepdubClient(api_key="YOUR_KEY") audio = client.tts( text="Hello", voice_prompt_id="...", model="dd-etts-3.0", locale="en-US", ) ``` **JavaScript SDK:** ```javascript theme={null} const { DeepdubClient } = require("@deepdub/node"); const deepdub = new DeepdubClient("YOUR_KEY"); await deepdub.connect(); const buffer = await deepdub.generateToBuffer("Hello", { locale: "en-US", voicePromptId: "...", model: "dd-etts-3.0", }); ``` **Error responses:** All errors return JSON: `success: false` with a `message` field. | Code | Meaning | | ---- | ----------------------------- | | 400 | Invalid or missing parameters | | 401 | Invalid or missing API key | | 402 | Insufficient credits | | 429 | Rate limit exceeded | | 500 | Internal server error | ## How it works Claude Code reads `AGENTS.md` files automatically when it starts a session in your repository. Subdirectory `AGENTS.md` files are scoped to that directory. The Deepdub reference can go at the project root or in a specific service directory. | Placement | Scope | | -------------------------------- | --------------------------------- | | `/AGENTS.md` | Entire repo — always available | | `/projects/my-service/AGENTS.md` | Only when working in that service | ## Example prompts After adding the `AGENTS.md`, try asking Claude Code: * *"Add a TTS endpoint that generates speech from user input"* * *"Integrate the Deepdub Python SDK to classify speaker gender"* * *"Create a streaming TTS service using the JavaScript SDK"* * *"Upload a custom voice sample via the REST API"* ## Combining with Cursor You can use both the Cursor Skill and Claude Code AGENTS.md in the same repo. They don't conflict — each tool reads its own format: | Tool | File | Location | | ----------- | ------------------------------------- | ------------------------- | | Cursor | `.cursor/skills/deepdub-api/SKILL.md` | Project `.cursor/skills/` | | Claude Code | `AGENTS.md` | Repository root | # Cursor Skill Source: https://docs.deepdub.ai/cursor-skill Integrate Deepdub API knowledge into Cursor IDE's AI agent ## Overview The Deepdub Cursor Skill gives the AI agent in [Cursor](https://cursor.com) full knowledge of the Deepdub API, SDKs, voice presets, and coding conventions. When activated, the agent can write Deepdub integration code, generate TTS calls, manage voices, and follow Deepdub best practices — without needing to look up the docs. ## Installation Create the skill directory in your project and download the `SKILL.md` file: ```bash theme={null} mkdir -p .cursor/skills/deepdub-api curl -o .cursor/skills/deepdub-api/SKILL.md \ https://raw.githubusercontent.com/deepdub-ai/deepdub-api/main/docs/skills/SKILL.md ``` View, copy, or download the full Cursor Skill file. ## Skill file The `SKILL.md` file contains the full Deepdub API reference in a format Cursor understands. It includes: * **Base URLs** — US, EU, and WebSocket endpoints * **Authentication** — API key format and free trial key * **REST endpoints** — TTS generation, voice management, gender detection * **Python SDK** — `pip install deepdub` with sync/async examples * **JavaScript SDK** — `npm install @deepdub/node` with WebSocket and HTTP examples * **Error codes** — 400, 401, 402, 403, 404, 429, 500 with meanings * **Voice presets** — common preset IDs for quick testing * **Rate limits** — concurrent request limits ### Frontmatter ```yaml theme={null} name: deepdub-api description: >- Deepdub Text-to-Speech API reference for building TTS integrations. Use when the user writes code that calls the Deepdub API, uses the Deepdub Python/JS SDK, generates speech, manages voices, classifies gender, or mentions deepdub, TTS, text-to-speech, or voice cloning. ``` ### Key sections **TTS request body:** ```json theme={null} { "model": "dd-etts-3.0", "targetText": "Hello world", "locale": "en-US", "voicePromptId": "bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773" } ``` **Python SDK:** ```python theme={null} from deepdub import DeepdubClient client = DeepdubClient(api_key="YOUR_KEY") audio = client.tts( text="Hello", voice_prompt_id="...", model="dd-etts-3.0", locale="en-US", ) ``` **JavaScript SDK:** ```javascript theme={null} const { DeepdubClient } = require("@deepdub/node"); const deepdub = new DeepdubClient("YOUR_KEY"); await deepdub.connect(); const buffer = await deepdub.generateToBuffer("Hello", { locale: "en-US", voicePromptId: "...", model: "dd-etts-3.0", }); ``` ## How it works Once installed, the Cursor agent automatically activates this skill when you: * Write code that imports `deepdub` or `@deepdub/node` * Ask about TTS, text-to-speech, or voice cloning * Reference voice prompts or the Deepdub API * Work on gender detection audio classification The agent will use the correct endpoints, default model (`dd-etts-3.0`), proper authentication headers, and real voice preset IDs from the skill. ## Example prompts After installing the skill, try asking Cursor: * *"Generate TTS audio with the Storyteller voice and save to file"* * *"Add accent blending to this TTS call — mix French into English at 30%"* * *"Classify the gender of speaker in this audio file using the Deepdub API"* * *"Stream TTS audio in real-time using the JavaScript SDK"* # Introduction Source: https://docs.deepdub.ai/introduction Generate expressive, high-quality speech with Deepdub's Text-to-Speech API ## Welcome to Deepdub Deepdub provides a powerful Text-to-Speech API for generating natural, expressive speech with voice cloning, accent control, and real-time streaming. Whether you're building voiceover pipelines, conversational agents, or content localization workflows, Deepdub delivers studio-quality audio at scale. ### Key capabilities Generate speech from text using state-of-the-art models with fine-grained control over tempo, variance, and duration. Clone any voice from a short audio sample. Upload voice prompts or pass a base64-encoded audio reference for instant cloning. Blend accents between locales with precise ratio control — generate an American English speaker with a French accent, or any combination. Stream audio in real-time over HTTP or WebSocket connections for low-latency applications. ## Try it now Use the free trial API key to generate speech instantly — no sign-up required: ``` dd-00000000000000000000000065c9cbfe ``` ```python theme={null} from deepdub import DeepdubClient client = DeepdubClient(api_key="dd-00000000000000000000000065c9cbfe") audio = client.tts( text="Welcome to Deepdub!", voice_prompt_id="bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773", model="dd-etts-3.0", locale="en-US", ) with open("output.mp3", "wb") as f: f.write(audio) ``` ```javascript theme={null} const { DeepdubClient } = require("@deepdub/node"); async function main() { const deepdub = new DeepdubClient("dd-00000000000000000000000065c9cbfe"); await deepdub.connect(); await deepdub.generateToFile("./output.wav", "Welcome to Deepdub!", { locale: "en-US", voicePromptId: "bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773", model: "dd-etts-3.0", }); } main(); ``` ## API access Deepdub offers two integration methods: | Method | Endpoint | Use case | | ----------------- | ----------------------------------- | ----------------------------------------------- | | **REST API** | `https://restapi.deepdub.ai/api/v1` | Streaming audio generation, voice management | | **WebSocket API** | `wss://wsapi.deepdub.ai` | Real-time streaming with chunked audio delivery | ## Quick links Get up and running with your first TTS generation in minutes. Browse ready-to-use voice presets across 6 languages. Install the Python SDK and start generating speech. Install the Node.js SDK with real-time streaming support. Explore the full REST API with interactive playground. Send one complete text and stream the audio back over a WebSocket. Stream text in as an LLM produces it and stream the audio back. # Subtitle-Only Flow Source: https://docs.deepdub.ai/managed-dub/subtitle-only-flow Generate subtitles and transcripts from a source video without producing a dubbed audio track The Managed Dub API can produce text deliverables — subtitles, captions, and transcripts — **without** rendering or delivering a dubbed video. This is the *subtitle-only* (deliverables-only) flow. It uses the same `POST /dubbing/job` endpoint as a full dubbing job (see the **Submit Dubbing Job** endpoint in the API Reference). The difference is entirely in the request body: you request one or more **additional products** and omit the dubbed-video **export path**. ## How it works A dubbing job always runs intake, transcription, and translation against the source video. What the job *delivers* depends on the request: | You provide | The job delivers | | ------------------------- | ------------------------------------------------------------- | | `exportPath` only | A dubbed video at that location | | `additionalProducts` only | The requested subtitle/transcript files — **no dubbed video** | | Both | A dubbed video **and** the requested deliverables | For a subtitle-only job you therefore set `additionalProducts` and leave `exportPath` unset. A request must contain at least one of `exportPath` or `additionalProducts`. A job with neither has nothing to deliver and will fail at delivery time. ## Available subtitle & transcript products Each entry in `additionalProducts` has a `product` type and an `assetPath` — the `s3://` destination where Deepdub writes the finished file. | `product` | Deliverable | | --------------- | ------------------------------------------ | | `SRT` | SubRip subtitles | | `WEBVTT` | WebVTT subtitles | | `SDH` | Subtitles for the Deaf and Hard of Hearing | | `ITT_SUBTITLES` | iTunes Timed Text (iTT) subtitles | | `TRANSCRIPT` | Plain-text transcript | ## Request Provide the source video and the target locale, then list the subtitle/transcript products you want under `additionalProducts`. Do **not** set `exportPath`. ```json theme={null} { "sourceLocale": "en-US", "targetLocale": "es-ES", "sourceVideoPath": "s3://your-bucket/input/episode-01.mp4", "additionalProducts": [ { "product": "SRT", "assetPath": "s3://your-bucket/output/episode-01.es-ES.srt" }, { "product": "TRANSCRIPT", "assetPath": "s3://your-bucket/output/episode-01.es-ES.txt" } ] } ``` ```bash cURL theme={null} curl -X POST https://dubbing.deepdub.app/dubbing/job \ -H "Content-Type: application/json" \ -H "x-api-key: YOUR_API_KEY" \ -d '{ "sourceLocale": "en-US", "targetLocale": "es-ES", "sourceVideoPath": "s3://your-bucket/input/episode-01.mp4", "additionalProducts": [ { "product": "SRT", "assetPath": "s3://your-bucket/output/episode-01.es-ES.srt" }, { "product": "TRANSCRIPT", "assetPath": "s3://your-bucket/output/episode-01.es-ES.txt" } ] }' ``` ```python Python theme={null} import requests response = requests.post( "https://dubbing.deepdub.app/dubbing/job", headers={ "Content-Type": "application/json", "x-api-key": "YOUR_API_KEY", }, json={ "sourceLocale": "en-US", "targetLocale": "es-ES", "sourceVideoPath": "s3://your-bucket/input/episode-01.mp4", "additionalProducts": [ {"product": "SRT", "assetPath": "s3://your-bucket/output/episode-01.es-ES.srt"}, {"product": "TRANSCRIPT", "assetPath": "s3://your-bucket/output/episode-01.es-ES.txt"}, ], }, ) response.raise_for_status() request_id = response.json()["requestId"] print(f"Submitted subtitle-only job: {request_id}") ``` ### Response ```json theme={null} { "requestId": "a1b2c3d4-5678-90ab-cdef-1234567890ab" } ``` Use the `requestId` to track the job. ## Tracking status Poll the `GET /dubbing/job/{requestId}` endpoint (see **Get Dubbing Job** in the API Reference) to follow the job. Each entry in the returned `additionalProducts` reports its delivery `state`: * `PENDING_DELIVERY` — awaiting delivery * `DELIVERED` — written to the requested `assetPath` Once every requested product reports `DELIVERED`, the subtitle files are available at their `assetPath` locations. ## Notes * All S3 paths (`sourceVideoPath`, `assetPath`) must start with `s3://` and be in a bucket Deepdub is authorized to read from and write to. * Authenticate every request with the `x-api-key` header. Contact your account manager to obtain an API key. * The base URL is `https://dubbing.deepdub.app`. Unlike the TTS APIs, Managed Dub is served from a single region — there is no EU host. # Quickstart Source: https://docs.deepdub.ai/quickstart Generate your first TTS audio in under 5 minutes ## Prerequisites All you need is: * **Free trial API key:** `dd-00000000000000000000000065c9cbfe` * **A voice preset** — see [Voice Presets](/voice-presets) for the full list ## Generate speech with the Python SDK ```bash theme={null} pip install deepdub ``` ```python theme={null} from deepdub import DeepdubClient client = DeepdubClient(api_key="dd-00000000000000000000000065c9cbfe") audio = client.tts( text="Welcome to Deepdub! This is your first generated audio.", voice_prompt_id="50a537cf-1ec8-4714-b07e-c589ab76be4b", # Promo / Commercials model="dd-etts-3.0", locale="en-US", ) with open("output.mp3", "wb") as f: f.write(audio) print(f"Generated {len(audio)} bytes of audio") ``` ## Generate speech with the JavaScript SDK ```bash theme={null} npm install --save @deepdub/node ``` ```javascript theme={null} const { DeepdubClient } = require("@deepdub/node"); async function main() { const deepdub = new DeepdubClient("dd-00000000000000000000000065c9cbfe"); await deepdub.connect(); const buffer = await deepdub.generateToBuffer( "Welcome to Deepdub! This is your first generated audio.", { locale: "en-US", voicePromptId: "50a537cf-1ec8-4714-b07e-c589ab76be4b", // Promo / Commercials model: "dd-etts-3.0", } ); require("fs").writeFileSync("output.wav", buffer); console.log(`Generated ${buffer.length} bytes of audio`); } main(); ``` ## Generate speech with cURL ```bash theme={null} curl -X POST https://restapi.deepdub.ai/api/v1/tts \ -H "Content-Type: application/json" \ -H "x-api-key: dd-00000000000000000000000065c9cbfe" \ -d '{ "model": "dd-etts-3.0", "targetText": "Welcome to Deepdub! This is your first generated audio.", "locale": "en-US", "voicePromptId": "50a537cf-1ec8-4714-b07e-c589ab76be4b" }' \ --output output.mp3 ``` ## Advanced generation options Fine-tune your audio with accent blending and tuned parameters: ```python theme={null} audio = client.tts( text="This demonstrates accent blending with tuned parameters.", voice_prompt_id="bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773", # Storyteller model="dd-etts-3.0", locale="en-US", temperature=0.7, tempo=1.1, variance=0.6, sample_rate=44100, format="mp3", prompt_boost=True, accent_base_locale="en-US", accent_locale="fr-FR", accent_ratio=0.3, ) with open("advanced_output.mp3", "wb") as f: f.write(audio) ``` ## Real-time streaming For low-latency applications, stream audio chunks as they're generated: ```python theme={null} import asyncio from deepdub import DeepdubClient client = DeepdubClient(api_key="dd-00000000000000000000000065c9cbfe") async def stream(): audio_data = bytearray() async with client.async_connect() as conn: async for chunk in conn.async_tts( text="Streaming audio in real time!", voice_prompt_id="bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773", model="dd-etts-3.0", locale="en-US", format="wav", sample_rate=16000, ): audio_data.extend(chunk) print(f"Received chunk: {len(chunk)} bytes") with open("streamed.wav", "wb") as f: f.write(audio_data) print(f"Total audio: {len(audio_data)} bytes") asyncio.run(stream()) ``` ```javascript theme={null} const { DeepdubClient } = require("@deepdub/node"); async function main() { const deepdub = new DeepdubClient("dd-00000000000000000000000065c9cbfe"); await deepdub.connect(); const buffer = await deepdub.generateToBuffer("Streaming audio in real time!", { locale: "en-US", voicePromptId: "50a537cf-1ec8-4714-b07e-c589ab76be4b", model: "dd-etts-3.0", onChunk: (chunk) => { console.log(`Received ${chunk.length} bytes`); }, }); require("fs").writeFileSync("streamed.wav", buffer); } main(); ``` ## Next steps Browse curated voice presets from our library of thousands of voices. Full Python SDK reference with all methods and async streaming. Full Node.js SDK reference with streaming chunks and concurrent generation. Explore all TTS parameters and response formats. # Python SDK Source: https://docs.deepdub.ai/sdk Install and use the Deepdub Python SDK for text-to-speech, voice management, and real-time streaming ## Installation ```bash theme={null} pip install deepdub ``` **Requirements:** Python 3.9+ **Dependencies:** `requests`, `websockets`, `click`, `audiosample` ## Initialization ```python theme={null} from deepdub import DeepdubClient # Option 1: Pass API key directly client = DeepdubClient(api_key="dd-your-api-key") # Option 2: Use DEEPDUB_API_KEY environment variable # export DEEPDUB_API_KEY=dd-your-api-key client = DeepdubClient() ``` ### Constructor parameters Your Deepdub API key. Falls back to `DEEPDUB_API_KEY` environment variable if not provided. Base URL for the REST API. Falls back to `DEEPDUB_BASE_URL` environment variable. Base URL for the WebSocket API. Falls back to `DEEPDUB_BASE_WEBSOCKET_URL` environment variable. Base URL for the WebSocket streaming API. Falls back to `DEEPDUB_BASE_WEBSOCKET_STREAMING_URL` environment variable. Use EU region endpoints (`restapi.eu.deepdub.ai`, `wsapi.eu.deepdub.ai`). Falls back to `DD_EU` environment variable (`"1"` to enable). ### Region endpoints | Region | REST API | WebSocket API | | ---------------- | -------------------------------------- | -------------------------------- | | **US (default)** | `https://restapi.deepdub.ai/api/v1` | `wss://wsapi.deepdub.ai/open` | | **EU** | `https://restapi.eu.deepdub.ai/api/v1` | `wss://wsapi.eu.deepdub.ai/open` | *** ## Text-to-Speech ### `tts()` — Synchronous generation Generate speech and receive the complete audio as bytes. ```python theme={null} audio_data = client.tts( text="Hello, welcome to Deepdub!", voice_prompt_id="your-voice-id", model="dd-etts-2.5", locale="en-US" ) with open("output.mp3", "wb") as f: f.write(audio_data) ``` **Returns:** `bytes` — binary audio data in the specified format. #### Parameters Text to convert to speech. Voice prompt ID to use. Either this or `voice_reference` must be provided. Audio reference for instant voice cloning. Accepts a file `Path`, raw `bytes`, or a base64-encoded `string`. Either this or `voice_prompt_id` must be provided. Model ID. Available models: `dd-etts-3.0`, `dd-etts-2.5`. Language locale code (e.g., `en-US`, `fr-FR`). Audio output format. REST API supports: `mp3`, `opus`, `mulaw`. WebSocket additionally supports: `wav` (default), `s16le`. Generation temperature (0.0–1.0). Higher values produce more varied output. Voice variation level (0.0–1.0). Target audio duration in seconds. Mutually exclusive with `tempo`. Playback speed multiplier. Mutually exclusive with `duration`. Random seed for deterministic generation. Applies to `dd-etts-1.1` only — newer models do not use it, and setting it has no effect on their output. Enhance voice prompt characteristics. Output sample rate in Hz. Supported: `8000`, `16000`, `22050`, `24000`, `44100`, `48000`. The REST and WebSocket APIs additionally accept `32000` and `36000`, which the Python SDK rejects — call those APIs directly if you need them. Base accent locale (e.g., `en-US`). Must be provided together with `accent_locale` and `accent_ratio`. Target accent locale (e.g., `fr-FR`). Must be provided together with `accent_base_locale` and `accent_ratio`. Accent blend ratio (0.0–1.0). Must be provided together with `accent_base_locale` and `accent_locale`. ### Full example with all parameters ```python theme={null} audio_data = client.tts( text="This demonstrates all available TTS parameters.", voice_prompt_id="your-voice-id", model="dd-etts-2.5", locale="en-US", format="mp3", temperature=0.7, variance=0.6, tempo=1.1, prompt_boost=True, sample_rate=44100, accent_base_locale="en-US", accent_locale="fr-FR", accent_ratio=0.3, ) with open("output.mp3", "wb") as f: f.write(audio_data) ``` ### Voice cloning from audio reference ```python theme={null} from pathlib import Path audio_data = client.tts( text="Cloning a voice from an audio sample.", voice_reference=Path("reference_audio.mp3"), model="dd-etts-2.5", locale="en-US", ) with open("cloned_output.mp3", "wb") as f: f.write(audio_data) ``` *** ## Async / WebSocket TTS ### `async_tts()` — Streaming generation Stream audio chunks over WebSocket for low-latency playback. Must be used within an `async_connect()` context. ```python theme={null} import asyncio from deepdub import DeepdubClient client = DeepdubClient(api_key="dd-your-api-key") async def stream_audio(): audio_data = bytearray() async with client.async_connect() as conn: async for chunk in conn.async_tts( text="Streaming audio in real time!", voice_prompt_id="bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773", model="dd-etts-3.0", locale="en-US", format="wav", sample_rate=16000, ): audio_data.extend(chunk) print(f"Received chunk: {len(chunk)} bytes") with open("streamed.wav", "wb") as f: f.write(audio_data) print(f"Total audio: {len(audio_data)} bytes") asyncio.run(stream_audio()) ``` **Yields:** `bytes` — audio chunks as they are generated. #### Parameters Same as `tts()`, plus: Optional UUID for request tracking. Auto-generated if not provided. Target gender for the output voice. Print debug information about sent/received messages. ### Multiple concurrent generations The WebSocket connection supports multiplexing — run multiple TTS requests on the same connection: ```python theme={null} import asyncio from deepdub import DeepdubClient client = DeepdubClient(api_key="dd-your-api-key") async def generate_multiple(): async with client.async_connect() as conn: async def generate_one(text, filename): audio = bytearray() async for chunk in conn.async_tts( text=text, voice_prompt_id="bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773", model="dd-etts-3.0", locale="en-US", format="wav", sample_rate=16000, ): audio.extend(chunk) with open(filename, "wb") as f: f.write(audio) await asyncio.gather( generate_one("First sentence.", "out1.wav"), generate_one("Second sentence.", "out2.wav"), generate_one("Third sentence.", "out3.wav"), ) asyncio.run(generate_multiple()) ``` *** ## Streaming Input For real-time text streaming (sending text incrementally), use `async_stream_connect()`: ```python theme={null} import asyncio from deepdub import DeepdubClient client = DeepdubClient(api_key="dd-your-api-key") async def streaming_input(): async with client.async_stream_connect( model="dd-etts-3.0", locale="en-US", voice_prompt_id="your-voice-id", format="wav", sample_rate=16000, ) as conn: await conn.async_stream_text("Hello, ") await conn.async_stream_text("this is streamed ") await conn.async_stream_text("text input.") await conn.async_stream_end() audio_data = bytearray() while True: audio = await conn.async_stream_recv_audio() if audio is None: break audio_data.extend(audio) print(f"Received chunk: {len(audio)} bytes") print(f"Total audio: {len(audio_data)} bytes") asyncio.run(streaming_input()) ``` *** ## Gender Classification Classify the gender of a speaker from an audio sample: ```python theme={null} import asyncio from pathlib import Path from deepdub import DeepdubClient client = DeepdubClient(api_key="dd-your-api-key") async def classify(): async with client.async_connect() as conn: result = await conn.gender_classify( audio_data=Path("speaker_sample.wav"), sample_rate=16000, timeout=5.0, ) print(result) asyncio.run(classify()) ``` Audio data as raw bytes, base64-encoded string, or file Path. Automatically trimmed to 1 second. Sample rate of the input audio. Timeout in seconds for the WebSocket response. Optional UUID for request tracking. *** ## Voice Management ### `list_voices()` — List all voice prompts ```python theme={null} voices = client.list_voices() for voice in voices.get("voicePrompts", []): print(f"{voice['id']}: {voice.get('name', voice.get('title', 'Untitled'))}") ``` **Returns:** `dict` with a `voicePrompts` key containing a list of voice prompt objects. ### `add_voice()` — Upload a voice sample ```python theme={null} from pathlib import Path response = client.add_voice( data=Path("voice_sample.wav"), name="Professional Narrator", gender="female", locale="en-US", publish=False, speaking_style="Neutral", age=30, ) print(f"Created voice: {response}") ``` **Returns:** `dict` with the created voice prompt information. #### Parameters Audio data — a file `Path`, raw `bytes`, or base64-encoded `string`. Display name for the voice prompt. Speaker gender: `"male"` or `"female"`. Language locale code (e.g., `en-US`). Whether to make the voice publicly available. Speaking style descriptor. Age of the speaker. *** ## CLI Reference The SDK includes a command-line interface: ```bash theme={null} # List available voices deepdub list-voices # Upload a new voice deepdub add-voice \ --file path/to/audio.mp3 \ --name "My Voice" \ --gender male \ --locale en-US # Generate text-to-speech deepdub tts \ --text "Hello from the CLI!" \ --voice-prompt-id your-voice-id # Set API key via flag or environment deepdub --api-key dd-your-key tts --text "Hello!" export DEEPDUB_API_KEY=dd-your-key ``` *** ## Environment Variables | Variable | Description | Default | | -------------------------------------- | ---------------------------------- | ----------------------------------- | | `DEEPDUB_API_KEY` | API key for authentication | — | | `DEEPDUB_BASE_URL` | REST API base URL | `https://restapi.deepdub.ai/api/v1` | | `DEEPDUB_BASE_WEBSOCKET_URL` | WebSocket API base URL | `wss://wsapi.deepdub.ai/open` | | `DEEPDUB_BASE_WEBSOCKET_STREAMING_URL` | Streaming WebSocket base URL | `wss://wss.deepdub.ai/ws` | | `DD_EU` | Use EU endpoints (`"1"` to enable) | `"0"` | *** ## Error Handling ```python theme={null} from deepdub import DeepdubClient import requests client = DeepdubClient(api_key="dd-your-api-key") try: audio = client.tts( text="Hello!", voice_prompt_id="your-voice-id", ) except requests.exceptions.HTTPError as e: if e.response.status_code == 401: print("Invalid API key") elif e.response.status_code == 400: print("Invalid request parameters") else: print(f"API error: {e}") except ValueError as e: print(f"Validation error: {e}") ``` For async operations, WebSocket errors are raised as `Exception` with the error message from the server: ```python theme={null} try: async with client.async_connect() as conn: async for chunk in conn.async_tts(text="Hello!", voice_prompt_id="id"): pass except Exception as e: error_msg = str(e) # Possible errors: "Rate limit exceeded", "Insufficient credits", etc. print(f"WebSocket error: {error_msg}") ``` *** ## Available Models | Model ID | Description | | ------------- | --------------------------------- | | `dd-etts-3.0` | Latest model with best quality | | `dd-etts-2.5` | Stable production model (default) | # JavaScript SDK Source: https://docs.deepdub.ai/sdk-javascript Install and use the Deepdub Node.js SDK for text-to-speech generation with streaming support ## Installation ```bash theme={null} npm install --save @deepdub/node # or yarn add @deepdub/node ``` **Requirements:** Node.js 18+ ## Initialization ```javascript theme={null} const { DeepdubClient } = require("@deepdub/node"); // Default: WebSocket protocol (supports streaming) const deepdub = new DeepdubClient("dd-your-api-key"); // HTTP protocol (supports voiceReference and sampleRate with all formats) const deepdub = new DeepdubClient("dd-your-api-key", { protocol: "http" }); ``` Your Deepdub API key. Must start with `dd-`. Transport protocol: `"websocket"` for real-time streaming, or `"http"` for REST API. ### Protocol comparison | Feature | WebSocket | HTTP | | ---------------------------- | --------- | ----------- | | Streaming chunks (`onChunk`) | Yes | No | | `sampleRate` option | mp3 only | All formats | | `voiceReference` option | No | Yes | | Concurrent generations | Yes | Yes | Use **WebSocket** (default) for real-time streaming and low-latency playback. Use **HTTP** when you need `voiceReference` for instant voice cloning or `sampleRate` with non-mp3 formats. *** ## Connection For WebSocket protocol, you must call `connect()` before generating audio: ```javascript theme={null} const deepdub = new DeepdubClient("dd-your-api-key"); await deepdub.connect(); ``` For HTTP protocol, no connection step is needed. *** ## Generate to buffer Generate audio and receive a `Buffer` of WAV data: ```javascript theme={null} const buffer = await deepdub.generateToBuffer("Hello, welcome to Deepdub!", { locale: "en-US", voicePromptId: "your-voice-id", }); console.log(`Generated ${buffer.length} bytes of audio`); ``` **Returns:** `Promise` — WAV audio data. ## Generate to file Generate audio and save directly to a file: ```javascript theme={null} await deepdub.generateToFile("./output.wav", "Hello, welcome to Deepdub!", { locale: "en-US", voicePromptId: "your-voice-id", }); ``` **Returns:** `Promise` *** ## Generation parameters Both `generateToBuffer` and `generateToFile` accept these options: Language locale code (e.g., `en-US`, `fr-FR`, `he-IL`). Voice prompt ID to use for generation. Model ID. Available: `dd-etts-3.0`, `dd-etts-2.5`. Optional UUID for tracking. Auto-generated if not provided. Output format: `mp3`, `wav`, `opus`, or `mulaw`. Sample rate in Hz. WebSocket protocol only supports this with `mp3` format. Use HTTP protocol for other formats. Generation temperature (0.0–1.0). Voice variation level (0.0–1.0). Playback speed multiplier, between 0 and 2. Mutually exclusive with `targetDuration`. Target audio duration in seconds. Random seed for deterministic output. Applies to `dd-etts-1.1` only — newer models do not use it, and setting it has no effect on their output. Enhance voice prompt characteristics. Enable super stretch for longer audio. Enable real-time priority processing. Base64-encoded audio for instant voice cloning. **HTTP protocol only.** Accent blending: `{ accentBaseLocale, accentLocale, accentRatio }`. Callback receiving each audio chunk as a `Buffer`. **WebSocket protocol only.** When `true`, chunks passed to `onChunk` have WAV headers stripped (raw PCM). **WebSocket protocol only.** *** ## Streaming chunks Receive audio data incrementally for real-time playback: ```javascript theme={null} const buffer = await deepdub.generateToBuffer("Streaming audio in real-time!", { locale: "en-US", voicePromptId: "your-voice-id", model: "dd-etts-3.0", onChunk: (chunk) => { console.log(`Received ${chunk.length} bytes`); // Stream to audio player, network, etc. }, }); ``` ### Headerless chunks Strip WAV headers from each chunk for raw PCM data (useful for audio players): ```javascript theme={null} const buffer = await deepdub.generateToBuffer("Raw PCM streaming.", { locale: "en-US", voicePromptId: "your-voice-id", headerless: true, onChunk: (chunk) => { audioPlayer.write(chunk); // Raw PCM data, no WAV header }, }); ``` *** ## Concurrent generations Run multiple generations in parallel on the same WebSocket connection: ```javascript theme={null} const { DeepdubClient } = require("@deepdub/node"); async function main() { const deepdub = new DeepdubClient("dd-your-api-key"); await deepdub.connect(); const sentences = [ "First sentence to generate.", "Second sentence in parallel.", "Third sentence simultaneously.", ]; const results = await Promise.all( sentences.map((text, i) => deepdub.generateToFile(`./output_${i}.wav`, text, { locale: "en-US", voicePromptId: "your-voice-id", model: "dd-etts-3.0", }) ) ); console.log("All generations complete!"); } main(); ``` *** ## Full example ```javascript theme={null} const { DeepdubClient } = require("@deepdub/node"); async function main() { const deepdub = new DeepdubClient(process.env.DEEPDUB_API_KEY); await deepdub.connect(); // Generate with accent blending const buffer = await deepdub.generateToBuffer( "This text has a subtle French accent.", { locale: "en-US", voicePromptId: "your-voice-id", model: "dd-etts-3.0", temperature: 0.7, variance: 0.6, accentControl: { accentBaseLocale: "en-US", accentLocale: "fr-FR", accentRatio: 0.3, }, } ); require("fs").writeFileSync("./accented.wav", buffer); console.log(`Generated ${buffer.length} bytes`); } main(); ``` ## Using HTTP protocol For voice cloning from an audio reference: ```javascript theme={null} const { DeepdubClient } = require("@deepdub/node"); const fs = require("fs"); async function main() { const deepdub = new DeepdubClient(process.env.DEEPDUB_API_KEY, { protocol: "http", }); const audioRef = fs.readFileSync("./reference_voice.wav"); const voiceReference = audioRef.toString("base64"); const buffer = await deepdub.generateToBuffer("Cloning a voice from audio.", { locale: "en-US", voiceReference, model: "dd-etts-3.0", sampleRate: 44100, }); fs.writeFileSync("./cloned_output.wav", buffer); } main(); ``` *** ## Error handling ```javascript theme={null} try { const buffer = await deepdub.generateToBuffer("Hello!", { locale: "en-US", voicePromptId: "your-voice-id", }); } catch (error) { // WebSocket errors are emitted as strings from the server // e.g. "Rate limit exceeded", "Insufficient credits" console.error("Generation failed:", error); } ``` *** ## Environment variables | Variable | Description | | ----------------- | --------------------------- | | `DEEPDUB_API_KEY` | API key (use with `dotenv`) | ```javascript theme={null} require("dotenv").config(); const deepdub = new DeepdubClient(process.env.DEEPDUB_API_KEY); ``` # Voice Presets Source: https://docs.deepdub.ai/voice-presets Ready-to-use voice presets for trying Deepdub's TTS API ## Free trial Use the free trial API key to start generating speech instantly — no sign-up required: ```bash API Key theme={null} dd-00000000000000000000000065c9cbfe ``` ```bash Recommended Model theme={null} dd-etts-3.0 ``` The free trial key is rate-limited by IP address. For production use, [contact us](mailto:support@deepdub.ai) for a dedicated API key. *** ## Voice presets Below is a small sample of curated voice presets from our library of **thousands of voices**. Use the **Voice Prompt ID** with the `voicePromptId` parameter in your API calls. These presets are just a starting point. Deepdub offers thousands of voices across dozens of languages and speaking styles. [Contact us](mailto:support@deepdub.ai) to explore the full voice catalog. ### English (EN) | Preset | Gender | Speaker | Speaking Style | Voice Prompt ID | | -------------------------------- | ------ | ---------------- | ---------------------------------- | -------------------------------------- | | Call Center Agent (Empathetic) | M | Guillermo Castro | Romantic 2 - Sentimental | `060acdd4-cd61-4b78-a513-90e00afa8835` | | Call Center Agent (Neutral) | M | Guillermo Castro | Neutral 2 - Interested | `b2abb241-ac92-48bf-a890-fec03f43e209` | | Sales Agent (Energetic) | F | Heather Long | Joy 2 - Happy | `02215cf5-04af-46f3-a061-48a4c81989bf` | | Storyteller | M | Mario Lopez | Romantic 1 - Intimate Affectionate | `bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773` | | Promo / Commercials | M | Thomas Young | Narration 3 - Dramatic Host | `50a537cf-1ec8-4714-b07e-c589ab76be4b` | | Meditation Guide | F | Kathy Evans | Narration 1 - Story | `c0866c1c-0731-45b2-9c90-21be974513b4` | | Young Influencer (Playful, Warm) | F | Anne Reed | Joy 2 - Happy | `ee96fc26-e96b-41f0-807b-146f595a824d` | | Emotional Acting | F | Janice Watson | Sadness 2 - Muffled | `81f7a995-37ea-40d2-be99-4c5614c165ce` | ### Hebrew (HE) | Preset | Gender | Speaker | Speaking Style | Voice Prompt ID | | --------------------- | ------ | --------------- | ------------------- | ------------------------------------------------------------------- | | Sales Agent (Neutral) | F | Raquel Alvarez | Spontaneous Speech | `fb158c16-af06-4a90-abbe-3599c942dd66_prompt-V2-Spontaneous-Speech` | | Meditation Guide | F | Dalia Eisenberg | Conversational | `080d003b-7701-4a97-8723-62e6bed6cab9` | | Storyteller | M | Flavio Ribeiro | Narration 1 - Story | `f0c91054-ca6e-4ad6-831d-5fd7921ea944` | ### German (DE) | Preset | Gender | Speaker | Speaking Style | Voice Prompt ID | | ------------------- | ------ | --------------- | -------------------- | ---------------------------------------------------------------------------- | | Promo / Commercials | M | Ulisses Pereira | Commercial Soft Sell | `24c20fcb-b04c-46fc-82d6-0eabac3cc563_prompt-V2-Commercial-Soft-Sell` | | Emotional Acting | F | Lenka Dvorak | Testimonial Sad | `a8300346-7eea-40b6-9b6d-f439beac8d4e_prompt-V2-Testimonial-Sad` | | News Broadcaster | M | Joao Rocha | Host Dramatic | `f55c5d08-9583-461e-a588-86b7ae44b6c7_prompt-V2-Host-dramatic-regular-speed` | ### Spanish (ES) | Preset | Gender | Speaker | Speaking Style | Voice Prompt ID | | -------------------- | ------ | -------------- | -------------- | -------------------------------------------------------------------------- | | Companion | M | Pedro Ayala | Angry #2 | `99e250e9-5c12-45e6-adf2-d00f582cc275_angry-talk-text-contempt` | | Personal Cheerleader | F | Lucia Mejia | Breathy | `9a5a5f6f-b27a-46fa-b7f7-ec49ed06a6dd_breathy-catcher-joy` | | Promo / Commercials | M | Johnny Ross | Host Bubbly | `609b778b-805e-4484-93e3-bd9eb73beb55_prompt-V2-Host-bubbly-regular-speed` | | Emotional Acting | F | Isabel Huerta | Panic #2 | `5a7cf008-9e48-45ca-beb8-7636a4f6f944_panting-text-panic` | | Storyteller | F | Marina Rosales | Reading | `4202cbc4-5862-4af5-83f4-286ef487d593_reading-neutral` | | Classic "Bad Guy" | M | Gonzalo Vega | Scream | `8f2a83c5-d6f7-4ad1-9d48-513b8992cd11_scream-catcher-anger` | ### Hindi (HI) | Preset | Gender | Speaker | Speaking Style | Voice Prompt ID | | ------------------- | ------ | ------------- | ------------------------------ | ---------------------------------------------------------------------------- | | Storyteller | M | Anil Keshri | Testimonial Intimate | `81647c16-f959-4284-91f4-23ca8de24d36_prompt-V2-Testimonial-Intimate` | | Promo / Commercials | M | Sunil Rastogi | Host Dramatic | `0e992325-d431-42f9-82c9-36c304296dc7_prompt-V2-Host-dramatic-regular-speed` | | News Broadcaster | M | Vijay Bose | Newscaster Headlines | `731912b7-7e63-4de9-acf6-b16c4bdb0c9e_prompt-V2-Newscaster-Headlines` | | Sports Narrator | M | Anil Jay | Sports Commentator High Energy | `4b51aefb-4d72-4e2f-8a41-6d546b23eba1_sports-commentator-high-energy` | | Emotional Acting | F | Priya Agarwal | Sad #2 | `2904f343-a223-4097-ac1b-1c92b33b2758_cry-text-sad` | ### Portuguese - Brazil (PT-BR) | Preset | Gender | Speaker | Speaking Style | Voice Prompt ID | | ---------------- | ------ | ------------- | -------------------- | ----------------------------------------------------------- | | News Broadcaster | M | Julio Machado | Newscaster Headlines | `9758dbd6-8be9-4db4-98cd-401c1be819b2_newscaster-headlines` | *** ## Quick test Try any preset immediately with the free trial key: ```python theme={null} from deepdub import DeepdubClient client = DeepdubClient(api_key="dd-00000000000000000000000065c9cbfe") # Storyteller - English audio = client.tts( text="Once upon a time, in a land far far away, there lived a brave knight.", voice_prompt_id="bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773", model="dd-etts-3.0", locale="en-US", ) with open("storyteller.mp3", "wb") as f: f.write(audio) ``` ```javascript theme={null} const { DeepdubClient } = require("@deepdub/node"); async function main() { const deepdub = new DeepdubClient("dd-00000000000000000000000065c9cbfe"); await deepdub.connect(); // Storyteller - English await deepdub.generateToFile( "./storyteller.wav", "Once upon a time, in a land far far away, there lived a brave knight.", { locale: "en-US", voicePromptId: "bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773", model: "dd-etts-3.0", } ); } main(); ``` ```bash theme={null} curl -X POST https://restapi.deepdub.ai/api/v1/tts \ -H "Content-Type: application/json" \ -H "x-api-key: dd-00000000000000000000000065c9cbfe" \ -d '{ "model": "dd-etts-3.0", "targetText": "Once upon a time, in a land far far away, there lived a brave knight.", "locale": "en-US", "voicePromptId": "bd1b00bb-be1c-4679-8eaa-0fcbfd4ff773" }' \ --output storyteller.mp3 ``` ### More examples ```python theme={null} audio = client.tts( text="Take a deep breath in... and slowly let it out. Feel the tension leaving your body.", voice_prompt_id="c0866c1c-0731-45b2-9c90-21be974513b4", model="dd-etts-3.0", locale="en-US", ) ``` ```javascript theme={null} await deepdub.generateToFile("./meditation.wav", "Take a deep breath in... and slowly let it out. Feel the tension leaving your body.", { locale: "en-US", voicePromptId: "c0866c1c-0731-45b2-9c90-21be974513b4", model: "dd-etts-3.0" } ); ``` ```python theme={null} audio = client.tts( text="आज की ताज़ा ख़बरें: प्रधानमंत्री ने नई योजना की घोषणा की।", voice_prompt_id="731912b7-7e63-4de9-acf6-b16c4bdb0c9e_prompt-V2-Newscaster-Headlines", model="dd-etts-3.0", locale="hi-IN", ) ``` ```javascript theme={null} await deepdub.generateToFile("./news_hindi.wav", "आज की ताज़ा ख़बरें: प्रधानमंत्री ने नई योजना की घोषणा की।", { locale: "hi-IN", voicePromptId: "731912b7-7e63-4de9-acf6-b16c4bdb0c9e_prompt-V2-Newscaster-Headlines", model: "dd-etts-3.0" } ); ``` ```python theme={null} audio = client.tts( text="Érase una vez, en un reino muy lejano, vivía una princesa valiente.", voice_prompt_id="4202cbc4-5862-4af5-83f4-286ef487d593_reading-neutral", model="dd-etts-3.0", locale="es-ES", ) ``` ```javascript theme={null} await deepdub.generateToFile("./storyteller_es.wav", "Érase una vez, en un reino muy lejano, vivía una princesa valiente.", { locale: "es-ES", voicePromptId: "4202cbc4-5862-4af5-83f4-286ef487d593_reading-neutral", model: "dd-etts-3.0" } ); ``` ```python theme={null} audio = client.tts( text="Entdecken Sie jetzt unser neuestes Angebot — nur für kurze Zeit verfügbar!", voice_prompt_id="24c20fcb-b04c-46fc-82d6-0eabac3cc563_prompt-V2-Commercial-Soft-Sell", model="dd-etts-3.0", locale="de-DE", ) ``` ```javascript theme={null} await deepdub.generateToFile("./promo_de.wav", "Entdecken Sie jetzt unser neuestes Angebot — nur für kurze Zeit verfügbar!", { locale: "de-DE", voicePromptId: "24c20fcb-b04c-46fc-82d6-0eabac3cc563_prompt-V2-Commercial-Soft-Sell", model: "dd-etts-3.0" } ); ``` ```python theme={null} audio = client.tts( text="और गेंद जाती है... छक्का! क्या शानदार शॉट!", voice_prompt_id="4b51aefb-4d72-4e2f-8a41-6d546b23eba1_sports-commentator-high-energy", model="dd-etts-3.0", locale="hi-IN", ) ``` ```javascript theme={null} await deepdub.generateToFile("./sports_hi.wav", "और गेंद जाती है... छक्का! क्या शानदार शॉट!", { locale: "hi-IN", voicePromptId: "4b51aefb-4d72-4e2f-8a41-6d546b23eba1_sports-commentator-high-energy", model: "dd-etts-3.0" } ); ```