ElevenLabs -- Voice AI Integration

---

name: elevenlabs description: > Build voice cloning, batch TTS, streaming pipelines, and multi-voice workflows with ElevenLabs. USE WHEN: User needs text-to-speech implementation, voice cloning, audio content production, or real-time voice streaming with ElevenLabs. DON'T USE WHEN: User needs general audio processing. This skill is specific to the ElevenLabs API. OUTPUTS: Voice profiles, TTS pipelines, streaming configs, batch processing scripts, cost optimization strategies, audio production workflows. version: 1.1.0 author: SpookyJuice tags: [elevenlabs, tts, voice, audio, streaming] price: 14 author_url: "https://www.shopclawmart.com" support: "brian@gorzelic.net" license: proprietary osps_version: "0.1" content_hash: "sha256:15b0f06554be8ac168397e606562cad0ea5a15170dcdacccc79b5706deebf58a"

#‍​‌‌​‌‌‌‌​‌‌‌​​‌‌​‌‌‌​​​​​‌‌‌​​‌‌​​‌‌‌​‌​​​‌‌​​​‌​​‌‌‌​‌​​‌‌‌​​‌‌​‌‌​‌​‌​​​‌​‌‌​‌​​‌‌‌​​​​​‌‌​​​​​​‌‌​‌‌‌​‌‌​​‌‌​​​‌‌‌​​​​‌‌​​​‌‌​​‌‌​​‌‌​‌‌​​​‌‌‍ ElevenLabs

Version: 1.1.0 Price: $14 Type: Skill

Description

Production-grade ElevenLabs integration patterns for voice cloning, batch audio generation, and real-time streaming. The API surface is deceptively simple but the production edge cases — chunking that avoids mid-sentence splits, voice consistency across long sessions, and streaming that doesn't stutter — require patterns the docs don't cover. This skill encodes those patterns so you get broadcast-quality output on the first pass.

Prerequisites

• ElevenLabs account with API access
• API key: ELEVENLABS_API_KEY
• Python 3.10+ or Node.js 18+ (SDK support)
• ffmpeg installed for audio processing: brew install ffmpeg

Setup

1. Copy SKILL.md into your OpenClaw skills directory
2. Set environment variables:

   export ELEVENLABS_API_KEY="your-api-key"
   

3. Reload OpenClaw

Commands

• "Clone a voice from [audio samples]"
• "Generate speech for [text/document]"
• "Set up streaming TTS for [use case]"
• "Build a batch TTS pipeline for [content type]"
• "Optimize my ElevenLabs costs"
• "Create a multi-voice project for [podcast/audiobook]"
• "What voice settings should I use for [use case]?"

Workflow

Voice Cloning Pipeline

1. Sample preparation — collect 1-5 minutes of clean audio per voice. Remove background noise, normalize volume, and ensure consistent microphone distance. Longer samples improve clone quality but have diminishing returns past 5 minutes.
2. Clone creation — use the Add Voice API with type: "clone". Upload samples as WAV or MP3. Name the voice descriptively (include use case and version: "narrator-v1").
3. Parameter tuning — adjust: stability (0.0-1.0, higher = more consistent but less expressive), similarity_boost (0.0-1.0, higher = more like the sample), and style (0.0-1.0, higher = more expressive). Start at stability=0.5, similarity=0.75, style=0.0 and adjust from there.
4. Quality validation — generate test phrases that cover: declarative sentences, questions, exclamations, numbers, proper nouns, and technical terms. Listen for: naturalness, pacing, pronunciation accuracy, and emotional range.
5. Voice profile documentation — save the optimal settings as a voice profile: voice ID, model ID, parameter settings, and notes on what the voice handles well vs. poorly.

Batch TTS Generation

1. Content preparation — split long-form text into chunks at natural sentence boundaries (never mid-sentence, never mid-word). Target 1000-2500 characters per chunk. Use SSML break tags for explicit pauses between sections.
2. Chunk sequencing — number each chunk and track: text content, voice assignment, model assignment, and output filename. This is your manifest for assembly.
3. API call management — implement request queue with: rate limit awareness (check X-RateLimit-* headers), exponential backoff on 429 responses, and parallel execution within rate limits (typically 2-5 concurrent for standard tiers).
4. Audio stitching — concatenate output files in order. Insert configurable silence gaps between sections (200-500ms between paragraphs, 500-1000ms between chapters). Use ffmpeg for concatenation with crossfade transitions if needed.
5. Quality check — automated: verify all chunks generated successfully, check for silence/noise anomalies in output files, verify total duration is within expected range. Manual: spot-check transitions between chunks for consistency.
6. Cost tracking — log characters consumed per generation. Compare against your tier quota. Implement caching for repeated content (same text + same voice + same settings = reuse the output).

Streaming Pipeline

1. WebSocket setup — connect to the streaming endpoint with your voice ID and model. Configure: chunk_length_schedule for latency/quality trade-off, output_format (mp3_44100_128 for quality, mp3_22050_32 for low bandwidth).
2. Text streaming — send text in chunks as they become available. Use flush: true on the final chunk to signal end of stream. The API generates audio incrementally as text arrives.
3. Audio buffering — implement a playback buffer (500ms-2s) to absorb network jitter. Start playback after the buffer is filled, not on the first audio chunk. This prevents stuttering.
4. Reconnection handling — WebSocket connections drop. Implement: automatic reconnection with exponential backoff, state restoration (which text was already sent), and seamless audio continuity.
5. Graceful degradation — if the API is slow or unavailable: queue text for batch processing later, show a loading state, or fall back to browser TTS as a degraded experience.

Output Format

🎙 ELEVENLABS — [IMPLEMENTATION TYPE]
Project: [Name]
Voice(s): [Voice IDs and names]
Model: [eleven_multilingual_v2 / eleven_turbo_v2]
Date: [YYYY-MM-DD]

═══ VOICE PROFILE ═══
| Parameter | Value | Notes |
|-----------|-------|-------|
| Voice ID | [id] | [name] |
| Stability | [0.0-1.0] | [why this value] |
| Similarity | [0.0-1.0] | [why] |
| Style | [0.0-1.0] | [why] |
| Model | [model_id] | [why this model] |

═══ PIPELINE ═══
[Processing flow: text → chunk → API → audio → stitch → output]

═══ COST ESTIMATE ═══
| Content | Characters | Cost | Tier |
|---------|-----------|------|------|
| [item] | [n] | $[x] | [tier] |
Total: [n] characters = $[x]

═══ IMPLEMENTATION ═══
[Code snippets with configuration]

Common Pitfalls

• Mid-sentence chunking — splitting text mid-sentence creates audible discontinuities. Always chunk at sentence boundaries, never at character count limits alone.
• Model mismatch — eleven_multilingual_v2 handles multiple languages but is slower. eleven_turbo_v2 is faster but English-only. Choose based on actual requirements, not defaults.
• Ignoring rate limits — hammering the API without rate limit awareness gets you throttled. Read the X-RateLimit-* response headers and implement proper queuing.
• No character counting before synthesis — generating audio for a 50,000-character document without estimating cost first leads to surprise bills. Always calculate before executing.
• WebSocket timeout — idle WebSocket connections are closed after ~20 seconds. Send keep-alive pings or reconnect on demand rather than maintaining a permanent connection.

Guardrails

• Never clones voices without authorization. Voice cloning requires explicit permission from the voice owner. This is both an ethical requirement and an ElevenLabs terms of service requirement.
• Cost estimation before execution. Every batch generation includes a character count and cost estimate BEFORE any API calls are made. No surprise bills.
• Respects rate limits. All implementations include proper rate limit handling. No brute-force retries that could get the account suspended.
• Caches aggressively. Same text + same voice + same settings = same output. Results are cached to avoid paying for identical regeneration.
• Quality checks are automated. Every pipeline includes verification steps: chunk completeness, audio duration validation, and silence detection.
• Never generates deceptive content. Does not assist in creating voice clones intended to impersonate real people for deception, fraud, or disinformation.
• Content rights verified before synthesis. Confirms that the user holds rights or a license to the source text before generating audio. No synthesizing copyrighted books, scripts, or articles without documented permission.

Support

Questions or issues with this skill? Contact brian@gorzelic.net Published by SpookyJuice — https://www.shopclawmart.com