Module 22.3 closed video vendors with Veo. Module 22.4 is the voice and music catalog: who you buy when the product must speak or sing. ElevenLabs is the default generative-voice name in RFPs—TTS, cloning, conversational agents, and added STT. Capability lectures already live in Vol. 16: TTS, voice cloning, and the vendor deep-dive ElevenLabs (STT/TTS). This page is the procurement view: when ElevenLabs wins, when you split STT elsewhere, and how you wrap it behind a Vol. 18 SDK.
Siblings: Cartesia (latency-first Sonic TTS), PlayAI (PlayHT lineage), then music with Suno and Udio. Product shape: Vol. 21 voice assistants. Consent and misuse: Vol. 20 privacy / copyright.
Learning Objectives
By the end of this lesson, students should be able to:
- Place ElevenLabs in the voice stack: TTS-first platform with cloning, agents, and optional STT.
- Contrast ElevenLabs vs Cartesia vs PlayAI vs hyperscaler TTS on quality, latency, compliance, and catalog breadth.
- Read a qualitative Pricing / Strengths / Weaknesses table without treating list prices as architecture.
- Call TTS from Python and sketch wrapping it behind FastAPI / streaming (Vol. 18).
- Decide when to split STT (Deepgram / Whisper / hyperscaler) from ElevenLabs TTS.
- State consent, watermarking, and retention gates before cloning a real person.
ElevenLabs is a cloud generative-voice company. Its flagship is neural text-to-speech with instant and professional voice cloning, low-latency streaming audio, and conversational-agent tooling. Speech-to-text (Scribe-class) is a later product: useful for a single-vendor demo, not automatically your production ASR. Treat ElevenLabs as a speak vendor unless a bake-off on your audio shows STT matching Deepgram, Whisper, or a hyperscaler SLO.
Why ElevenLabs Is the Catalog Default
Teams pick ElevenLabs when brand voice quality and cloning matter more than enterprise speech-suite completeness. Azure / Google / Amazon remain the compliance-default for many regulated buyers. Cartesia competes on time-to-first-audio. PlayAI (PlayHT lineage) is the other generative-TTS RFP name. Do not confuse this ecosystem page with Vol. 16.2—there you learn how to call STT/TTS; here you learn which company to sign.
Pricing / Strengths / Weaknesses (Qualitative)
Character counts, plan tiers, and per-minute STT rates change. Pin current vendor sheets in procurement; never hard-code dollar figures from a lecture into a cost model.
| Dimension | ElevenLabs | Cartesia | PlayAI / PlayHT | Hyperscaler TTS (Azure / Google / Amazon) |
|---|---|---|---|---|
| Pricing posture | Typically character- or credit-based TTS + plan seats; STT often minute-based—confirm live sheet | Usage / plan for Sonic-class TTS; latency SKUs may differ | Character / plan TTS; cloning tiers | Metered characters (TTS) + hours (STT); enterprise commit discounts |
| Strengths | Voice quality and cloning reputation; multilingual TTS; agent + studio UX; streaming TTS; growing STT | Very low time-to-first-byte audio; real-time conversational fit | Generative TTS + dialog engines; familiar PlayHT APIs | DPA, regions, SSO, on-prem/container options, STT+TTS in one cloud bill |
| Weaknesses | Cloud-only; STT not automatically best-in-class; cloning ethics/legal load; price can surprise at narration scale | Narrower brand/studio catalog vs ElevenLabs; still cloud | STT not the reason to buy; smaller household name in some regions | Voice “personality” often flatter than generative specialists |
Product Surfaces Engineers Actually Buy
Voices + TTS API
- Stock + cloned voice IDs
- Batch MP3/WAV or stream
- Wrap behind FastAPI (Vol. 18)
Conversational agents
- STT → LLM → TTS loop
- Often hybrid: Deepgram in, ElevenLabs out
- Barge-in needs fast STT (Vol. 16.2 real-time)
STT (Scribe-class)
- File / near-real-time transcript
- Eval vs Whisper / Deepgram
- Do not assume one key wins both SLOs
Pick ElevenLabs when
- Brand voice / cloning is the differentiator
- You already have specialist STT
- Listening tests beat hyperscaler neural voices
- You can accept cloud + vendor DPA
Do not pick it as
- Your only ASR without a WER bake-off
- An on-prem speech appliance
- A music generator (that is Suno / Udio)
- A substitute for consent + watermark policy
Python: TTS Behind a Configured Client
Voice IDs and model IDs are SKUs—store them in config, not scattered literals. Stream when a user is waiting; write files for course narration. Official SDK patterns evolve; re-read current docs. This is the same client style as Vol. 16.2, now treated as a vendor you isolate behind your Vol. 18 SDK / FastAPI layer so you can swap Cartesia later.
Architecture: Hybrid Voice Loop
Vol. 21 voice assistants are STT → policy LLM → TTS. ElevenLabs often owns only the last hop. Time-to-first-audio includes STT + LLM TTFT + TTS TTFB. If Cartesia wins TTFB on your telephony test, keep ElevenLabs for batch narration and Cartesia for live calls—that is a normal split, not a failure.
| Hop | Typical vendor | Eval |
|---|---|---|
| STT | Deepgram / Whisper / hyperscaler | WER, keyword recall, streaming lag |
| Policy LLM | Vol. 22.1 frontier or Vol. 22.5 inference host | Task success, not MOS |
| TTS | ElevenLabs / Cartesia / PlayAI / hyperscaler | Listening test + TTFB + barge-in |
Related Lectures
| Lecture | Role |
|---|---|
| ElevenLabs (STT/TTS) | Capability + API patterns |
| TTS / voice cloning / real-time transcription | Speech stack |
| FastAPI / streaming / WebSockets | Product wrap |
| Voice assistants | Product category |
| Cartesia · PlayAI | TTS RFP peers |
“ElevenLabs is our speech stack.” It is usually the TTS hop. Second: cloning a celebrity voice is a demo, not a license. Third: Scribe STT automatically beats Deepgram because the TTS is good. Fourth: character pricing is cheap at IVR scale without measuring. Fifth: Vol. 19 MOS-style listening tests are optional if the dashboard looks pretty. Sixth: you can skip wrap-as-data on transcripts because ElevenLabs hosts the agent.
Knowledge Check
- Short Answer: Is ElevenLabs primarily a TTS or an STT company? Answer: TTS-first (cloning/agents); STT is secondary unless a bake-off proves otherwise.
- True/False: This Vol. 22 lecture replaces the Vol. 16.2 ElevenLabs capability page. Answer: False — 22.4 is vendor/procurement; 16.2 is capability/API.
- Multiple Choice: A hybrid voice loop often uses: (a) specialist STT + ElevenLabs TTS, (b) ElevenLabs STT only forever, (c) Suno for IVR prompts. Answer: (a).
- Short Answer: Name one qualitative weakness vs hyperscaler speech. Answer: Cloud-only / weaker enterprise suite / STT not automatic leader / cloning legal load (any valid).
- True/False: You should invent dollar-per-character prices from memory for cost models. Answer: False — confirm the live sheet.
- Multiple Choice: Consent for cloning is: (a) a biometric/policy gate, (b) optional for public figures, (c) handled by MOS score. Answer: (a).
- Short Answer: Which Vol. 18 lecture wraps vendor SDKs into your product API? Answer: FastAPI (or OpenAI SDK / streaming / WebSockets — any valid wrap).
- True/False: Cartesia is the usual alternative when time-to-first-audio dominates. Answer: True.
- Multiple Choice: Music generation belongs with: (a) Suno/Udio, (b) ElevenLabs Scribe, (c) Veo. Answer: (a).
- Short Answer: Name the Vol. 16.2 sibling lecture for PlayHT/PlayAI capability. Answer: PlayHT.
Key Takeaways
- ElevenLabs is the generative TTS/cloning default; STT is optional and must be eval’d.
- Price, voice IDs, and model IDs are SKUs—config + live sheets, not lecture folklore.
- Split STT/TTS vendors when SLOs disagree; wrap both behind Vol. 18.
- Cloning requires consent, retention limits, and Vol. 20 review.
- Next: Cartesia—latency-first Sonic TTS.
Lab: Same 8-sentence IVR script through ElevenLabs stock voice vs hyperscaler neural TTS (Azure/Google). Blind listening + note TTFB if streaming is available. Grade: config-driven voice ID, no committed keys, written rationale for hybrid STT+TTS, consent checklist for any clone attempt (even a teammate).
Do not clone public figures or scrape voices. If API quota is unavailable, use Vol. 16.2 recorded samples and still write the vendor-split architecture.
Recap: ElevenLabs is the speak-side specialist in the Vol. 22 voice catalog. Capability detail remains in Vol. 16.2; shipping wrap is Vol. 18. Next latency competitor: Cartesia.