Voice got good enough to sell this year. The demos are genuinely convincing now — a caller asks a messy question, the bot answers in a warm human voice, no robotic pauses. So the request lands on your desk: "Can we do a voice assistant for support?" And the first thing everyone reaches for is a brand shortlist — usually Hume versus ElevenLabs.
That's picking the paint before you've decided the building. The choice that actually shapes the project — cost, latency, how much control you keep, whether it survives an EU compliance review — is the architecture. Get that right and the platform is half-chosen already.
Two architectures, not two brands
Every voice bot is one of two shapes under the hood.
Cascading: STT → LLM → TTS
Three stages in a row. STT turns the caller's speech into text, an LLM decides what to say and which tools to call, and TTS speaks the answer. Each stage streams into the next, so the model starts thinking before you've finished talking and the voice starts before the model has finished its sentence.
This is what roughly 90% of production voice agents run on in 2026 — and for good reasons. You can swap any stage independently, log every step for debugging and compliance, and reuse the exact tool-calling and retrieval you already built for a chat assistant. The cost is latency that compounds: even with a 75ms TTS step, most cascaded agents land somewhere between 800ms and 2 seconds round-trip once every stage stacks up.
Speech-to-speech: audio in, audio out
One model takes the caller's audio and emits audio directly, with no text round-trip in the middle. Less is lost — tone, hesitation, emphasis survive — and the latency floor is far lower: sub-300ms end-to-end is realistic. The trade is control. There's no clean text seam to log, filter or hand to a separate reasoning model, and your app logic gets tightly coupled to one vendor's streaming API. Migrating later means rewriting the integration, not swapping a component.
Cascading is a pipeline you own stage by stage. Speech-to-speech is a black box that feels more human. Support usually wants the first; a few flows genuinely need the second.
Where ElevenLabs fits
ElevenLabs is the strongest option for the voice layer of a cascading agent, and its ElevenAgents platform wraps the whole pipeline if you don't want to assemble it yourself. Its Flash TTS model runs around 75ms with 32-language coverage, the voice quality and cloning are best-in-class, and turn-taking (knowing when the caller has actually finished) is handled well enough to feel natural.
For a support line, that maps to the work you actually have: transcribe accurately, call your APIs (order status, password reset, appointment booking), pull answers from the client's documents with RAG, speak the result, and hand off to a human cleanly. Telephony and WhatsApp are first-class. If the project is task-driven and multilingual, this is the default.
Where Hume fits
Hume's Empathic Voice Interface (EVI 3) is a speech-to-speech model built around one thing the others treat as an afterthought: emotional expression. It reads prosody — the rhythm and tone underneath the words — and responds in kind, at sub-300ms. Hume also sells a separate Expression Measurement API that scores emotional signals in voice.
That's not a better support bot; it's a different tool. It earns its place where how something is said carries the value: de-escalation on an angry call, wellbeing and care lines, coaching, any high-touch flow where a flat, task-completing voice would feel wrong. For "check my order status," the empathy is overhead. For "I've been on hold for an hour and I'm furious," it might be the whole point.
The EU catch nobody quotes you for
Emotion features are where a voice project quietly becomes a compliance project. Two things every EU client's reviewer will raise:
- The workplace ban is absolute. The EU AI Act (Article 5) prohibits inferring emotions from biometric data in the workplace and in education, with only narrow medical and safety exceptions. Using Hume to score your own support agents' emotions is off the table — that prohibition has been enforceable since February 2025, and the enforcement structures go fully live on 2 August 2026.
- Customer emotion isn't banned, but it isn't free either. Reading a caller's emotion isn't caught by the workplace ban, but it's still an emotion-recognition system with transparency duties (you must tell people), and voice is biometric data under the GDPR — so you need a lawful basis, clear disclosure, and usually a DPIA. Plus recording consent on the call itself.
None of this kills the Hume option. It just means the emotional-nuance flow carries paperwork the plain task bot doesn't — and that belongs in your estimate, not in a surprise the week before launch.
How we actually decide
- Is it task-driven? Orders, bookings, account actions, FAQs, transfer-to-human — cascading, ElevenLabs-style. This is most support work.
- Does emotional tone carry real value? Only then consider a speech-to-speech, empathy-first model like Hume — and confirm you're outside the workplace/education ban and have disclosure plus a DPIA lined up.
- Where does the audio go? Voice is personal data. Check processing location and sub-processors against the client's data-residency rules before you commit to a vendor — the same call we make on every model stack for EU clients.
- Design the human fallback first. Low confidence, repeated failure, or "give me a person" must route to a human with the transcript attached. Clients judge the whole system on that one moment.
Nine times out of ten the answer is a cascading pipeline with a great TTS voice and honest disclosure. The tenth is a genuinely emotional flow that justifies both the latency win and the compliance homework of speech-to-speech.
FAQ
Cascading pipeline or speech-to-speech model?
For most support flows, cascading (STT → LLM → TTS). It's what ~90% of production voice agents run on in 2026, because you can swap any stage, log every step, and reuse the tool-calling and retrieval you already built for chat. Reach for speech-to-speech only when emotional tone and sub-300ms responsiveness are the product.
Hume or ElevenLabs for a customer support line?
For a task-driven line — check an order, reset a password, book an appointment, hand off to a human — ElevenLabs-style cascading wins: fast TTS, strong voice quality, telephony, tool calls, multilingual. Hume's EVI is built for emotional nuance, so it fits high-touch or de-escalation flows where how something is said matters as much as what is said.
Can we use emotion detection on support calls in the EU?
Carefully. The AI Act (Article 5) bans inferring emotions from biometric data in the workplace and education, so you can't use it to monitor your own agents. Inferring a customer's emotion on a call isn't caught by that ban, but it's an emotion-recognition system with transparency duties, and the voice data is biometric under the GDPR — expect a lawful basis, disclosure and usually a DPIA.
What actually makes a voice bot feel slow?
Compounding latency. Each stage streams, but STT, the LLM and TTS still stack up, and most cascaded agents land at 800ms–2s round-trip even though a single TTS step can be 75ms. Budget the whole loop under ~1s, stream every stage, and handle barge-in so the caller can interrupt.
Do we need a fallback to a human?
Always. A voice bot without a clean escalation path is a complaint generator. Detect low confidence, repeated failures and explicit requests, then transfer to a human with the transcript and context attached. The handoff quality is what clients judge the whole system on.
Sources: AssemblyAI, the voice AI stack for building agents (2026) · Coval, speech-to-speech vs cascaded voice AI · Hume AI, introducing EVI 3 and Empathic Voice Interface · ElevenLabs, agents platform · EU AI Act, Article 5 (prohibited practices) · Future of Privacy Forum, emotion recognition in the workplace. We're engineers, not your legal counsel — for anything touching emotion data, loop in your client's DPO and lawyers.
Scoping a voice project and want a second opinion before you quote it? Plan a call — we'll help you pick the architecture, white-label.
