Voice in Production
The Real-Time Voice Agent Stack
The 800-millisecond budget
Human conversation gaps average ~200-300ms; under ~800ms total, a voice agent still feels 'alive'. That budget must cover the whole loop:
1. Endpointing — detect the user finished speaking (150-300ms of silence analysis) 2. ASR finalize — commit the transcript (or hand the partial to the LLM) 3. LLM TTFT — first response tokens 4. TTS TTFB — first audio bytes 5. Playback + network — the last mile
The budget is shared: over-spend on endpointing and the LLM's speed can't save you. This is why voice-agent engineering is a systems discipline — nobody's stage is optimal; the sum is the product.
insight
Voice agents are won or lost in the gaps. Every stage's p95 latency contributes to the sum — and a single 2-second tail anywhere makes the whole agent feel dead.