voice pipeline — target p50 first-audio-out < 800ms
streaming stages (all pipelined)
audio in (WebRTC)
20ms PCM frames
ASR streaming
Deepgram Nova-3
VAD + turn-detect
Silero v5 + LK detector
LLM streaming
GPT-4o-realtime
TTS streaming
Cartesia Sonic-2
latency budget (800ms total)
ASR partial + VAD silence detect: ~320ms
Silero runs per 20ms frame; turn-detector reads partial when silence >= 500ms
turn completion decision: 40ms
completion score from partial transcript, threshold 0.6; else keep listening
LLM time-to-first-token: 140ms
streaming completion; 1M context supported; short system prompt cached
barge-in arbitration
VAD speech-on during TTS -> cancel
drop remaining LLM output, clear TTS buffer
re-arm ASR for new turn
publish tts_canceled span (OpenTelemetry)
false-cutoff rate target < 3%
Hamming VAD benchmark measures this
tool side channel
mid-conversation call (weather, calendar)
runs concurrent to LLM stream
if > 300ms: emit filler "one second, let me check"
resume LLM on tool return, splice result into prompt
FastMCP transport, per-tool timeout
never stall the audio stream