Thinker-Talker — streaming pipeline for real-time voice
parallel streaming: Thinker and Talker run concurrently
user audio
mic 16 kHz
+ webcam 4 fps
streaming
tokens in
VAD for turn-taking
Thinker (7-80B)
text-generating
large LLM reasoning
TMRoPE timestamps
emits text tokens
first token ~40ms
streams per token
into Talker
Talker (200M-1B)
speech-generating
small + fast
residual-VQ output
8 codebooks
50 tok/s throughput
keeps pace with speech
waveform
SNAC decoder
16 kHz samples
speaker output
~70 ms decode
streaming
TTFAB budget and open implementations
mic -> audio tokens
40 ms
Thinker prefill
100 ms at 7B
first text token
40 ms
Talker first tokens
20 ms
RVQ + waveform
100 ms
open implementations
Mini-Omni : first open streaming
Moshi : 160 ms, inner monologue
Qwen2.5-Omni : ~350 ms, TMRoPE
Qwen3-Omni : close to GPT-4o
GLM-4-Voice : Chinese-first
GPT-4o reference: ~250 ms