Moshi — full-duplex dialogue, 3 parallel streams, 200 ms latency
user audio in
Mimi 12.5 Hz × 8 codebooks
Temporal Transformer (7B, 80 ms/step)
sees three streams, predicts all three
user Mimi t-k...t
moshi Mimi t-k...t-1
moshi text t-k...t-1
predict moshi text[t]
= inner monologue
predict moshi Mimi[t]
via Depth Transformer (8 codebooks)
text out
transcript for free
moshi audio out
via Mimi decoder
why full-duplex beats a pipeline
pipeline (VAD + STT + LLM + TTS): latency bound by longest stage, rigid turn structure
full-duplex (Moshi): both streams active; interrupt, back-channel, 200 ms theoretical floor
2026 streaming S2S picks
Moshi — lowest-latency voice companion, EN + FR · CC-BY 4.0
Hibiki / Hibiki-Zero — streaming speech-to-speech translation · CC-BY 4.0
GPT-4o Realtime / Gemini Live — closed commercial peers · commercial