speculative decoding loop — EAGLE-3 style
prefix x_1 ... x_k
draft N tokens: d_1, d_2, ..., d_N ~ p
verifier q(prefix + drafts) — ONE forward pass
Leviathan rule (for each i = 1..N)
accept d_i if u < min(1, q(d_i) / p(d_i)) else emit correction ~ (q - p)_+ / ||(q - p)_+||_1 and STOP
all N accepted
emit d_1 ... d_N + bonus ~ q_{N+1}
kv_length += N + 1 | up to N+1 tokens per verify
reject at position j
emit d_1 ... d_{j-1} + correction ~ residual
kv truncate_to(k + j) | discard drafts after j
the draft evolution — what each generation bought
Leviathan 2023
separate small LLM · alpha ~ 0.60 · 2x speedup
EAGLE-1 (2024)
1-layer transformer on verifier hidden states · alpha ~ 0.75 · 2.5x
EAGLE-2 (2024)
+ dynamic draft tree · alpha ~ 0.85 · 3x-4x
EAGLE-3 (2025)
drop feature loss + training-time test · alpha ~ 0.90+ · 3x-6.5x
Medusa (2024)
extra LM heads on verifier (no draft model) · alpha ~ 0.70 · 2x-3x
output distribution = verifier distribution. exactly. the only variable is alpha and N.