EAGLE-3 speculative decoding — alpha is the only metric
the flow (K=5)
draft head
5 candidate tokens
target verify (1 forward)
K * alpha accepted
rest rerolled (second pass)
alpha by workload
general chat + EAGLE-3 trained on ShareGPT : alpha 0.60 - 0.80
specialized (code, legal, medical) : alpha 0.40 - 0.60
domain mismatch without retraining draft : alpha 0.30 - 0.45
classic draft-model (Llama 3.2 1B for 70B) : alpha 0.30 - 0.50
break-even alpha at 256 concurrent : ~0.55
below that : P99 tail gets worse even if mean improves
generations
classic draft-model
two models loaded; K forwards per target; alpha 0.3-0.5
EAGLE-1
draft head on last target layer; alpha ~0.5-0.6
EAGLE-2
tree drafts, adaptive K; alpha ~0.6-0.7
EAGLE-3 (2025-2026)
multi-layer draft head; alpha ~0.6-0.8
all EAGLE variants train fast and add small parameter overhead to the target
production checklist
1. baseline target model plain : TTFT, ITL, throughput at target concurrency
2. enable speculative_config with EAGLE-3 draft ; rerun benchmark
3. log alpha (vLLM V1 reports accepted tokens per request) ; gate on alpha >= 0.55
4. watch P99 ITL, not P50 — rejected-draft two-pass drives tail
5. vLLM v0.18.0 : check the feature matrix ; spec decode composes with chunked prefill