On a mixture-of-experts model the EAGLE-3 gain holds as the batch grows
Speedup of median time per output token with EAGLE-3 over the BF16 default for Qwen3-30B-A3B on a GB10. It is 1.60 times at concurrency 1 and 1.56 at 2, peaks at 1.89 at 32 and is 1.84 at 64, while the accept length stays near 2.7 tokens per step.
x0
x1
x2
x3
1
2
4
8
16
32
64
max concurrency (benchmark client)
default TPOT divided by EAGLE-3 TPOT
no gain
x1.60
x1.56
x1.64
x1.72
x1.87
x1.89
x1.84
accept length 2.71 to 2.74 tokens per step at every concurrency
EAGLE-3 speedup