On a 24 GB card each token got faster, but the server did not
Two ratios for Qwen3-8B with EAGLE-3 against the BF16 default on one NVIDIA L4, three seeds per point. Time per output token stays about two times shorter at every concurrency. Output throughput is 2.37 times higher at concurrency 1 and falls to 1.00 at 32 and 0.98 at 64, because the KV pool shrank from 24,094 to 9,951 tokens.
x0
x1
x2
x3
1
2
4
8
16
32
64
max concurrency
ratio
no gain
x2.37
x1.00
x0.98
time per token, default over EAGLE-3
output tok/s, EAGLE-3 over default