Output throughput against concurrency on a GB10, three seeds per point
Qwen3-30B-A3B on one NVIDIA GB10, 512 tokens in and 128 out. FP8 rises to 441 tokens per second at concurrency 64, BF16 to 235. EAGLE-3 leads BF16 up to 32 and reaches 276 at 64. The run without CUDA graphs lies on top of the default. Bands show the lowest and highest of three seeds.
0
100
200
300
400
500
1
2
4
8
16
32
64
max concurrency (benchmark client)
output tokens per second
441
235
FP8 + FP8 KV
EAGLE-3
BF16 default
no CUDA graph