More users, more tokens in total, a slower stream for each user
vLLM 0.31.0 on a DGX Spark with Qwen3-30B-A3B, three seeds per point. Left: output tokens per second for all users rise from 29.5 at one user to 239.3 at 64 users. Right: the median time per output token for each user rises from 32.3 ms to 243.8 ms.
0
100
200
300
1
4
16
64
users at the same time
output tokens per second, all users
0
100
200
300
1
4
16
64
users at the same time
median ms per token, each user
median of three runs, band from lowest to highest