A larger token budget at 0.5 requests per second on the DGX Spark
vLLM with Qwen3-30B-A3B on an NVIDIA GB10, prompts of 1,024 to 4,096 tokens, 64 to 256 output tokens, 0.5 requests per second, three seeds. With a budget of 2,048 tokens, the median first token took 1,145 ms and the P99 ITL was 574 ms. With 16,384 tokens, the first token took 906 ms and the P99 ITL was 798 ms.
median first token, ms
0
750
1,500
1,145
2,048
906
16,384
P99 ITL, ms
0
500
1,000
574
2,048
798
16,384
Lower is better on both. Bars are labelled by --max-num-batched-tokens.