The same server on the same L4, measured hot and measured with a cool-down
Output throughput of Qwen3-8B on one NVIDIA L4 against concurrency, in two sweeps with the same command. The first sweep ran with the card near 87 C and no telemetry. The second waited for 72 C before each run and recorded the clock. The cooled sweep is higher at every point, by up to 39 percent at concurrency 32, and it falls at 64 when the KV pool is full.
0
100
200
300
1
2
4
8
16
32
64
max concurrency (benchmark client)
output tokens per second
270
194
cool-down to 72 C, clock recorded
hot, no telemetry