Two copies of the model served less than one
Output throughput against concurrency from 1 to 64 for Qwen3-8B, three seeds per point with a band from the lowest to the highest run. One L4 climbs to 270 tokens per second at 32 and 251 at 64. With --dp-size 2 on two L4s, where the second card was shared, the line peaks at 64 tokens per second at 8 and falls to 42 at 64.
0
100
200
300
1
2
4
8
16
32
64
max concurrency
output tokens per second
251
42
one L4, 24,094-token pool
--dp-size 2, pools of 24,094 and 2,925