Eager mode cost 1.2 to 2.3 percent of the time per token on the L4
Median time per output token against users at the same time on an NVIDIA L4 with Qwen3-8B, three seeds per point, both runs on 2026-10-21. Eager mode is 1.2 to 2.3 percent slower at every concurrency, for example 61.1 against 60.4 ms at one user and 131.4 against 129.2 ms at 64 users.
50
80
110
140
1
2
4
8
16
32
64
users at the same time
median ms per token
default, -O2
--enforce-eager