TTFT quantiles of three runs at a cap of 16
Quantile curve of time to first token for the pooled requests of three seeds at a cap of 16 on a GB10, drawn from 51 quantiles between 0 and 100 percent. BF16 and FP8 rise steeply near 140 to 240 milliseconds, then flatten and finish near 1.2 and 1.7 seconds. EAGLE-3 starts later, near 580 milliseconds at the median, and has the shortest tail.
0%
25%
50%
75%
100%
30
100
300
1k
3k
time to first token, ms (log scale)
share of requests
P90
FP8
BF16
EAGLE-3