Two prefix caches, one result
Median time to first token against shared prompt length from 0 to 4,096 tokens for SGLang 0.5.22 with its radix cache, vLLM 0.31.0 with prefix caching, and SGLang with the cache off, serving Qwen3-8B on one NVIDIA L4 at 8 users, three seeds per point, on a log scale. The two cached lines stay between 354 and 488 ms at every length. Without the cache the median rises to 14,706 ms at 4,096 shared tokens.
200
500
1k
2k
5k
10k
20k
0
256
1k
2k
4.1k
shared prompt tokens
median TTFT, ms (log scale)
SGLang, radix cache
vLLM, prefix caching
SGLang, no cache