With the cache on, the first token stays near 400 ms at every shared length
Median time to first token against the shared system prompt length, on an NVIDIA L4 with Qwen3-8B, 8 users, three seeds, log scale. With the cache on, it stays between 375 and 488 ms. With it off, it grows to 1,213 ms at 1,024 tokens and 13,353 ms at 4,096 tokens, where the GPU clock also fell.
300
1k
3k
10k
0
1,024
2,048
3,072
4,096
shared system prompt, tokens
median TTFT, ms (log scale)
13,353
1,213
prefix cache on, the default
prefix cache off