Six findings and the mechanism behind each
Six rows from the independent field reports. Each row names a workload, the result that was reported, and the SGLang mechanism that explains it: kernels, the radix cache, chunked prefill, the decode graph limit, attention data and expert parallelism, and the cost of draft tokens.
WORKLOAD
REPORTED RESULT
MECHANISM
unique prompts, dense
tie within a few percent
same kernel families
repeated prefixes
first token 583 vs 2,141 ms
radix cache
16k-token prompts
slower first token, faster TPOT
chunked prefill 8,192
no concurrency cap
ITL 100.92 vs 28.84 ms
graph limit, a guess
DeepSeek MoE, many GPUs
reference engine, audited MLPerf
attention DP and EP
MTP on a B200
TPOT 18.43 to 9.92 ms
draft cost, acceptance
A dashed border marks our hypothesis, not a finding of the report.