BF16 against FP8 KV cache on Kimi K2.6
Paired bars of output tokens per second at concurrency 1, 8, 16, 32, and 64, as Cloudflare reported. BF16 against FP8: 137 and 125, 731 and 689, 1106 and 1028, 1558 and 1489. At concurrency 64 the BF16 cache ran out of memory and the FP8 cache produced 2192 tokens per second.
OUTPUT TOK/S BY CONCURRENCY
137
125
c=1
731
689
c=8
1106
1028
c=16
1558
1489
c=32
OOM
2192
c=64
BF16 KV cache
FP8 KV cache
Cloudflare, 2026-08-03