What one token costs in Qwen3-0.6B
Five factors from the Qwen3-0.6B config multiply to 114,688 bytes per token: 2 for key and value, 28 layers, 8 KV heads, a head dimension of 128, and 2 bytes for bfloat16. The startup log prints the same number as bytes_per_slot, and 181,574 slots of that size make a pool of 20.82 GB, which is 19.39 GiB.
ONE TOKEN, FIVE FACTORS
2
K and V
×
28
layers
×
8
KV heads
×
128
head dim
×
2 B
bfloat16
=
114,688 B
112 KiB
THE SAME NUMBER IN THE STARTUP LOG
Auto-sized attention KV pool
kv_budget=19.39 GB · bytes_per_slot=114688 · pool_size=181574
POOL SIZE
181,574 slots × 114,688 bytes = 20.82 GB, or 19.39 GiB, of KV