On a DGX Spark, vLLM takes a share of one memory pool
The GB10 reports 121.7 GiB in total. With a fraction of 0.75, vLLM takes 91.3 GiB: 56.93 GiB of weights, 31.32 GiB of KV cache, and about 3 GiB for the startup profile and the CUDA graphs. The other 30.4 GiB stays with the operating system, the page cache, and other programs.
ONE 121.7 GIB POOL, CPU AND GPU TOGETHER
vLLM, --gpu-memory-utilization 0.75, 91.3 GiB
56.93 GiB
weights
31.32 GiB
KV cache
about 3 GiB
profile and graphs
30.4 GiB
operating system, page cache, other programs