GPU memory across one colocated RL step
Six phases of one Miles step on shared GPUs. SGLang holds weights, KV cache, and CUDA graphs while it generates, releases all three, the trainer runs alone, then SGLang gets its weights back, receives the new weights, and finally gets its KV cache and CUDA graphs back.
GENERATE
RELEASE
TRAIN
ONLOAD
UPDATE
ONLOAD KV
SGLang weights
SGLang KV cache
CUDA graphs
trainer
POST /release_memory_occupation
POST /update_weights_from_tensor
filled cell: the memory is resident on the GPU ยท empty cell: released or offloaded