One GRPO step on one GPU
Six stages repeat every step on one DGX Spark GB10 with Qwen3-0.6B. Prompts go to the SGLang engine, which writes 256 answers in 47.5 s. A math reward scores each answer. The FSDP trainer computes log probabilities in 23.4 s, then takes the gradient step in 77.0 s. Together they are the training phase of 100.3 s. A weight update of 0.64 s gives the engine the new weights. Medians of 48 steps of a run made before we added the heat guard, so no pause could occur.
GSM8K prompts
32 prompts a step
SGLang engine
sglang::scheduler
256 answers, 47.5 s
math reward
1 or 0 per answer
log probabilities
FSDPTrainRayActor
23.4 s
GRPO update
Adam, one step
77.0 s
weight update
GPU handles, GPU copy
0.64 s
1
2
3
4
5
6
ONE STEP, MEDIAN OF 48 STEPS, RUN BEFORE THE HEAT GUARD
152.6 s in all. Training takes the most time, and the weight update the least.
Engine and trainer take turns on the same GPU memory.