The three processes of one SGLang server
A client sends POST /generate to the main process, pid 5782, which holds the HTTP server and the TokenizerManager. A TokenizedGenerateReqInput goes over ZMQ to scheduler_0, pid 5909, which runs the MLX model. A BatchTokenIDOutput goes to the detokenizer, which sends a BatchStrOutput back to the main process, and the main process answers the client.
client
curl or the benchmark
main · pid 5782
HTTP server
TokenizerManager
scheduler_0 · pid 5909
queue and batches
KV pool and radix tree
detokenizer
token ids to text
MLX model
Qwen3-0.6B-4bit
1
2
3
4
5
6
THE NUMBERED STEPS
1
POST /generate over HTTP
2
TokenizedGenerateReqInput over ZMQ
3
forward pass and sampling
4
BatchTokenIDOutput over ZMQ
5
BatchStrOutput over ZMQ
6
HTTP response with the text