Reading a sequence figure in this manual
Six lifelines: the client script, the tokenizer manager, the scheduler, the model on MLX, the detokenizer manager, and macOS. Nine numbered rows use the eight arrow styles: POST /v1/chat/completions as a solid ink call, TokenizedGenerateReqInput as a dotted indigo ZMQ message, waiting to running as a dashed amber state change, a prefill of 1 new token with 2998 cached as a solid plum model call, the Metal memory limit of 51.8 GB as a dashed olive outside effect, the model files in the Hugging Face cache as a solid teal durable write, BatchTokenIDOutput as a dotted indigo message, 200 OK with cached_tokens 2998 as a dashed ink reply, and Insufficient Memory at concurrency 32 as a dashed rose failure.
client
Python script
tokenizer
manager
scheduler
event loop
model
MLX, Qwen3
detokenizer
manager
macOS
Metal, disk
1
POST /v1/chat/completions
a call · solid ink
2
TokenizedGenerateReqInput
a ZMQ message · dotted indigo
3
request waiting to running
a state change · dashed amber
4
prefill 1 new token, 2998 cached
model compute · solid plum
5
Metal memory limit 51.8 GB
an outside effect · dashed olive
6
9 model files to the HF cache
a durable write · solid teal
7
BatchTokenIDOutput
a ZMQ message · dotted indigo
8
200 OK, cached_tokens 2998
a reply · dashed ink
9
Insufficient Memory at c=32
a failure · dashed rose