What SGLang owns
A stack of nine layers from the client at the top to the hardware at the bottom. SGLang owns the tokenizer manager, the scheduler, the KV pool with its radix cache, the forward pass, and the detokenizer. The model gateway is a separate component. The model weights decide answer quality, and the operator decides the memory budget.
LAYER
OWNER
client
curl, the OpenAI SDK, sglang.benchmark.serving
yours
sgl-model-gateway
routes requests across servers, section 6.4
separate
TokenizerManager
HTTP and OpenAI API, text to token ids
SGLang
Scheduler
batches, admission, retraction, overlap
SGLang
KV pool and UnifiedRadixCache
181,574 slots, cached prefixes
SGLang
forward pass
attention backend, CUDA graphs, MLX on a Mac
SGLang
DetokenizerManager
token ids to text, stop strings, streaming
SGLang
model weights
mlx-community/Qwen3-0.6B-4bit, answer quality
the model
memory budget and hardware
--mem-fraction-static, other programs
operator