Thomas Piacentini
← Atelier

Chapitre

Sources

Texte original · anglais

The manual ranks its sources by authority, and this page lists every external source the sections cite, with the sections that use each one.

The ranks match how this manual was made. When two sources disagree, the higher one wins and the section says so. The SGLang source files each section used are recorded in the manual source and checked by the audit. They are not listed here. Every page below was read on 2026-10-16. The studies and reports added for the research update were read on 2026-10-22.

The ranked sources

Rank Source What the manual takes from it Source key
1 the SGLang source at the tag, baa9befa every default and every mechanism the source files of each section, in the manual source
2 the captured runs on the Mac and on the GPUs every listing and every number marked as measured the capture file name
3 SGLang issues and pull requests incidents and the design history of each feature issue and its number
4 papers the original design of RadixAttention and the methods SGLang implements paper and a title
5 the SGLang documentation at the tag documented behaviour, checked against the code docs and a heading
6 independent field reports GPUs and scales that this manual could not run field, the author, and the month
7 SGLang team and vendor reports claims, always labeled as the claim of the author team or the vendor, and the month

Rank 1 and rank 2: the code and the captures

Source Used in
SGLang 0.5.22, tag v0.5.22, commit baa9befa381511cd4d56f83980951e5a45fffc6d, github.com/sgl-project/sglang every section
Miles 0.1.2, tag v0.1.2, commit a22fd824, github.com/radixark/miles 7.2
The capture kit and its Mac runs, recorded on 2026-10-16 every section with a listing
The GPU kit and its runs, recorded from 2026-10-16 to 2026-10-21 on an NVIDIA L4 node and a DGX Spark (GB10) every section with a GPU measurement
The Miles image radixark/miles:latest at Miles commit 08752a12, with SGLang 0.5.22.dev46+g7b6993a 7.2

Rank 3: pull requests

The issues are in the incident index. These pull requests explain a design change.

Pull request What it changed Used in
#9241 warm-up followed by a frozen garbage collector 1.2
#11210 overlap with two CUDA streams in one thread 2.4
#14312 renamed sgl-router to sgl-model-gateway 6.4
#23906 moved the prefill graph field into the per-phase config 4.1
#29458 breakable prefill graphs on CUDA 4.1
#32114 deleted Cutlass MLA, Dual Chunk FlashAttention, and the plain GPTQ CUDA kernel 4.2, 4.3
#37898 raised the decode graph cap on 20 to 35 GB GPUs from 24 to 48 4.1
#35455 loaded the KV scales of compressed-tensors checkpoints, merged 2026-08-20 4.3
#40795 deleted the old benchmark aliases 5.1
#41818 added --attn-dp-size and gave --dp-size one meaning 6.1
#42285, #42286, #42287 remove the torch.compile path for prefill graphs 4.1

Rank 4: papers

Paper Used in
Zheng and others, SGLang: Efficient Execution of Structured Language Model Programs, arXiv 2312.07104 2.2, 3.1, 3.3, 4.5
The tail-optimized LRU policy that the code names, arXiv 2510.15152 3.5
Lee and others, From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill, arXiv 2510.08055 2.2
Liu, Yu, and Yang, When Fancy Eviction Fails: Rethinking Cache Replacement For LLM Prefix Reuse, arXiv 2609.28870 3.5
Abramovich and others, SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding, arXiv 2604.09557 4.4, 6.2
Kumar, Sanghavi, and Das, Test-Time Speculation, arXiv 2605.09329 4.4
Agrawal and others, On Evaluating Performance of LLM Inference Serving Systems, arXiv 2507.09019 4.4
Zhao and others, CoRun: Padding is Simple and Efficient for Deterministic LLM Inference, arXiv 2608.14376 4.6
Qin and others, Vosti: Specifying, Implementing, and Verifying Deterministic LLM Inference, arXiv 2609.38981 4.6
Patodiya, Same Request, Different Answer: Quantization Amplifies Cache-Induced Divergence in LLM Serving, arXiv 2609.04748 4.6
Chandrasekar and Kramberger, Identifying and Mitigating Systemic Measurement Bias in Production LLM Inference Benchmarks, arXiv 2605.24217 5.1
Li and others, Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implications, arXiv 2601.08833 6.3

Rank 5: SGLang documentation at the tag

Page Used in
docs/docs/hardware-platforms/apple_metal.mdx the front matter, 2.1, 2.4
docs/docs/advanced_features/hyperparameter_tuning.mdx 2.3
docs/docs/advanced_features/hicache_design.mdx and radix_eviction_policy.mdx 3.5
docs/docs/advanced_features/attention_backend.mdx 4.2
docs/docs/advanced_features/quantization.mdx 4.3
docs/docs/advanced_features/speculative_decoding.mdx 4.4
docs/docs/advanced_features/expert_parallelism.mdx 6.2
docs/docs/advanced_features/pd_disaggregation.mdx 6.3
docs/docs/get-started/install.mdx 7.1
docs/docs/advanced_features/sglang_for_rl.mdx 7.2

Rank 6: independent field reports

Author Report Month Used in
AIMultiple Inference engines benchmark 2026-01, answer 2026-10 1.3, 5.3
Alibaba Cloud, RBG and Mooncake teams A production inference platform with SGLang RBG and Mooncake 2026-03 3.5
Ant Group DeepSeek-R1 on H20 with SGLang, on the LMSYS blog 2025-09 4.4, 6.1, 6.2, 7.1
basbunarhasan on the NVIDIA developer forum SGLang with NVFP4 and DSpark on a DGX Spark 2026-08 3.2
CERIT-SC Running LLMs at CERIT-SC and a later post 2025-10, 2026-04 7.1
Cloudflare Smaller, faster, safer models 2026-08 3.1, 4.3
Crusoe MLPerf Inference v6.1 results on AMD MI355X 2026-09 5.3, 6.2
David Noel Ng GH200 benchmarking, part 4 2026-08 4.4
DeepInfra vLLM vs SGLang 2026-08 5.3
Doubleword Fast SGLang starts 2026-04 7.1
dstack Benchmarking prefill-decode ratios 2025-09 6.3
Hazy Research Tensor-parallel Llama megakernel 2025-09 4.1
InferenceX by SemiAnalysis GLM-5 FP8 on MI355X, DeepSeek-V4-Pro on MI355X in 26 days, and GLM-5 NVFP4 on B200 2026-05 4.1, 5.3
Jarvislabs SGLang vs vLLM on H100 2026-05 4.1, 5.3
Jarvislabs Managed endpoints 2026-08 3.4, 6.4
joshua8.ai LLM inference benchmark 2026-01 4.1, 4.3, 5.3
LinkedIn Engineering Scaling LLM-based ranking systems with SGLang at LinkedIn 2026-02 1.2, 2.1, 7.1
Luca Lowndes SGLang optimisations 2025-11 6.1
Modal LLM almanac 2025-06 5.3
Novita AI SGLang with GLM MoE models, on Hugging Face 2026-01 6.3
Pivot News A summary of a mixed Blackwell benchmark 2026-05 4.3, 5.3
RunPod When to choose SGLang over vLLM 2025-06 3.3, 5.3
SemiAnalysis InferenceMAX 2025-10 5.3
Spheron vLLM vs TensorRT-LLM vs SGLang 2026 5.3
Steve Rawlinson vLLM vs SGLang on dual H100 2025-12 3.2, 5.3
Thinking Machines Defeating nondeterminism in LLM inference 2025-09 4.6
Varad More Inference engine benchmark system 2026-04 3.3, 4.2, 4.5, 5.3
vLLM community Discussion #17221 2025-04 5.3

Rank 7: SGLang team and vendor reports

Author Report Month Used in
The vLLM team The state of FP8 KV cache and attention quantization in vLLM 2026-04 4.3
LMSYS SGLang with Llama 3 2024-07 5.3
LMSYS SGLang v0.4: zero-overhead batch scheduler 2024-12 2.4, 6.1, 6.4
LMSYS DeepSeek with PD disaggregation and large-scale EP on 96 H100 GPUs 2025-05 6.2, 6.3
The Mooncake team on the LMSYS blog Kimi K2 with large-scale EP 2025-07 6.3
LMSYS SGLang HiCache 2025-09 3.5
SGLang v0.5.21 release notes 2026-10 3.5
Google Cloud EAGLE-3 on Vertex AI 2025-11 4.4
Modal with Decagon Decagon case study 2025-11 4.4
LMSYS and RadixArk Miles 2025-11 7.2
RadixArk Miles v0.1 2026-08 7.2
The Miles team docs/advanced/p2p-weight-transfer.md at Miles v0.1.2 2026-10 7.2

Sources: front.md; the Sources line and the citations of every file under sections/.

Code et livrables 1
Ce que vous en retenez

Enregistré dans votre carnet sur cet appareil.

Sources et limites

Les repères de lecture sont déclaratifs. Les résultats de tests sont enregistrés séparément ; ils valent pour les cas exécutés.

AI Engineering from Scratch · 3f9715c · MIT

Télécharger ce fichier ↓