The default attention backend at v0.5.22 A decision tree. The model attention type splits into MHA on the left and MLA on the right. For MHA: Hopper with CUDA 12.3 gives fa3 because of issue 17411, Blackwell SM100 gives trtllm_mha or fa4 for unequal K and V widths, AMD gives aiter, Apple MPS gives torch_native, and any other GPU gives flashinfer, or triton when the model has attention sinks. For MLA: Hopper gives fa3, Blackwell gives flashinfer, which DeepSeek V3 overrides to trtllm_mla, AMD gives aiter for 16 or 128 KV heads, and others give triton. get_default_attn_backend no --attention-backend given MHA model Llama, Qwen, Gemma MLA model DeepSeek V3, GLM MoE Hopper fa3 issue 17411 Blackwell SM100 trtllm_mha fa4 for unequal K, V AMD ROCm aiter all MHA models Apple MPS torch_native PyTorch SDPA other GPUs flashinfer triton with sinks Hopper fa3 same as MHA Blackwell SM100 flashinfer DeepSeek V3 override AMD ROCm aiter if 16 or 128 KV heads Apple MPS torch_native PyTorch SDPA other GPUs triton portable kernels