Which decode batches replay a graph A decode batch arrives at the decode CUDA graph runner on an 80 GB GPU at tensor parallel size 1, where max_bs is 256. A batch of 13 fits and replays the graph captured for 16. A batch of 256 replays the largest graph. A batch of 300 is above max_bs and runs eager, one launch per kernel, and the scheduler log shows cuda graph False. DECODE BATCH CAN_RUN_GRAPH WHAT RUNS bs = 13 running requests 13 <= 256 fits replay graph 16 3 padded slots bs = 256 running requests 256 <= 256 fits replay graph 256 largest bucket bs = 300 running requests 300 > 256 too large eager forward cuda graph: False captured sizes: 1, 2, 4, 8, 12, then 16 to 256 in steps of 8 max_bs 256 is the default on an 80 GB GPU at TP 1