One turn of the scheduler loop
ingest_requests drains new requests, calc_priority orders the waiting queue with the fcfs default, PrefillAdder.add_one_req admits requests while the KV budget allows. If anything is admitted the turn runs a prefill batch, otherwise one decode step for every running request. process_batch_result then appends tokens and checks stops, and the next turn starts.
ingest_requests
drain ZMQ, add to the waiting queue
calc_priority
order the queue: fcfs by default
PrefillAdder.add_one_req
admit while the KV budget allows
admitted
nothing admitted
prefill batch
EXTEND forward of admitted requests
decode step
one token for every running request
process_batch_result
append tokens, check stops, send results
next turn