Start from what you see
Five symptoms from our runs, each with the first check and the fix. An idle GPU with a growing log means a dead engine. A FlashAttention 3 error means a GPU that is not Hopper, because the FA3 build in the image is for Hopper only. A Ray out-of-memory error means the node was full. Step times that differ mean heat pauses. A rejected trainer attention name means a name that the installed transformers library does not accept.
SYMPTOM
FIRST CHECK
FIX
GPU idle, log grows
count 503 lines
read the first error
FlashAttention 3 error
compute capability
not Hopper: no FA3
Ray OutOfMemoryError
memory at the kill
use a smaller model
step times differ
heat pauses
compare clean steps
attention name rejected
the ValueError list
pick a listed name
Section 7.1 has every symptom we hit, and six more from issue reports.