causal mask — one triangle, half a decade of progress
unmasked (BERT-style)
every position sees every position
fits encoders (classification, embeddings)
causal (GPT-style)
position i attends to positions ≤ i only
fits decoders (GPT, Llama, Claude)
what the mask enables
training
full N-token forward pass, N parallel next-token losses in one batch
inference
autoregressive: feed tokens 1..t, generate t+1, repeat. KV cache (lesson 12) avoids re-computation
loss
cross-entropy shift-by-one: predict t_{i+1} from t_{0..i}
the mask is added before softmax; exp(-inf) = 0; masked positions get zero weight.