capstone: a decoder-only transformer, end to end
token IDs (B, N)
token emb + positional emb
repeat L times
RMSNorm
causal multi-head self-attention
+
RMSNorm
SwiGLU FFN
+
RMSNorm
lm_head (tied to token emb)
shift-by-one cross-entropy
one config, thirteen lessons compressed into a single training script.