encoder block vs decoder block — the 2026 skeleton
encoder block (bidirectional)
input x (N, d)
RMSNorm
multi-head self-attention
+
residual
RMSNorm
SwiGLU FFN
+
out (N, d)
decoder block (causal + cross-attn)
input x (M, d)
RMSNorm
masked self-attention
+
RMSNorm
cross-attention (Q:dec, KV:enc)
enc_out
+
RMSNorm
SwiGLU FFN
encoder: 2 sublayers per block. decoder: 3 sublayers per block. residuals are dashed, additions are circles.