autoencoder vs VAE — the one trick
plain autoencoder
x
encoder
z
loss = ||x - x̂||²
z-space: lumpy, not a distribution
sample random z -> garbage out
variational autoencoder
x
encoder
μ(x)
log σ²(x)
z = μ + σ·ε
reparameterize
ε ~ N(0, I)
decoder
x̂
loss = reconstruction + β · KL
||x - x̂||² + ½ Σ( σ² + μ² - logσ² - 1 )
recon term: push x̂ → x
KL term: push q(z|x) → N(0, I)
β knob trades sharpness for well-shaped latent
inference: sample z ~ N(0, I), forward through decoder → new x̂
one forward pass; no iteration; decoder is the whole generator