adversarial training: two networks, one loss
z ~ N(0, I)
G(z)
generator
x̂ (fake)
x (real)
D(x)
discriminator
real? 0-1
BCE loss
real -> 1, fake -> 0
minimax: min_G max_D E_real[log D(x)] + E_fake[log(1 - D(G(z)))]
use non-saturating G loss -log D(G(z)) to avoid vanishing gradients
failure: D wins
D(fake) → 0, gradient to G vanishes
fix: cut D lr, add input noise, WGAN
failure: G collapses
G outputs one mode, D can't penalize
fix: minibatch disc, spectral norm, PacGAN