Why spectral normalization stabilizes GANs: analysis and improvements

Figure 1: Training instability is one of the biggest challenges in training GANs. Despite the existence of successful heuristics like Spectral Normalization (SN) for improving stability, it is poorly-understood why they work. In our research, we theoretically explain why SN stabilizes GAN training. Using these insights, we further propose a better normalization technique for improving GANs’ stability called Bidirectional Scaled Spectral Normalization. Generative adversarial networks (GANs) are a class of popular generative models enabling many cutting-edge applications such as photorealistic …

analysis articles deep dive gans