Using Constant Learning Rate of Two Time-Scale Update Rule for Training Generative Adversarial Networks. (arXiv:2201.11989v1 [cs.LG])

Web: http://arxiv.org/abs/2201.11989

Previous numerical results have shown that a two time-scale update rule
(TTUR) using constant learning rates is practically useful for training
generative adversarial networks (GANs). Meanwhile, a theoretical analysis of
TTUR to find a stationary local Nash equilibrium of a Nash equilibrium problem
with two players, a discriminator and a generator, has been given using
decaying learning rates. In this paper, we give a theoretical analysis of TTUR
using constant learning rates to bridge the gap between theory and practice. …

arxiv learning networks rate scale time training