Eigenvalues of Autoencoders in Training and at Initialization. (arXiv:2201.11813v1 [cs.LG])

Web: http://arxiv.org/abs/2201.11813

In this paper, we investigate the evolution of autoencoders near their
initialization. In particular, we study the distribution of the eigenvalues of
the Jacobian matrices of autoencoders early in the training process, training
on the MNIST data set. We find that autoencoders that have not been trained
have eigenvalue distributions that are qualitatively different from those which
have been trained for a long time ($>$100 epochs). Additionally, we find that
even at early epochs, these eigenvalue distributions rapidly become
qualitatively …

arxiv training