The Evidence Lower Bound of Variational Autoencoders Converges to a Sum of Three Entropies. (arXiv:2010.14860v4 [stat.ML] UPDATED)

The central objective function of a variational autoencoder (VAE) is its
variational lower bound (the ELBO). Here we show that for standard (i.e.,
Gaussian) VAEs the ELBO converges to a value given by the sum of three
entropies: the (negative) entropy of the prior distribution, the expected
(negative) entropy of the observable distribution, and the average entropy of
the variational distributions (the latter is already part of the ELBO). Our
derived analytical results are exact and apply for small as …

arxiv evidence variational autoencoders