On the Convergence of the ELBO to Entropy Sums

arXiv:2209.03077v5 Announce Type: replace-cross
Abstract: The variational lower bound (a.k.a. ELBO or free energy) is the central objective for many established as well as many novel algorithms for unsupervised learning. During learning such algorithms change model parameters to increase the variational lower bound. Learning usually proceeds until parameters have converged to values close to a stationary point of the learning dynamics. In this purely theoretical contribution, we show that (for a very large class of generative models) the variational lower …

abstract algorithms arxiv change convergence cs.it cs.lg energy entropy free math.it math.pr math.st novel parameters stat.ml stat.th type unsupervised unsupervised learning