An Ordering of Divergences for Variational Inference with Factorized Gaussian Approximations

arXiv:2403.13748v1 Announce Type: cross
Abstract: Given an intractable distribution $p$, the problem of variational inference (VI) is to compute the best approximation $q$ from some more tractable family $\mathcal{Q}$. Most commonly the approximation is found by minimizing a Kullback-Leibler (KL) divergence. However, there exist other valid choices of divergences, and when $\mathcal{Q}$ does not contain~$p$, each divergence champions a different solution. We analyze how the choice of divergence affects the outcome of VI when a Gaussian with a dense covariance …

abstract approximation arxiv compute cs.lg distribution divergence family found however inference stat.co stat.ml tractable type