Variational Inference with Sequential Sample-Average Approximations

arXiv:2403.09429v1 Announce Type: cross
Abstract: We present variational inference with sequential sample-average approximation (VISA), a method for approximate inference in computationally intensive models, such as those based on numerical simulations. VISA extends importance-weighted forward-KL variational inference by employing a sequence of sample-average approximations, which are considered valid inside a trust region. This makes it possible to reuse model evaluations across multiple gradient steps, thereby reducing computational cost. We perform experiments on high-dimensional Gaussians, Lotka-Volterra dynamics, and a Pickover attractor, which …

abstract approximate inference approximation arxiv cs.lg importance inference inside numerical sample simulations stat.ml trust type visa