Sample Average Approximation for Black-Box VI. (arXiv:2304.06803v1 [cs.LG])

We present a novel approach for black-box VI that bypasses the difficulties
of stochastic gradient ascent, including the task of selecting step-sizes. Our
approach involves using a sequence of sample average approximation (SAA)
problems. SAA approximates the solution of stochastic optimization problems by
transforming them into deterministic ones. We use quasi-Newton methods and line
search to solve each deterministic optimization problem and present a heuristic
policy to automate hyperparameter selection. Our experiments show that our
method simplifies the VI problem …

approximation arxiv automate box gradient hyperparameter line novel optimization performance policy search solution stochastic