Variance Control for Black Box Variational Inference Using The James-Stein Estimator

arXiv:2405.05485v1 Announce Type: new
Abstract: Black Box Variational Inference is a promising framework in a succession of recent efforts to make Variational Inference more ``black box". However, in basic version it either fails to converge due to instability or requires some fine-tuning of the update steps prior to execution that hinder it from being completely general purpose. We propose a method for regulating its parameter updates by reframing stochastic gradient ascent as a multivariate estimation problem. We examine the properties …

abstract arxiv basic black box box control converge cs.lg estimator fine-tuning framework however inference james prior stat.ml succession type update variance