Discriminative Bayesian filtering lends momentum to the stochastic Newton method for minimizing log-convex functions. (arXiv:2104.12949v2 [stat.ML] UPDATED)

Web: http://arxiv.org/abs/2104.12949

To minimize the average of a set of log-convex functions, the stochastic
Newton method iteratively updates its estimate using subsampled versions of the
full objective's gradient and Hessian. We contextualize this optimization
problem as sequential Bayesian inference on a latent state-space model with a
discriminatively-specified observation process. Applying Bayesian filtering
then yields a novel optimization algorithm that considers the entire history of
gradients and Hessians when forming an update. We establish matrix-based
conditions under which the effect of older observations …

arxiv bayesian filtering ml stochastic