Stochastic Hessian Fitting on Lie Group

arXiv:2402.11858v1 Announce Type: cross
Abstract: This paper studies the fitting of Hessian or its inverse with stochastic Hessian-vector products. A Hessian fitting criterion, which can be used to derive most of the commonly used methods, e.g., BFGS, Gaussian-Newton, AdaGrad, etc., is used for the analysis. Our studies reveal different convergence rates for different Hessian fitting methods, e.g., sublinear rates for gradient descent in the Euclidean space and a commonly used closed-form solution, linear rates for gradient descent on the manifold …

abstract analysis arxiv convergence criterion cs.lg etc math.oc paper products stat.ml stochastic studies type vector