Momentum-based gradient descent methods for Lie groups

arXiv:2404.09363v1 Announce Type: cross
Abstract: Polyak's Heavy Ball (PHB; Polyak, 1964), a.k.a. Classical Momentum, and Nesterov's Accelerated Gradient (NAG; Nesterov, 1983) are well know examples of momentum-descent methods for optimization. While the latter outperforms the former, solely generalizations of PHB-like methods to nonlinear spaces have been described in the literature. We propose here a generalization of NAG-like methods for Lie group optimization based on the variational one-to-one correspondence between classical and accelerated momentum methods (Campos et al., 2023). Numerical experiments …

abstract arxiv cs.lg cs.na examples gradient literature math.dg math.na math.oc optimization spaces type