Streamlining in the Riemannian Realm: Efficient Riemannian Optimization with Loopless Variance Reduction

arXiv:2403.06677v1 Announce Type: new
Abstract: In this study, we investigate stochastic optimization on Riemannian manifolds, focusing on the crucial variance reduction mechanism used in both Euclidean and Riemannian settings. Riemannian variance-reduced methods usually involve a double-loop structure, computing a full gradient at the start of each loop. Determining the optimal inner loop length is challenging in practice, as it depends on strong convexity or smoothness constants, which are often unknown or hard to estimate. Motivated by Euclidean methods, we introduce …

abstract arxiv computing cs.ai cs.dc cs.lg gradient loop optimization stochastic study type variance