Shuffling Momentum Gradient Algorithm for Convex Optimization

arXiv:2403.03180v1 Announce Type: cross
Abstract: The Stochastic Gradient Descent method (SGD) and its stochastic variants have become methods of choice for solving finite-sum optimization problems arising from machine learning and data science thanks to their ability to handle large-scale applications and big datasets. In the last decades, researchers have made substantial effort to study the theoretical performance of SGD and its shuffling variants. However, only limited work has investigated its shuffling momentum variants, including shuffling heavy-ball momentum schemes for non-convex …

abstract algorithm applications arxiv become big cs.lg data data science datasets gradient machine machine learning machine learning and data science math.oc optimization researchers scale science stochastic type variants