A New Random Reshuffling Method for Nonsmooth Nonconvex Finite-sum Optimization

arXiv:2312.01047v2 Announce Type: replace-cross
Abstract: Random reshuffling techniques are prevalent in large-scale applications, such as training neural networks. While the convergence and acceleration effects of random reshuffling-type methods are fairly well understood in the smooth setting, much less studies seem available in the nonsmooth case. In this work, we design a new normal map-based proximal random reshuffling (norm-PRR) method for nonsmooth nonconvex finite-sum problems. We show that norm-PRR achieves the iteration complexity $O(n^{-1/3}T^{-2/3})$ where $n$ denotes the number of component …

abstract applications arxiv case convergence cs.lg design effects math.oc networks neural networks optimization random scale studies sum training type while work