Decentralized Stochastic Subgradient Methods for Nonsmooth Nonconvex Optimization

arXiv:2403.11565v1 Announce Type: cross
Abstract: In this paper, we concentrate on decentralized optimization problems with nonconvex and nonsmooth objective functions, especially on the decentralized training of nonsmooth neural networks. We introduce a unified framework, named DSM, to analyze the global convergence of decentralized stochastic subgradient methods. We prove the global convergence of our proposed framework under mild conditions, by establishing that the generated sequence asymptotically approximates the trajectories of its associated differential inclusion. Furthermore, we establish that our proposed framework …

abstract analyze arxiv convergence cs.lg decentralized framework functions global math.oc networks neural networks optimization paper prove stochastic training type