SDEs for Minimax Optimization

arXiv:2402.12508v1 Announce Type: new
Abstract: Minimax optimization problems have attracted a lot of attention over the past few years, with applications ranging from economics to machine learning. While advanced optimization methods exist for such problems, characterizing their dynamics in stochastic scenarios remains notably challenging. In this paper, we pioneer the use of stochastic differential equations (SDEs) to analyze and compare Minimax optimizers. Our SDE models for Stochastic Gradient Descent-Ascent, Stochastic Extragradient, and Stochastic Hamiltonian Gradient Descent are provable approximations of …

abstract advanced applications arxiv attention cs.lg differential dynamics economics machine machine learning math.oc minimax optimization paper stochastic type