AdaGDA: Faster Adaptive Gradient Descent Ascent Methods for Minimax Optimization. (arXiv:2106.16101v4 [math.OC] UPDATED)

In the paper, we propose a class of faster adaptive Gradient Descent Ascent
(GDA) methods for solving the nonconvex-strongly-concave minimax problems based
on unified adaptive matrices, which include almost existing coordinate-wise and
global adaptive learning rates. Specifically, we propose a fast Adaptive
Gradient Decent Ascent (AdaGDA) method based on the basic momentum technique,
which reaches a lower gradient complexity of $O(\kappa^4\epsilon^{-4})$ for
finding an $\epsilon$-stationary point without large batches, which improves
the results of the existing adaptive GDA methods by …

arxiv gradient math minimax optimization