FEDNEST: Federated Bilevel, Minimax, and Compositional Optimization. (arXiv:2205.02215v1 [cs.LG])

Web: http://arxiv.org/abs/2205.02215

Standard federated optimization methods successfully apply to stochastic
problems with \textit{single-level} structure. However, many contemporary ML
problems -- including adversarial robustness, hyperparameter tuning, and
actor-critic -- fall under nested bilevel programming that subsumes minimax and
compositional optimization. In this work, we propose FedNest: A federated
alternating stochastic gradient method to address general nested problems. We
establish provable convergence rates for FedNest in the presence of
heterogeneous data and introduce variations for bilevel, minimax, and
compositional optimization. FedNest introduces multiple innovations …

arxiv minimax optimization