Efficient Algorithms for Empirical Group Distributional Robust Optimization and Beyond

arXiv:2403.03562v1 Announce Type: new
Abstract: We investigate the empirical counterpart of group distributionally robust optimization (GDRO), which aims to minimize the maximal empirical risk across $m$ distinct groups. We formulate empirical GDRO as a $\textit{two-level}$ finite-sum convex-concave minimax optimization problem and develop a stochastic variance reduced mirror prox algorithm. Unlike existing methods, we construct the stochastic gradient by per-group sampling technique and perform variance reduction for all groups, which fully exploits the $\textit{two-level}$ finite-sum structure of empirical GDRO. Furthermore, we …

abstract algorithm algorithms arxiv beyond cs.lg minimax optimization risk robust stat.ml stochastic type variance