How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimization

arXiv:2402.11173v1 Announce Type: new
Abstract: We provide a simple and flexible framework for designing differentially private algorithms to find approximate stationary points of non-convex loss functions. Our framework is based on using a private approximate risk minimizer to "warm start" another private algorithm for finding stationary points. We use this framework to obtain improved, and sometimes optimal, rates for several classes of non-convex loss functions. First, we obtain improved rates for finding stationary points of smooth non-convex empirical loss functions. …

abstract algorithm algorithms arxiv cs.cr cs.lg designing framework functions loss math.oc optimization risk simple small type warm