Differentially Private Optimization with Sparse Gradients

arXiv:2404.10881v1 Announce Type: cross
Abstract: Motivated by applications of large embedding models, we study differentially private (DP) optimization problems under sparsity of individual gradients. We start with new near-optimal bounds for the classic mean estimation problem but with sparse data, improving upon existing algorithms particularly for the high-dimensional regime. Building on this, we obtain pure- and approximate-DP algorithms with almost optimal rates for stochastic convex optimization with sparse gradients; the former represents the first nearly dimension-independent rates for this problem. …

abstract algorithms applications arxiv building cs.lg data embedding embedding models improving math.oc mean near optimization sparsity stat.ml study type