Differential Privacy of Noisy (S)GD under Heavy-Tailed Perturbations

arXiv:2403.02051v1 Announce Type: cross
Abstract: Injecting heavy-tailed noise to the iterates of stochastic gradient descent (SGD) has received increasing attention over the past few years. While various theoretical properties of the resulting algorithm have been analyzed mainly from learning theory and optimization perspectives, their privacy preservation properties have not yet been established. Aiming to bridge this gap, we provide differential privacy (DP) guarantees for noisy SGD, when the injected noise follows an $\alpha$-stable distribution, which includes a spectrum of heavy-tailed …

abstract algorithm arxiv attention cs.cr cs.lg differential differential privacy gradient math.st noise optimization perspectives preservation privacy stat.ml stat.th stochastic theory type