Differentially Private Learning with Adaptive Clipping. (arXiv:1905.03871v5 [cs.LG] UPDATED)

Web: http://arxiv.org/abs/1905.03871

Existing approaches for training neural networks with user-level differential
privacy (e.g., DP Federated Averaging) in federated learning (FL) settings
involve bounding the contribution of each user's model update by clipping it to
some constant value. However there is no good a priori setting of the clipping
norm across tasks and learning settings: the update norm distribution depends
on the model architecture and loss, the amount of data on each device, the
client learning rate, and possibly various other parameters. We …

arxiv learning