Differentially Private Latent Diffusion Models

arXiv:2305.15759v4 Announce Type: replace-cross
Abstract: Diffusion models (DMs) are widely used for generating high-quality high-dimensional images in a non-differentially private manner. To address this challenge, recent papers suggest pre-training DMs with public data, then fine-tuning them with private data using DP-SGD for a relatively short period. In this paper, we further improve the current state of DMs with DP by adopting the Latent Diffusion Models (LDMs). LDMs are equipped with powerful pre-trained autoencoders that map the high-dimensional pixels into lower-dimensional …

abstract arxiv challenge cs.cr cs.lg current data diffusion diffusion models fine-tuning images latent diffusion models paper papers pre-training private data public public data quality state stat.ml them training type