DP-RDM: Adapting Diffusion Models to Private Domains Without Fine-Tuning

arXiv:2403.14421v1 Announce Type: new
Abstract: Text-to-image diffusion models have been shown to suffer from sample-level memorization, possibly reproducing near-perfect replica of images that they are trained on, which may be undesirable. To remedy this issue, we develop the first differentially private (DP) retrieval-augmented generation algorithm that is capable of generating high-quality image samples while providing provable privacy guarantees. Specifically, we assume access to a text-to-image diffusion model trained on a small amount of public data, and design a DP retrieval …

abstract algorithm arxiv cs.cr cs.cv cs.lg diffusion diffusion models domains fine-tuning image image diffusion images issue near quality replica retrieval retrieval-augmented sample text text-to-image type