Espresso: Robust Concept Filtering in Text-to-Image Models

arXiv:2404.19227v1 Announce Type: new
Abstract: Diffusion-based text-to-image (T2I) models generate high-fidelity images for given textual prompts. They are trained on large datasets scraped from the Internet, potentially containing unacceptable concepts (e.g., copyright infringing or unsafe). Retraining T2I models after filtering out unacceptable concepts in the training data is inefficient and degrades utility. Hence, there is a need for concept removal techniques (CRTs) which are effective in removing unacceptable concepts, utility-preserving on acceptable concepts, and robust against evasion with adversarial prompts. …

abstract arxiv concept concepts copyright cs.cr cs.cv data datasets diffusion espresso fidelity filtering generate image images internet large datasets prompts retraining robust text text-to-image textual training training data type utility