Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation

arXiv:2311.17216v2 Announce Type: replace
Abstract: Diffusion-based models have gained significant popularity for text-to-image generation due to their exceptional image-generation capabilities. A risk with these models is the potential generation of inappropriate content, such as biased or harmful images. However, the underlying reasons for generating such undesired content from the perspective of the diffusion model's internal representation remain unclear. Previous work interprets vectors in an interpretable latent space of diffusion models as semantic concepts. However, existing approaches cannot discover directions for …

arxiv cs.cv diffusion image image generation responsible text text-to-image type