Implicit and Explicit Language Guidance for Diffusion-based Visual Perception

arXiv:2404.07600v1 Announce Type: new
Abstract: Text-to-image diffusion models have shown powerful ability on conditional image synthesis. With large-scale vision-language pre-training, diffusion models are able to generate high-quality images with rich texture and reasonable structure under different text prompts. However, it is an open problem to adapt the pre-trained diffusion model for visual perception. In this paper, we propose an implicit and explicit language guidance framework for diffusion-based perception, named IEDP. Our IEDP comprises of an implicit language guidance branch and …

abstract adapt arxiv cs.cv diffusion diffusion model diffusion models generate guidance however image image diffusion images language perception pre-training prompts quality scale synthesis text text-to-image texture training type vision visual