Enhancing Prompt Following with Visual Control Through Training-Free Mask-Guided Diffusion

arXiv:2404.14768v1 Announce Type: new
Abstract: Recently, integrating visual controls into text-to-image~(T2I) models, such as ControlNet method, has received significant attention for finer control capabilities. While various training-free methods make efforts to enhance prompt following in T2I models, the issue with visual control is still rarely studied, especially in the scenario that visual controls are misaligned with text prompts. In this paper, we address the challenge of ``Prompt Following With Visual Control" and propose a training-free approach named Mask-guided Prompt Following …

abstract arxiv attention capabilities control controlnet cs.cv diffusion free image issue prompt text text-to-image through training type visual