ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback

arXiv:2404.07987v1 Announce Type: cross
Abstract: To enhance the controllability of text-to-image diffusion models, existing efforts like ControlNet incorporated image-based conditional controls. In this paper, we reveal that existing methods still face significant challenges in generating images that align with the image conditional controls. To this end, we propose ControlNet++, a novel approach that improves controllable generation by explicitly optimizing pixel-level cycle consistency between generated images and conditional controls. Specifically, for an input conditional control, we use a pre-trained discriminative reward …

abstract arxiv challenges controlnet cs.ai cs.cv cs.lg diffusion diffusion models face feedback image image diffusion images improving novel paper text text-to-image type