[R] ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback

ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback

​

To enhance the controllability of text-to-image diffusion models, existing efforts like ControlNet incorporated image-based conditional controls. In this paper, we reveal that existing methods still face significant challenges in generating images that align with the image conditional controls. To this end, we propose ControlNet++, a novel approach that improves controllable generation by explicitly optimizing pixel-level cycle consistency between generated images and conditional controls. Specifically, for an input conditional control, we use …

challenges controlnet diffusion diffusion models face feedback image image diffusion images improving machinelearning paper text text-to-image