Shape-Guided Diffusion with Inside-Outside Attention

arXiv:2212.00210v3 Announce Type: replace-cross
Abstract: We introduce precise object silhouette as a new form of user control in text-to-image diffusion models, which we dub Shape-Guided Diffusion. Our training-free method uses an Inside-Outside Attention mechanism during the inversion and generation process to apply a shape constraint to the cross- and self-attention maps. Our mechanism designates which spatial region is the object (inside) vs. background (outside) then associates edits to the correct region. We demonstrate the efficacy of our method on the …

arxiv attention cs.ai cs.cv cs.lg diffusion inside type