Paint by Inpaint: Learning to Add Image Objects by Removing Them First

arXiv:2404.18212v1 Announce Type: new
Abstract: Image editing has advanced significantly with the introduction of text-conditioned diffusion models. Despite this progress, seamlessly adding objects to images based on textual instructions without requiring user-provided input masks remains a challenge. We address this by leveraging the insight that removing objects (Inpaint) is significantly simpler than its inverse process of adding them (Paint), attributed to the utilization of segmentation mask datasets alongside inpainting models that inpaint within these masks. Capitalizing on this realization, by …

abstract advanced arxiv challenge cs.ai cs.cv diffusion diffusion models editing image images insight introduction masks objects paint progress text textual them type