DM-Align: Leveraging the Power of Natural Language Instructions to Make Changes to Images

arXiv:2404.18020v1 Announce Type: new
Abstract: Text-based semantic image editing assumes the manipulation of an image using a natural language instruction. Although recent works are capable of generating creative and qualitative images, the problem is still mostly approached as a black box sensitive to generating unexpected outputs. Therefore, we propose a novel model to enhance the text-based control of an image editor by explicitly reasoning about which parts of the image to alter or preserve. It relies on word alignments between …

abstract arxiv black box box creative cs.cv editing image images language manipulation natural natural language power semantic text type