InstructAny2Pix: Flexible Visual Editing via Multimodal Instruction Following

arXiv:2312.06738v3 Announce Type: replace
Abstract: The ability to provide fine-grained control for generating and editing visual imagery has profound implications for computer vision and its applications. Previous works have explored extending controllability in two directions: instruction tuning with text-based prompts and multi-modal conditioning. However, these works make one or more unnatural assumptions on the number and/or type of modality inputs used to express controllability. We propose InstructAny2Pix, a flexible multi-modal instruction-following system that enables users to edit an input image …

arxiv cs.cv editing multimodal type via visual