MoEController: Instruction-based Arbitrary Image Manipulation with Mixture-of-Expert Controllers

arXiv:2309.04372v2 Announce Type: replace
Abstract: Diffusion-model-based text-guided image generation has recently made astounding progress, producing fascinating results in open-domain image manipulation tasks. Few models, however, currently have complete zero-shot capabilities for both global and local image editing due to the complexity and diversity of image manipulation tasks. In this work, we propose a method with a mixture-of-expert (MOE) controllers to align the text-guided capacity of diffusion models with different kinds of human instructions, enabling our model to handle various open-domain …

arxiv cs.cl cs.cv expert image manipulation type