Diffusion-driven GAN Inversion for Multi-Modal Face Image Generation

arXiv:2405.04356v1 Announce Type: new
Abstract: We present a new multi-modal face image generation method that converts a text prompt and a visual input, such as a semantic mask or scribble map, into a photo-realistic face image. To do this, we combine the strengths of Generative Adversarial networks (GANs) and diffusion models (DMs) by employing the multi-modal features in the DM into the latent space of the pre-trained GANs. We present a simple mapping and a style modulation network to link …

arxiv cs.cv diffusion face gan image image generation modal multi-modal type