MM-Diff: High-Fidelity Image Personalization via Multi-Modal Condition Integration

arXiv:2403.15059v1 Announce Type: new
Abstract: Recent advances in tuning-free personalized image generation based on diffusion models are impressive. However, to improve subject fidelity, existing methods either retrain the diffusion model or infuse it with dense visual embeddings, both of which suffer from poor generalization and efficiency. Also, these methods falter in multi-subject image generation due to the unconstrained cross-attention mechanism. In this paper, we propose MM-Diff, a unified and tuning-free image personalization framework capable of generating high-fidelity images of both …

abstract advances arxiv cs.ai cs.cv diff diffusion diffusion model diffusion models efficiency embeddings fidelity free however image image generation integration modal multi-modal personalization personalized type via visual