CAT: Contrastive Adapter Training for Personalized Image Generation

arXiv:2404.07554v1 Announce Type: new
Abstract: The emergence of various adapters, including Low-Rank Adaptation (LoRA) applied from the field of natural language processing, has allowed diffusion models to personalize image generation at a low cost. However, due to the various challenges including limited datasets and shortage of regularization and computation resources, adapter training often results in unsatisfactory outcomes, leading to the corruption of the backbone model's prior knowledge. One of the well known phenomena is the loss of diversity in object …

abstract adapter arxiv challenges computation cost cs.ai cs.cv datasets diffusion diffusion models emergence however image image generation language language processing lora low low-rank adaptation natural natural language natural language processing personalized processing regularization resources shortage training type