SELMA: Learning and Merging Skill-Specific Text-to-Image Experts with Auto-Generated Data

arXiv:2403.06952v1 Announce Type: cross
Abstract: Recent text-to-image (T2I) generation models have demonstrated impressive capabilities in creating images from text descriptions. However, these T2I generation models often fall short of generating images that precisely match the details of the text inputs, such as incorrect spatial relationship or missing objects. In this paper, we introduce SELMA: Skill-Specific Expert Learning and Merging with Auto-Generated Data, a novel paradigm to improve the faithfulness of T2I models by fine-tuning models on automatically generated, multi-skill image-text …

abstract arxiv auto capabilities cs.ai cs.cl cs.cv cs.lg data experts generated however image images inputs match merging objects relationship spatial text text-to-image type