Selectively Informative Description can Reduce Undesired Embedding Entanglements in Text-to-Image Personalization

arXiv:2403.15330v1 Announce Type: new
Abstract: In text-to-image personalization, a timely and crucial challenge is the tendency of generated images overfitting to the biases present in the reference images. We initiate our study with a comprehensive categorization of the biases into background, nearby-object, tied-object, substance (in style re-contextualization), and pose biases. These biases manifest in the generated images due to their entanglement into the subject embedding. This undesired embedding entanglement not only results in the reflection of biases from the reference …

abstract arxiv biases challenge contextualization cs.cv embedding generated image images object overfitting personalization reduce reference study style text text-to-image type