MyVLM: Personalizing VLMs for User-Specific Queries

arXiv:2403.14599v1 Announce Type: new
Abstract: Recent large-scale vision-language models (VLMs) have demonstrated remarkable capabilities in understanding and generating textual descriptions for visual content. However, these models lack an understanding of user-specific concepts. In this work, we take a first step toward the personalization of VLMs, enabling them to learn and reason over user-provided concepts. For example, we explore whether these models can learn to recognize you in an image and communicate what you are doing, tailoring the model to reflect …

abstract arxiv capabilities concepts cs.cv enabling however language language models learn personalization queries reason scale textual them type understanding vision vision-language models visual vlms work