Training-Free Unsupervised Prompt for Vision-Language Models

arXiv:2404.16339v1 Announce Type: new
Abstract: Prompt learning has become the most effective paradigm for adapting large pre-trained vision-language models (VLMs) to downstream tasks. Recently, unsupervised prompt tuning methods, such as UPL and POUF, directly leverage pseudo-labels as supervisory information to fine-tune additional adaptation modules on unlabeled data. However, inaccurate pseudo labels easily misguide the tuning process and result in poor representation capabilities. In light of this, we propose Training-Free Unsupervised Prompts (TFUP), which maximally preserves the inherent representation capabilities and …

abstract arxiv become cs.ai cs.cv data free however information labels language language models modules paradigm prompt prompt learning prompt tuning tasks training type unsupervised vision vision-language vision-language models vlms