ArGue: Attribute-Guided Prompt Tuning for Vision-Language Models

arXiv:2311.16494v2 Announce Type: replace
Abstract: Although soft prompt tuning is effective in efficiently adapting Vision-Language (V&L) models for downstream tasks, it shows limitations in dealing with distribution shifts. We address this issue with Attribute-Guided Prompt Tuning (ArGue), making three key contributions. 1) In contrast to the conventional approach of directly appending soft prompts preceding class names, we align the model with primitive visual attributes generated by Large Language Models (LLMs). We posit that a model's ability to express high confidence …

abstract arxiv contrast cs.cv distribution issue key language language models limitations making prompt prompts prompt tuning shows tasks type vision vision-language models