Soft Prompt Generation for Domain Generalization

arXiv:2404.19286v1 Announce Type: new
Abstract: Large pre-trained vision language models (VLMs) have shown impressive zero-shot ability on downstream tasks with manually designed prompt, which are not optimal for specific domains. To further adapt VLMs to downstream tasks, soft prompt is proposed to replace manually designed prompt, which acts as a learning vector that undergoes fine-tuning based on specific domain data. Prior prompt learning methods primarily learn a fixed prompt and residuled prompt from training samples. However, the learned prompts lack …

abstract adapt arxiv cs.cv domain domains language language models prompt tasks type vector vision vlms zero-shot