Prompt Tuning with Soft Context Sharing for Vision-Language Models

arXiv:2208.13474v2 Announce Type: replace
Abstract: Vision-language models have recently shown great potential on many tasks in computer vision. Meanwhile, prior work demonstrates prompt tuning designed for vision-language models could acquire superior performance on few-shot image recognition compared to linear probe, a strong baseline. In practice, many few-shot tasks are inherently correlated, particularly within specialized domains. However, such information is overlooked previously. Inspired by the fact that modeling task relationship by multi-task learning can usually boost performance, we propose a novel …

arxiv context cs.ai cs.cv language language models prompt prompt tuning type vision vision-language models