PromptKD: Unsupervised Prompt Distillation for Vision-Language Models

arXiv:2403.02781v1 Announce Type: new
Abstract: Prompt learning has emerged as a valuable technique in enhancing vision-language models (VLMs) such as CLIP for downstream tasks in specific domains. Existing work mainly focuses on designing various learning forms of prompts, neglecting the potential of prompts as effective distillers for learning from larger teacher models. In this paper, we introduce an unsupervised domain prompt distillation framework, which aims to transfer the knowledge of a larger teacher model to a lightweight target model through …

abstract arxiv clip cs.cv designing distillation domains forms language language models prompt prompt learning prompts tasks type unsupervised vision vision-language models vlms work