Data-free Multi-label Image Recognition via LLM-powered Prompt Tuning

arXiv:2403.01209v1 Announce Type: new
Abstract: This paper proposes a novel framework for multi-label image recognition without any training data, called data-free framework, which uses knowledge of pre-trained Large Language Model (LLM) to learn prompts to adapt pretrained Vision-Language Model (VLM) like CLIP to multilabel classification. Through asking LLM by well-designed questions, we acquire comprehensive knowledge about characteristics and contexts of objects, which provides valuable text descriptions for learning prompts. Then we propose a hierarchical prompt learning method by taking the …

abstract adapt arxiv classification clip cs.cv data framework free image image recognition knowledge language language model large language large language model learn learn prompts llm novel paper prompt prompts prompt tuning questions recognition through training training data type via vision vlm