A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models

arXiv:2312.12730v2 Announce Type: replace
Abstract: Efficient transfer learning (ETL) is receiving increasing attention to adapt large pre-trained language-vision models on downstream tasks with a few labeled samples. While significant progress has been made, we reveal that state-of-the-art ETL approaches exhibit strong performance only in narrowly-defined experimental setups, and with a careful adjustment of hyperparameters based on a large corpus of labeled samples. In particular, we make two interesting, and surprising empirical observations. First, to outperform a simple Linear Probing baseline, …

abstract adapt art arxiv attention closer look cs.cv etl experimental few-shot language language models look performance progress samples state tasks transfer transfer learning type vision vision-language models vision models