Cross-Modal Adapter: Parameter-Efficient Transfer Learning Approach for Vision-Language Models

arXiv:2404.12588v1 Announce Type: cross
Abstract: Adapter-based parameter-efficient transfer learning has achieved exciting results in vision-language models. Traditional adapter methods often require training or fine-tuning, facing challenges such as insufficient samples or resource limitations. While some methods overcome the need for training by leveraging image modality cache and retrieval, they overlook the text modality's importance and cross-modal cues for the efficient adaptation of parameters in visual-language models. This work introduces a cross-modal parameter-efficient approach named XMAdapter. XMAdapter establishes cache models for …

abstract adapter arxiv cache challenges cs.cv cs.lg fine-tuning image language language models limitations modal results retrieval samples training transfer transfer learning type vision vision-language vision-language models