Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters

arXiv:2403.11549v1 Announce Type: new
Abstract: Continual learning can empower vision-language models to continuously acquire new knowledge, without the need for access to the entire historical dataset. However, mitigating the performance degradation in large-scale models is non-trivial due to (i) parameter shifts throughout lifelong learning and (ii) significant computational burdens associated with full-model tuning. In this work, we present a parameter-efficient continual learning framework to alleviate long-term forgetting in incremental learning with vision-language models. Our approach involves the dynamic expansion of …

arxiv boosting continual cs.cv experts language language models type via vision vision-language models