Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning

arXiv:2403.18886v1 Announce Type: new
Abstract: Continual learning aims to learn from a stream of continuously arriving data with minimum forgetting of previously learned knowledge. While previous works have explored the effectiveness of leveraging the generalizable knowledge from pre-trained models in continual learning, existing parameter-efficient fine-tuning approaches focus on the use of a predetermined or task-wise set of adapters or prompts. However, these approaches still suffer from forgetting due to task interference on jointly used parameters or restricted flexibility. The reliance …

abstract arxiv continual cs.cv cs.lg data expansion fine-tuning focus knowledge learn pre-trained models type