Dynamically Anchored Prompting for Task-Imbalanced Continual Learning

arXiv:2404.14721v1 Announce Type: new
Abstract: Existing continual learning literature relies heavily on a strong assumption that tasks arrive with a balanced data stream, which is often unrealistic in real-world applications. In this work, we explore task-imbalanced continual learning (TICL) scenarios where the distribution of task data is non-uniform across the whole learning process. We find that imbalanced tasks significantly challenge the capability of models to control the trade-off between stability and plasticity from the perspective of recent prompt-based continual learning …

arxiv continual cs.lg prompting type