Semantic Residual Prompts for Continual Learning

arXiv:2403.06870v1 Announce Type: new
Abstract: Prompt-tuning methods for Continual Learning (CL) freeze a large pre-trained model and focus training on a few parameter vectors termed prompts. Most of these methods organize these vectors in a pool of key-value pairs, and use the input image as query to retrieve the prompts (values). However, as keys are learned while tasks progress, the prompting selection strategy is itself subject to catastrophic forgetting, an issue often overlooked by existing approaches. For instance, prompts introduced …

abstract arxiv continual cs.lg focus however image key keys organize pool prompt prompts query residual semantic training type value values vectors