Gradient Correlation Subspace Learning against Catastrophic Forgetting

arXiv:2403.02334v1 Announce Type: new
Abstract: Efficient continual learning techniques have been a topic of significant research over the last few years. A fundamental problem with such learning is severe degradation of performance on previously learned tasks, known also as catastrophic forgetting. This paper introduces a novel method to reduce catastrophic forgetting in the context of incremental class learning called Gradient Correlation Subspace Learning (GCSL). The method detects a subspace of the weights that is least affected by previous tasks and …

arxiv catastrophic forgetting correlation cs.ai cs.cv cs.lg gradient type