Distal Interference: Exploring the Limits of Model-Based Continual Learning

Continual learning is the sequential learning of different tasks by a machine learning model. Continual learning is known to be hindered by catastrophic interference or forgetting, i.e. rapid unlearning of earlier learned tasks when new tasks are learned. Despite their practical success, artificial neural networks (ANNs) are prone to catastrophic interference. This study analyses how gradient descent and overlapping representations between distant input points lead to distal interference and catastrophic interference. Distal interference refers to the phenomenon where training a …

anns artificial artificial neural networks continual cs.ai cs.lg cs.ne interference machine machine learning machine learning model networks neural networks practical success tasks unlearning