HyperMask: Adaptive Hypernetwork-based Masks for Continual Learning

Artificial neural networks suffer from catastrophic forgetting when they are sequentially trained on multiple tasks. Many continual learning (CL) strategies are trying to overcome this problem. One of the most effective is the hypernetwork-based approach. The hypernetwork generates the weights of a target model based on the task's identity. The model's main limitation is that, in practice, the hypernetwork can produce completely different architectures for subsequent tasks. To solve such a problem, we use the lottery ticket hypothesis, which postulates …

artificial artificial neural networks catastrophic forgetting continual cs.ai cs.lg identity masks multiple networks neural networks strategies tasks