Automating Continual Learning

arXiv:2312.00276v2 Announce Type: replace
Abstract: General-purpose learning systems should improve themselves in open-ended fashion in ever-changing environments. Conventional learning algorithms for neural networks, however, suffer from catastrophic forgetting (CF) -- previously acquired skills are forgotten when a new task is learned. Instead of hand-crafting new algorithms for avoiding CF, we propose Automated Continual Learning (ACL) to train self-referential neural networks to meta-learn their own in-context continual (meta-)learning algorithms. ACL encodes all desiderata -- good performance on both old and new …

abstract acl acquired algorithms arxiv automated catastrophic forgetting continual cs.lg environments fashion general however learning systems networks neural networks skills systems train type