Memory Bounds for Continual Learning. (arXiv:2204.10830v1 [cs.LG])

Continual learning, or lifelong learning, is a formidable current challenge
to machine learning. It requires the learner to solve a sequence of $k$
different learning tasks, one after the other, while retaining its aptitude for
earlier tasks; the continual learner should scale better than the obvious
solution of developing and maintaining a separate learner for each of the $k$
tasks. We embark on a complexity-theoretic study of continual learning in the
PAC framework. We make novel uses of communication complexity …

arxiv continual learning memory