Continual Learning on a Diet: Learning from Sparsely Labeled Streams Under Constrained Computation

arXiv:2404.12766v1 Announce Type: new
Abstract: We propose and study a realistic Continual Learning (CL) setting where learning algorithms are granted a restricted computational budget per time step while training. We apply this setting to large-scale semi-supervised Continual Learning scenarios with sparse label rates. Previous proficient CL methods perform very poorly in this challenging setting. Overfitting to the sparse labeled data and insufficient computational budget are the two main culprits for such a poor performance. Our new setting encourages learning methods …

abstract algorithms apply arxiv budget computation computational continual cs.cv cs.lg diet per scale semi-supervised study training type