Learning Equi-angular Representations for Online Continual Learning

arXiv:2404.01628v1 Announce Type: cross
Abstract: Online continual learning suffers from an underfitted solution due to insufficient training for prompt model update (e.g., single-epoch training). To address the challenge, we propose an efficient online continual learning method using the neural collapse phenomenon. In particular, we induce neural collapse to form a simplex equiangular tight frame (ETF) structure in the representation space so that the continuously learned model with a single epoch can better fit to the streamed data by proposing preparatory …

abstract angular arxiv challenge continual cs.ai cs.cv cs.lg form neural collapse prompt solution training type update