Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation

arXiv:2404.00417v1 Announce Type: new
Abstract: To accommodate real-world dynamics, artificial intelligence systems need to cope with sequentially arriving content in an online manner. Beyond regular Continual Learning (CL) attempting to address catastrophic forgetting with offline training of each task, Online Continual Learning (OCL) is a more challenging yet realistic setting that performs CL in a one-pass data stream. Current OCL methods primarily rely on memory replay of old training samples. However, a notable gap from CL to OCL stems from …

abstract artificial artificial intelligence arxiv beyond catastrophic forgetting continual cs.ai cs.cv cs.lg distillation dynamics expertise intelligence offline supervision systems training type world