Adaptive Hyperparameter Optimization for Continual Learning Scenarios

arXiv:2403.07015v1 Announce Type: new
Abstract: Hyperparameter selection in continual learning scenarios is a challenging and underexplored aspect, especially in practical non-stationary environments. Traditional approaches, such as grid searches with held-out validation data from all tasks, are unrealistic for building accurate lifelong learning systems. This paper aims to explore the role of hyperparameter selection in continual learning and the necessity of continually and automatically tuning them according to the complexity of the task at hand. Hence, we propose leveraging the nature …

abstract arxiv building continual cs.lg data environments explore grid hyperparameter learning systems lifelong learning optimization paper practical role systems tasks type validation