Hyperparameters in Continual Learning: a Reality Check

arXiv:2403.09066v1 Announce Type: new
Abstract: Various algorithms for continual learning (CL) have been designed with the goal of effectively alleviating the trade-off between stability and plasticity during the CL process. To achieve this goal, tuning appropriate hyperparameters for each algorithm is essential. As an evaluation protocol, it has been common practice to train a CL algorithm using diverse hyperparameter values on a CL scenario constructed with a benchmark dataset. Subsequently, the best performance attained with the optimal hyperparameter value serves …

abstract algorithm algorithms arxiv check continual cs.cv cs.lg evaluation practice process protocol reality stability trade trade-off train type