Hyperparameter Selection in Continual Learning

arXiv:2404.06466v1 Announce Type: new
Abstract: In continual learning (CL) -- where a learner trains on a stream of data -- standard hyperparameter optimisation (HPO) cannot be applied, as a learner does not have access to all of the data at the same time. This has prompted the development of CL-specific HPO frameworks. The most popular way to tune hyperparameters in CL is to repeatedly train over the whole data stream with different hyperparameter settings. However, this end-of-training HPO is unrealistic …

abstract arxiv continual cs.lg data development frameworks hyperparameter optimisation popular standard stat.ml trains type