Online Estimation with Rolling Validation: Adaptive Nonparametric Estimation with Streaming Data

arXiv:2310.12140v2 Announce Type: replace-cross
Abstract: Online nonparametric estimators are gaining popularity due to their efficient computation and competitive generalization abilities. An important example includes variants of stochastic gradient descent. These algorithms often take one sample point at a time and instantly update the parameter estimate of interest. In this work we consider model selection and hyperparameter tuning for such online algorithms. We propose a weighted rolling-validation procedure, an online variant of leave-one-out cross-validation, that costs minimal extra computation for many …

abstract algorithms arxiv computation data example gradient math.st sample stat.me stat.ml stat.th stochastic streaming streaming data type update validation variants