Minimum discrepancy principle strategy for choosing $k$ in $k$-NN regression

arXiv:2008.08718v5 Announce Type: replace-cross
Abstract: We present a novel data-driven strategy to choose the hyperparameter $k$ in the $k$-NN regression estimator without using any hold-out data. We treat the problem of choosing the hyperparameter as an iterative procedure (over $k$) and propose using an easily implemented in practice strategy based on the idea of early stopping and the minimum discrepancy principle. This model selection strategy is proven to be minimax-optimal, under the fixed-design assumption on covariates, over some smoothness function …

abstract arxiv cs.lg data data-driven estimator hyperparameter iterative math.st minimum novel practice regression stat.ml stat.th strategy type