Robust Prediction Error Estimation with Monte-Carlo Methodology. (arXiv:2207.13612v1 [stat.ME])

In this paper, we aim to estimate the prediction error of machine learning
models under the true distribution of the data on hand. We consider the
prediction model as a data-driven black-box function and quantify its
statistical properties using non-parametric methods. We propose a novel
sampling technique that takes advantage of the underlying probability
distribution information embedded in the data. The proposed method combines two
existing frameworks for estimating the prediction inaccuracy error; $m$ out of
$n$ bootstrapping and iterative …

arxiv error methodology monte-carlo prediction