Bayesian Inference for Consistent Predictions in Overparameterized Nonlinear Regression

arXiv:2404.04498v1 Announce Type: cross
Abstract: The remarkable generalization performance of overparameterized models has challenged the conventional wisdom of statistical learning theory. While recent theoretical studies have shed light on this behavior in linear models or nonlinear classifiers, a comprehensive understanding of overparameterization in nonlinear regression remains lacking. This paper explores the predictive properties of overparameterized nonlinear regression within the Bayesian framework, extending the methodology of adaptive prior based on the intrinsic spectral structure of the data. We establish posterior contraction …

abstract arxiv bayesian bayesian inference behavior classifiers consistent cs.lg inference light linear paper performance predictions regression statistical stat.me stat.ml studies theory type understanding