Minimum-Norm Interpolation Under Covariate Shift

arXiv:2404.00522v1 Announce Type: new
Abstract: Transfer learning is a critical part of real-world machine learning deployments and has been extensively studied in experimental works with overparameterized neural networks. However, even in the simplest setting of linear regression a notable gap still exists in the theoretical understanding of transfer learning. In-distribution research on high-dimensional linear regression has led to the identification of a phenomenon known as \textit{benign overfitting}, in which linear interpolators overfit to noisy training labels and yet still generalize …

abstract arxiv cs.lg deployments distribution experimental gap however linear linear regression machine machine learning networks neural networks norm part regression research shift stat.ml transfer transfer learning type understanding world