all AI news
Near-Interpolators: Rapid Norm Growth and the Trade-Off between Interpolation and Generalization
March 13, 2024, 4:42 a.m. | Yutong Wang, Rishi Sonthalia, Wei Hu
cs.LG updates on arXiv.org arxiv.org
Abstract: We study the generalization capability of nearly-interpolating linear regressors: $\boldsymbol{\beta}$'s whose training error $\tau$ is positive but small, i.e., below the noise floor. Under a random matrix theoretic assumption on the data distribution and an eigendecay assumption on the data covariance matrix $\boldsymbol{\Sigma}$, we demonstrate that any near-interpolator exhibits rapid norm growth: for $\tau$ fixed, $\boldsymbol{\beta}$ has squared $\ell_2$-norm $\mathbb{E}[\|{\boldsymbol{\beta}}\|_{2}^{2}] = \Omega(n^{\alpha})$ where $n$ is the number of samples and $\alpha >1$ is the exponent …
abstract arxiv beta capability covariance cs.lg data distribution error growth linear matrix near noise norm positive random small stat.ml study trade trade-off training type
More from arxiv.org / cs.LG updates on arXiv.org
Jobs in AI, ML, Big Data
Senior Machine Learning Engineer
@ GPTZero | Toronto, Canada
Werkstudent Data Architecture & Governance (w/m/d)
@ E.ON | Essen, DE
Data Architect, Data Lake, Professional Services
@ Amazon.com | Bogota, DC, COL
Data Architect, Data Lake, Professional Services
@ Amazon.com | Buenos Aires City, Buenos Aires Autonomous City, ARG
Data Architect
@ Bitful | United States - Remote
GCP Data Architect (Presales)
@ Rackspace | India - Remote