Double Descent and Overfitting under Noisy Inputs and Distribution Shift for Linear Denoisers

arXiv:2305.17297v3 Announce Type: replace
Abstract: Despite the importance of denoising in modern machine learning and ample empirical work on supervised denoising, its theoretical understanding is still relatively scarce. One concern about studying supervised denoising is that one might not always have noiseless training data from the test distribution. It is more reasonable to have access to noiseless training data from a different dataset than the test dataset. Motivated by this, we study supervised denoising and noisy-input regression under distribution shift. …

abstract arxiv cs.lg data denoising distribution importance inputs linear machine machine learning math.st modern overfitting shift stat.ml stat.th studying test training training data type understanding work