Sharp analysis of out-of-distribution error for "importance-weighted" estimators in the overparameterized regime

arXiv:2405.06546v1 Announce Type: cross
Abstract: Overparameterized models that achieve zero training error are observed to generalize well on average, but degrade in performance when faced with data that is under-represented in the training sample. In this work, we study an overparameterized Gaussian mixture model imbued with a spurious feature, and sharply analyze the in-distribution and out-of-distribution test error of a cost-sensitive interpolating solution that incorporates "importance weights". Compared to recent work Wang et al. (2021), Behnia et al. (2022), our …

abstract analysis arxiv cs.it cs.lg data distribution error importance math.it performance sample stat.ml study training type work