Training Data Influence Analysis and Estimation: A Survey

arXiv:2212.04612v3 Announce Type: replace
Abstract: Good models require good training data. For overparameterized deep models, the causal relationship between training data and model predictions is increasingly opaque and poorly understood. Influence analysis partially demystifies training's underlying interactions by quantifying the amount each training instance alters the final model. Measuring the training data's influence exactly can be provably hard in the worst case; this has led to the development and use of influence estimators, which only approximate the true influence. This …

analysis arxiv cs.lg data influence survey training training data type