DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models

arXiv:2310.00902v2 Announce Type: replace
Abstract: Quantifying the impact of training data points is crucial for understanding the outputs of machine learning models and for improving the transparency of the AI pipeline. The influence function is a principled and popular data attribution method, but its computational cost often makes it challenging to use. This issue becomes more pronounced in the setting of large language models and text-to-image models. In this work, we propose DataInf, an efficient influence approximation method that is …

abstract arxiv attribution computational cost cs.lg data diffusion diffusion models function impact influence llms lora machine machine learning machine learning models pipeline popular stat.ml training training data transparency type understanding