Data-Efficient Learning via Clustering-Based Sensitivity Sampling: Foundation Models and Beyond

arXiv:2402.17327v1 Announce Type: new
Abstract: We study the data selection problem, whose aim is to select a small representative subset of data that can be used to efficiently train a machine learning model. We present a new data selection approach based on $k$-means clustering and sensitivity sampling. Assuming access to an embedding representation of the data with respect to which the model loss is H\"older continuous, our approach provably allows selecting a set of ``typical'' $k + 1/\varepsilon^2$ elements whose …

abstract aim arxiv beyond clustering cs.ds cs.lg data foundation machine machine learning machine learning model sampling sensitivity small study train type via