Data Selection: A General Principle for Building Small Interpretable Models

arXiv:2210.03921v3 Announce Type: replace
Abstract: We present convincing empirical evidence for an effective and general strategy for building accurate small models. Such models are attractive for interpretability and also find use in resource-constrained environments. The strategy is to learn the training distribution and sample accordingly from the provided training data. The distribution learning algorithm is not a contribution of this work; our contribution is a rigorous demonstration of the broad utility of this strategy in various practical settings. We apply …

abstract arxiv building cs.lg data distribution environments evidence general interpretability learn sample small strategy training training data type