Analyzing Data Selection Techniques with Tools from the Theory of Information Losses. (arXiv:1902.09602v4 [cs.LG] UPDATED)

In this paper, we present and illustrate some new tools for rigorously
analyzing training data selection methods. These tools focus on the information
theoretic losses that occur when sampling data. We use this framework to prove
that two methods, Facility Location Selection and Transductive Experimental
Design, reduce these losses. These are meant to act as generalizable
theoretical examples of applying the field of Information Theoretic Deep
Learning Theory to the fields of data selection and active learning. Both
analyses yield …

arxiv data information theory tools