Beyond Test Accuracies for Studying Deep Neural Networks

Abstract: Already in 2015, Leon Bottou discussed the prevalence and end of the training/test experimental paradigm in machine learning. The machine learning community has however continued to stick to this paradigm until now (2023), relying almost entirely and exclusively on the test-set accuracy, which is a rough proxy to the true quality of a machine learning system we want to measure. There are however many aspects in building a machine learning system that require more attention. Specifically, I will discuss …

abstract accuracy beyond community experimental machine machine learning networks neural networks paradigm quality set studying test training true