A Lightweight Measure of Classification Difficulty from Application Dataset Characteristics

arXiv:2404.05981v1 Announce Type: new
Abstract: Despite accuracy and computation benchmarks being widely available to help choose among neural network models, these are usually trained on datasets with many classes, and do not give a precise idea of performance for applications of few (< 10) classes. The conventional procedure to predict performance is to train and test repeatedly on the different models and dataset variations of interest. However, this is computationally expensive. We propose an efficient classification difficulty measure that is …

abstract accuracy application applications arxiv benchmarks classification computation cs.cv cs.lg dataset datasets network neural network performance type