Limits to classification performance by relating Kullback-Leibler divergence to Cohen's Kappa

arXiv:2403.01571v1 Announce Type: cross
Abstract: The performance of machine learning classification algorithms are evaluated by estimating metrics, often from the confusion matrix, using training data and cross-validation. However, these do not prove that the best possible performance has been achieved. Fundamental limits to error rates can be estimated using information distance measures. To this end, the confusion matrix has been formulated to comply with the Chernoff-Stein Lemma. This links the error rates to the Kullback-Leibler divergences between the probability density …

abstract algorithms arxiv classification cohen cs.lg data divergence error machine machine learning matrix metrics performance physics.data-an prove stat.ml training training data type validation