Robust performance metrics for imbalanced classification problems

arXiv:2404.07661v1 Announce Type: cross
Abstract: We show that established performance metrics in binary classification, such as the F-score, the Jaccard similarity coefficient or Matthews' correlation coefficient (MCC), are not robust to class imbalance in the sense that if the proportion of the minority class tends to $0$, the true positive rate (TPR) of the Bayes classifier under these metrics tends to $0$ as well. Thus, in imbalanced classification problems, these metrics favour classifiers which ignore the minority class. To alleviate …

abstract arxiv binary class classification correlation cs.lg jaccard metrics performance positive rate robust sense show stat.me stat.ml true type