Statistical Multicriteria Benchmarking via the GSD-Front

arXiv:2406.03924v1 Announce Type: cross
Abstract: Given the vast number of classifiers that have been (and continue to be) proposed, reliable methods for comparing them are becoming increasingly important. The desire for reliability is broken down into three main aspects: (1) Comparisons should allow for different quality metrics simultaneously. (2) Comparisons should take into account the statistical uncertainty induced by the choice of benchmark suite. (3) The robustness of the comparisons under small deviations in the underlying assumptions should be verifiable. …

abstract arxiv benchmarking classifiers cs.lg front metrics quality reliability statistical stat.me stat.ml them type vast via