Partial Rankings of Optimizers

arXiv:2402.16565v1 Announce Type: cross
Abstract: We introduce a framework for benchmarking optimizers according to multiple criteria over various test functions. Based on a recently introduced union-free generic depth function for partial orders/rankings, it fully exploits the ordinal information and allows for incomparability. Our method describes the distribution of all partial orders/rankings, avoiding the notorious shortcomings of aggregation. This permits to identify test functions that produce central or outlying rankings of optimizers and to assess the quality of benchmarking suites.

abstract arxiv benchmarking cs.lg distribution exploits framework free function functions information multiple orders ordinal rankings stat.ml test type union