Criterion collapse and loss distribution control

arXiv:2402.09802v1 Announce Type: cross
Abstract: In this work, we consider the notion of "criterion collapse," in which optimization of one metric implies optimality in another, with a particular focus on conditions for collapse into error probability minimizers under a wide variety of learning criteria, ranging from DRO and OCE risks (CVaR, tilted ERM) to non-monotonic criteria underlying recent ascent-descent algorithms explored in the literature (Flooding, SoftAD). We show how collapse in the context of losses with a Bernoulli distribution goes …

abstract arxiv control criterion cs.lg distribution erm error focus loss notion optimization probability risks stat.ml type work