Controlling Multiple Errors Simultaneously with a PAC-Bayes Bound

arXiv:2202.05560v2 Announce Type: replace-cross
Abstract: Current PAC-Bayes generalisation bounds are restricted to scalar metrics of performance, such as the loss or error rate. However, one ideally wants more information-rich certificates that control the entire distribution of possible outcomes, such as the distribution of the test loss in regression, or the probabilities of different mis classifications. We provide the first PAC-Bayes bound capable of providing such rich information by bounding the Kullback-Leibler divergence between the empirical and true probabilities of a …

abstract arxiv bayes control cs.lg current distribution error errors information loss math.st metrics multiple performance rate regression stat.ml stat.th test type