More PAC-Bayes bounds: From bounded losses, to losses with general tail behaviors, to anytime-validity

arXiv:2306.12214v3 Announce Type: replace-cross
Abstract: In this paper, we present new high-probability PAC-Bayes bounds for different types of losses. Firstly, for losses with a bounded range, we recover a strengthened version of Catoni's bound that holds uniformly for all parameter values. This leads to new fast rate and mixed rate bounds that are interpretable and tighter than previous bounds in the literature. In particular, the fast rate bound is equivalent to the Seeger--Langford bound. Secondly, for losses with more general …

abstract arxiv bayes cs.lg general leads losses paper probability stat.ml type types values