Integral Probability Metrics PAC-Bayes Bounds. (arXiv:2207.00614v6 [stat.ML] UPDATED)

We present a PAC-Bayes-style generalization bound which enables the
replacement of the KL-divergence with a variety of Integral Probability Metrics
(IPM). We provide instances of this bound with the IPM being the total
variation metric and the Wasserstein distance. A notable feature of the
obtained bounds is that they naturally interpolate between classical uniform
convergence bounds in the worst case (when the prior and posterior are far away
from each other), and improved bounds in favorable cases (when the posterior …

arxiv bayes metrics probability