Better-than-KL PAC-Bayes Bounds

arXiv:2402.09201v1 Announce Type: new
Abstract: Let $f(\theta, X_1),$ $ \dots,$ $ f(\theta, X_n)$ be a sequence of random elements, where $f$ is a fixed scalar function, $X_1, \dots, X_n$ are independent random variables (data), and $\theta$ is a random parameter distributed according to some data-dependent posterior distribution $P_n$. In this paper, we consider the problem of proving concentration inequalities to estimate the mean of the sequence. An example of such a problem is the estimation of the generalization error of …

abstract arxiv bayes cs.lg data distributed distribution function independent paper posterior random stat.ml type variables