Generalization Bounds via Convex Analysis. (arXiv:2202.04985v2 [stat.ML] UPDATED)

Web: http://arxiv.org/abs/2202.04985

Since the celebrated works of Russo and Zou (2016,2019) and Xu and Raginsky
(2017), it has been well known that the generalization error of supervised
learning algorithms can be bounded in terms of the mutual information between
their input and the output, given that the loss of any fixed hypothesis has a
subgaussian tail. In this work, we generalize this result beyond the standard
choice of Shannon's mutual information to measure the dependence between the
input and the output. Our …

analysis arxiv ml