Generalization Bounds: Perspectives from Information Theory and PAC-Bayes

arXiv:2309.04381v2 Announce Type: replace
Abstract: A fundamental question in theoretical machine learning is generalization. Over the past decades, the PAC-Bayesian approach has been established as a flexible framework to address the generalization capabilities of machine learning algorithms, and design new ones. Recently, it has garnered increased interest due to its potential applicability for a variety of learning algorithms, including deep neural networks. In parallel, an information-theoretic view of generalization has developed, wherein the relation between generalization and various information measures …

abstract algorithms arxiv bayes bayesian capabilities cs.ai cs.it cs.lg design framework information machine machine learning machine learning algorithms math.it math.st perspectives question stat.ml stat.th theory type