Empirical Risk Minimization with Relative Entropy Regularization

arXiv:2211.06617v4 Announce Type: replace-cross
Abstract: The empirical risk minimization (ERM) problem with relative entropy regularization (ERM-RER) is investigated under the assumption that the reference measure is a $\sigma$-finite measure, and not necessarily a probability measure. Under this assumption, which leads to a generalization of the ERM-RER problem allowing a larger degree of flexibility for incorporating prior knowledge, numerous relevant properties are stated. Among these properties, the solution to this problem, if it exists, is shown to be a unique probability …

abstract arxiv cs.it cs.lg entropy erm flexibility leads math.it math.st probability reference regularization risk stat.th type