Continuity of Generalized Entropy and Statistical Learning. (arXiv:2012.15829v2 [cs.LG] UPDATED)

We study the continuity property of the generalized entropy as a function of
the underlying probability distribution, defined with an action space and a
loss function, and use this property to answer the basic questions in
statistical learning theory: the excess risk analyses for various learning
methods. We first derive upper and lower bounds for the entropy difference of
two distributions in terms of several commonly used f-divergences, the
Wasserstein distance, a distance that depends on the action space and …

arxiv entropy learning statistical