Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EM. (arXiv:2206.08873v1 [math.OC])

Web: http://arxiv.org/abs/2206.08873

Many problems in machine learning can be formulated as optimizing a convex
functional over a space of measures. This paper studies the convergence of the
mirror descent algorithm in this infinite-dimensional setting. Defining Bregman
divergences through directional derivatives, we derive the convergence of the
scheme for relatively smooth and strongly convex pairs of functionals. Applying
our result to joint distributions and the Kullback--Leibler (KL) divergence, we
show that Sinkhorn's primal iterations for entropic optimal transport in the
continuous setting correspond …

application arxiv math