Fisher-Rao Gradient Flows of Linear Programs and State-Action Natural Policy Gradients

arXiv:2403.19448v1 Announce Type: cross
Abstract: Kakade's natural policy gradient method has been studied extensively in the last years showing linear convergence with and without regularization. We study another natural gradient method which is based on the Fisher information matrix of the state-action distributions and has received little attention from the theoretical side. Here, the state-action distributions follow the Fisher-Rao gradient flow inside the state-action polytope with respect to a linear potential. Therefore, we study Fisher-Rao gradient flows of linear programs …

abstract arxiv attention convergence cs.lg cs.na cs.sy eess.sy fisher gradient information linear math.na math.oc matrix natural policy regularization state stat.ml study type