Provable Policy Gradient Methods for Average-Reward Markov Potential Games

arXiv:2403.05738v1 Announce Type: new
Abstract: We study Markov potential games under the infinite horizon average reward criterion. Most previous studies have been for discounted rewards. We prove that both algorithms based on independent policy gradient and independent natural policy gradient converge globally to a Nash equilibrium for the average reward criterion. To set the stage for gradient-based methods, we first establish that the average reward is a smooth function of policies and provide sensitivity bounds for the differential value functions, …

abstract algorithms arxiv converge criterion cs.gt cs.lg equilibrium games gradient horizon independent markov nash equilibrium natural policy prove studies study type