Linear Convergence of Independent Natural Policy Gradient in Games with Entropy Regularization

arXiv:2405.02769v1 Announce Type: new
Abstract: This work focuses on the entropy-regularized independent natural policy gradient (NPG) algorithm in multi-agent reinforcement learning. In this work, agents are assumed to have access to an oracle with exact policy evaluation and seek to maximize their respective independent rewards. Each individual's reward is assumed to depend on the actions of all the agents in the multi-agent system, leading to a game between agents. We assume all agents make decisions under a policy with bounded …

abstract access agent agents algorithm arxiv convergence cs.lg cs.ma entropy evaluation games gradient independent linear math.oc multi-agent natural oracle policy regularization reinforcement reinforcement learning seek type work