Faster Last-iterate Convergence of Policy Optimization in Zero-Sum Markov Games. (arXiv:2210.01050v2 [cs.GT] UPDATED)

Multi-Agent Reinforcement Learning (MARL) -- where multiple agents learn to
interact in a shared dynamic environment -- permeates across a wide range of
critical applications. While there has been substantial progress on
understanding the global convergence of policy optimization methods in
single-agent RL, designing and analysis of efficient policy optimization
algorithms in the MARL setting present significant challenges, which
unfortunately, remain highly inadequately addressed by existing theory. In this
paper, we focus on the most basic setting of competitive multi-agent …

arxiv convergence games iterate markov optimization policy