Refined Sample Complexity for Markov Games with Independent Linear Function Approximation

Markov Games (MG) is an important model for Multi-Agent Reinforcement Learning (MARL). It was long believed that the "curse of multi-agents" (i.e., the algorithmic performance drops exponentially with the number of agents) is unavoidable until several recent works (Daskalakis et al., 2023; Cui et al., 2023; Wang et al., 2023. While these works did resolve the curse of multi-agents, when the state spaces are prohibitively large and (linear) function approximations are deployed, they either had a slower convergence rate of …

agent agents approximation complexity cs.gt cs.lg function games independent linear markov multi-agent multi-agents performance reinforcement reinforcement learning sample stat.ml