Off-Policy Correction For Multi-Agent Reinforcement Learning

arXiv:2111.11229v3 Announce Type: replace
Abstract: Multi-agent reinforcement learning (MARL) provides a framework for problems involving multiple interacting agents. Despite apparent similarity to the single-agent case, multi-agent problems are often harder to train and analyze theoretically. In this work, we propose MA-Trace, a new on-policy actor-critic algorithm, which extends V-Trace to the MARL setting. The key advantage of our algorithm is its high scalability in a multi-worker setting. To this end, MA-Trace utilizes importance sampling as an off-policy correction method, which …

abstract actor actor-critic agent agents algorithm analyze arxiv case cs.ai cs.lg cs.ma framework multi-agent multiple policy reinforcement reinforcement learning train type work