Near-Optimal Reinforcement Learning with Self-Play under Adaptivity Constraints

We study the problem of multi-agent reinforcement learning (MARL) with adaptivity constraints -- a new problem motivated by real-world applications where deployments of new policies are costly and the number of policy updates must be minimized. For two-player zero-sum Markov Games, we design a (policy) elimination based algorithm that achieves a regret of $\widetilde{O}(\sqrt{H^3 S^2 ABK})$, while the batch complexity is only $O(H+\log\log K)$. In the above, $S$ denotes the number of states, $A,B$ are the number of actions for …

agent algorithm applications constraints cs.ai cs.lg cs.ma deployments design games markov multi-agent near policy reinforcement reinforcement learning self-play stat.ml study updates world