Suppressing Overestimation in Q-Learning through Adversarial Behaviors

arXiv:2310.06286v2 Announce Type: replace
Abstract: The goal of this paper is to propose a new Q-learning algorithm with a dummy adversarial player, which is called dummy adversarial Q-learning (DAQ), that can effectively regulate the overestimation bias in standard Q-learning. With the dummy player, the learning can be formulated as a two-player zero-sum game. The proposed DAQ unifies several Q-learning variations to control overestimation biases, such as maxmin Q-learning and minmax Q-learning (proposed in this paper) in a single framework. The …

abstract adversarial algorithm arxiv bias cs.ai cs.lg game paper q-learning standard through type zero-sum game