The Mean-Squared Error of Double Q-Learning. (arXiv:2007.05034v3 [cs.LG] UPDATED)

Web: http://arxiv.org/abs/2007.05034

In this paper, we establish a theoretical comparison between the asymptotic
mean-squared error of Double Q-learning and Q-learning. Our result builds upon
an analysis for linear stochastic approximation based on Lyapunov equations and
applies to both tabular setting and with linear function approximation,
provided that the optimal policy is unique and the algorithms converge. We show
that the asymptotic mean-squared error of Double Q-learning is exactly equal to
that of Q-learning if Double Q-learning uses twice the learning rate of …

arxiv error learning lg mean q-learning