Symmetric Q-learning: Reducing Skewness of Bellman Error in Online Reinforcement Learning

arXiv:2403.07704v1 Announce Type: new
Abstract: In deep reinforcement learning, estimating the value function to evaluate the quality of states and actions is essential. The value function is often trained using the least squares method, which implicitly assumes a Gaussian error distribution. However, a recent study suggested that the error distribution for training the value function is often skewed because of the properties of the Bellman operator, and violates the implicit assumption of normal error distribution in the least squares method. …

abstract arxiv cs.ai cs.lg distribution error function however least online reinforcement learning q-learning quality reinforcement reinforcement learning skewness squares study type value