Exploiting Estimation Bias in Deep Double Q-Learning for Actor-Critic Methods

arXiv:2402.09078v1 Announce Type: new
Abstract: This paper introduces innovative methods in Reinforcement Learning (RL), focusing on addressing and exploiting estimation biases in Actor-Critic methods for continuous control tasks, using Deep Double Q-Learning. We propose two novel algorithms: Expectile Delayed Deep Deterministic Policy Gradient (ExpD3) and Bias Exploiting - Twin Delayed Deep Deterministic Policy Gradient (BE-TD3). ExpD3 aims to reduce overestimation bias with a single $Q$ estimate, offering a balance between computational efficiency and performance, while BE-TD3 is designed to dynamically …

abstract actor actor-critic algorithms arxiv bias biases continuous control cs.ai cs.lg gradient novel paper policy q-learning reinforcement reinforcement learning tasks twin type