Iterated $Q$-Network: Beyond the One-Step Bellman Operator

arXiv:2403.02107v1 Announce Type: new
Abstract: Value-based Reinforcement Learning (RL) methods rely on the application of the Bellman operator, which needs to be approximated from samples. Most approaches consist of an iterative scheme alternating the application of the Bellman operator and a subsequent projection step onto a considered function space. However, we observe that these algorithms can be improved by considering multiple iterations of the Bellman operator at once. Thus, we introduce iterated $Q$-Networks (iQN), a novel approach that learns a …

abstract application arxiv beyond cs.ai cs.lg function iterative network observe projection reinforcement reinforcement learning samples space type value