Finite-Time Error Analysis of Online Model-Based Q-Learning with a Relaxed Sampling Model

arXiv:2402.11877v1 Announce Type: new
Abstract: Reinforcement learning has witnessed significant advancements, particularly with the emergence of model-based approaches. Among these, $Q$-learning has proven to be a powerful algorithm in model-free settings. However, the extension of $Q$-learning to a model-based framework remains relatively unexplored. In this paper, we delve into the sample complexity of $Q$-learning when integrated with a model-based approach. Through theoretical analyses and empirical evaluations, we seek to elucidate the conditions under which model-based $Q$-learning excels in terms of …

abstract algorithm analysis arxiv cs.ai cs.lg emergence error extension framework free paper q-learning reinforcement reinforcement learning sampling type