Unified ODE Analysis of Smooth Q-Learning Algorithms

arXiv:2404.14442v1 Announce Type: new
Abstract: Convergence of Q-learning has been the focus of extensive research over the past several decades. Recently, an asymptotic convergence analysis for Q-learning was introduced using a switching system framework. This approach applies the so-called ordinary differential equation (ODE) approach to prove the convergence of the asynchronous Q-learning modeled as a continuous-time switching system, where notions from switching system theory are used to prove its asymptotic stability without using explicit Lyapunov arguments. However, to prove stability, …

abstract algorithms analysis arxiv asynchronous convergence cs.ai cs.lg differential differential equation equation focus framework ordinary prove q-learning research type