Koopman-Assisted Reinforcement Learning

arXiv:2403.02290v1 Announce Type: cross
Abstract: The Bellman equation and its continuous form, the Hamilton-Jacobi-Bellman (HJB) equation, are ubiquitous in reinforcement learning (RL) and control theory. However, these equations quickly become intractable for systems with high-dimensional states and nonlinearity. This paper explores the connection between the data-driven Koopman operator and Markov Decision Processes (MDPs), resulting in the development of two new RL algorithms to address these limitations. We leverage Koopman operator techniques to lift a nonlinear system into new coordinates where …

abstract arxiv become continuous control cs.ai cs.lg data data-driven decision development equation form hamilton markov math.ds math.oc paper processes reinforcement reinforcement learning systems theory type