Reinforcement Learning in Non-Markovian Environments

arXiv:2211.01595v4 Announce Type: replace-cross
Abstract: Motivated by the novel paradigm developed by Van Roy and coauthors for reinforcement learning in arbitrary non-Markovian environments, we propose a related formulation and explicitly pin down the error caused by non-Markovianity of observations when the Q-learning algorithm is applied on this formulation. Based on this observation, we propose that the criterion for agent design should be to seek good approximations for certain conditional laws. Inspired by classical stochastic control, we show that our problem …

abstract algorithm arxiv cs.lg cs.sy eess.sy environments error novel observation paradigm pin q-learning reinforcement reinforcement learning type van