Continuous-time q-learning for mean-field control problems

arXiv:2306.16208v3 Announce Type: replace
Abstract: This paper studies the q-learning, recently coined as the continuous time counterpart of Q-learning by Jia and Zhou (2023), for continuous time Mckean-Vlasov control problems in the setting of entropy-regularized reinforcement learning. In contrast to the single agent's control problem in Jia and Zhou (2023), the mean-field interaction of agents renders the definition of the q-function more subtle, for which we reveal that two distinct q-functions naturally arise: (i) the integrated q-function (denoted by $q$) …

abstract agent arxiv continuous contrast control cs.lg entropy math.oc mean paper q-fin.cp q-learning reinforcement reinforcement learning studies type zhou