Continuous-time Risk-sensitive Reinforcement Learning via Quadratic Variation Penalty

arXiv:2404.12598v1 Announce Type: new
Abstract: This paper studies continuous-time risk-sensitive reinforcement learning (RL) under the entropy-regularized, exploratory diffusion process formulation with the exponential-form objective. The risk-sensitive objective arises either as the agent's risk attitude or as a distributionally robust approach against the model uncertainty. Owing to the martingale perspective in Jia and Zhou (2023) the risk-sensitive RL problem is shown to be equivalent to ensuring the martingale property of a process involving both the value function and the q-function, augmented …

abstract agent arxiv attitude continuous cs.lg cs.sy diffusion eess.sy entropy exploratory form paper perspective process q-fin.cp q-fin.pm reinforcement reinforcement learning risk robust studies type uncertainty variation via