Off-Policy Risk Assessment in Markov Decision Processes. (arXiv:2209.10444v1 [cs.LG])

Addressing such diverse ends as safety alignment with human preferences, and
the efficiency of learning, a growing line of reinforcement learning research
focuses on risk functionals that depend on the entire distribution of returns.
Recent work on \emph{off-policy risk assessment} (OPRA) for contextual bandits
introduced consistent estimators for the target policy's CDF of returns along
with finite sample guarantees that extend to (and hold simultaneously over) all
risk. In this paper, we lift OPRA to Markov decision processes (MDPs), where …

arxiv decision markov policy processes risk risk assessment