Percentile Criterion Optimization in Offline Reinforcement Learning

arXiv:2404.05055v1 Announce Type: new
Abstract: In reinforcement learning, robust policies for high-stakes decision-making problems with limited data are usually computed by optimizing the \emph{percentile criterion}. The percentile criterion is approximately solved by constructing an \emph{ambiguity set} that contains the true model with high probability and optimizing the policy for the worst model in the set. Since the percentile criterion is non-convex, constructing ambiguity sets is often challenging. Existing work uses \emph{Bayesian credible regions} as ambiguity sets, but they are often …

abstract arxiv criterion cs.ai cs.lg data decision making offline optimization policies policy probability reinforcement reinforcement learning robust set true type