Nearly Minimax Optimal Reinforcement Learning for Discounted MDPs. (arXiv:2010.00587v3 [cs.LG] UPDATED)

We study the reinforcement learning problem for discounted Markov Decision
Processes (MDPs) under the tabular setting. We propose a model-based algorithm
named UCBVI-$\gamma$, which is based on the \emph{optimism in the face of
uncertainty principle} and the Bernstein-type bonus. We show that
UCBVI-$\gamma$ achieves an $\tilde{O}\big({\sqrt{SAT}}/{(1-\gamma)^{1.5}}\big)$
regret, where $S$ is the number of states, $A$ is the number of actions,
$\gamma$ is the discount factor and $T$ is the number of steps. In addition, we
construct a class of hard …

arxiv learning minimax reinforcement learning