Provably Efficient Exploration in Policy Optimization

arXiv:1912.05830v4 Announce Type: replace
Abstract: While policy-based reinforcement learning (RL) achieves tremendous successes in practice, it is significantly less understood in theory, especially compared with value-based RL. In particular, it remains elusive how to design a provably efficient policy optimization algorithm that incorporates exploration. To bridge such a gap, this paper proposes an Optimistic variant of the Proximal Policy Optimization algorithm (OPPO), which follows an ``optimistic version'' of the policy gradient direction. This paper proves that, in the problem of …

abstract algorithm arxiv bridge cs.lg design exploration gap math.oc optimization paper policy practice reinforcement reinforcement learning stat.ml theory type value