Exploration is Harder than Prediction: Cryptographically Separating Reinforcement Learning from Supervised Learning

arXiv:2404.03774v1 Announce Type: new
Abstract: Supervised learning is often computationally easy in practice. But to what extent does this mean that other modes of learning, such as reinforcement learning (RL), ought to be computationally easy by extension? In this work we show the first cryptographic separation between RL and supervised learning, by exhibiting a class of block MDPs and associated decoding functions where reward-free exploration is provably computationally harder than the associated regression problem. We also show that there is …

abstract arxiv cs.cc cs.cr cs.ds cs.lg easy exploration extension mean practice prediction reinforcement reinforcement learning show supervised learning type work