The Role of Coverage in Online Reinforcement Learning. (arXiv:2210.04157v1 [cs.LG])

Coverage conditions -- which assert that the data logging distribution
adequately covers the state space -- play a fundamental role in determining the
sample complexity of offline reinforcement learning. While such conditions
might seem irrelevant to online reinforcement learning at first glance, we
establish a new connection by showing -- somewhat surprisingly -- that the mere
existence of a data distribution with good coverage can enable sample-efficient
online RL. Concretely, we show that coverability -- that is, existence of a …

arxiv online reinforcement learning reinforcement reinforcement learning role