Offline Policy Evaluation for Reinforcement Learning with Adaptively Collected Data

arXiv:2306.14063v2 Announce Type: replace
Abstract: Developing theoretical guarantees on the sample complexity of offline RL methods is an important step towards making data-hungry RL algorithms practically viable. Currently, most results hinge on unrealistic assumptions about the data distribution -- namely that it comprises a set of i.i.d. trajectories collected by a single logging policy. We consider a more general setting where the dataset may have been gathered adaptively. We develop theory for the TMIS Offline Policy Evaluation (OPE) estimator in …

abstract algorithms arxiv assumptions complexity cs.ai cs.lg data distribution evaluation hinge making offline policy reinforcement reinforcement learning results sample set type