Offline Reinforcement Learning: Role of State Aggregation and Trajectory Data

arXiv:2403.17091v1 Announce Type: new
Abstract: We revisit the problem of offline reinforcement learning with value function realizability but without Bellman completeness. Previous work by Xie and Jiang (2021) and Foster et al. (2022) left open the question whether a bounded concentrability coefficient along with trajectory-based offline data admits a polynomial sample complexity. In this work, we provide a negative answer to this question for the task of offline policy evaluation. In addition to addressing this question, we provide a rather …

abstract aggregation arxiv cs.ai cs.lg data function offline polynomial question reinforcement reinforcement learning role state stat.ml trajectory type value work