Sample Complexity of Preference-Based Nonparametric Off-Policy Evaluation with Deep Networks

arXiv:2310.10556v2 Announce Type: replace
Abstract: A recently popular approach to solving reinforcement learning is with data from human preferences. In fact, human preference data are now used with classic reinforcement learning algorithms such as actor-critic methods, which involve evaluating an intermediate policy over a reward learned from human preference data with distribution shift, known as off-policy evaluation (OPE). Such algorithm includes (i) learning reward function from human preference dataset, and (ii) learning expected cumulative reward of a target policy. Despite …

abstract actor actor-critic algorithms arxiv complexity cs.lg data evaluation human intermediate networks policy popular reinforcement reinforcement learning sample stat.ml type