Making RL with Preference-based Feedback Efficient via Randomization

arXiv:2310.14554v2 Announce Type: replace
Abstract: Reinforcement Learning algorithms that learn from human feedback (RLHF) need to be efficient in terms of statistical complexity, computational complexity, and query complexity. In this work, we consider the RLHF setting where the feedback is given in the format of preferences over pairs of trajectories. In the linear MDP model, using randomization in algorithm design, we present an algorithm that is sample efficient (i.e., has near-optimal worst-case regret bounds) and has polynomial running time (i.e., …

abstract algorithms arxiv complexity computational cs.ai cs.hc cs.lg feedback format human human feedback learn making query randomization reinforcement reinforcement learning rlhf statistical terms type via work