Imitation Bootstrapped Reinforcement Learning

arXiv:2311.02198v4 Announce Type: replace
Abstract: Despite the considerable potential of reinforcement learning (RL), robotic control tasks predominantly rely on imitation learning (IL) due to its better sample efficiency. However, it is costly to collect comprehensive expert demonstrations that enable IL to generalize to all possible scenarios, and any distribution shift would require recollecting data for finetuning. Therefore, RL is appealing if it can build upon IL as an efficient autonomous self-improvement procedure. We propose imitation bootstrapped reinforcement learning (IBRL), a …

abstract arxiv control cs.ai cs.lg data distribution efficiency expert finetuning imitation learning reinforcement reinforcement learning robotic sample shift tasks type