A Model-Based Approach for Improving Reinforcement Learning Efficiency Leveraging Expert Observations

arXiv:2402.18836v1 Announce Type: new
Abstract: This paper investigates how to incorporate expert observations (without explicit information on expert actions) into a deep reinforcement learning setting to improve sample efficiency. First, we formulate an augmented policy loss combining a maximum entropy reinforcement learning objective with a behavioral cloning loss that leverages a forward dynamics model. Then, we propose an algorithm that automatically adjusts the weights of each component in the augmented loss function. Experiments on a variety of continuous control tasks …

abstract arxiv cloning cs.lg efficiency entropy expert information loss paper policy reinforcement reinforcement learning sample type