MEReQ: Max-Ent Residual-Q Inverse RL for Sample-Efficient Alignment from Intervention

arXiv:2406.16258v1 Announce Type: cross
Abstract: Aligning robot behavior with human preferences is crucial for deploying embodied AI agents in human-centered environments. A promising solution is interactive imitation learning from human intervention, where a human expert observes the policy's execution and provides interventions as feedback. However, existing methods often fail to utilize the prior policy efficiently to facilitate learning, thus hindering sample efficiency. In this work, we introduce MEReQ (Maximum-Entropy Residual-Q Inverse Reinforcement Learning), designed for sample-efficient alignment from human intervention. …

abstract agents ai agents alignment arxiv behavior cs.ai cs.lg cs.ro deploying embodied embodied ai environments expert fail feedback however human human intervention imitation learning interactive max policy residual robot sample solution type