How Generalizable Is My Behavior Cloning Policy? A Statistical Approach to Trustworthy Performance Evaluation

arXiv:2405.05439v1 Announce Type: cross
Abstract: With the rise of stochastic generative models in robot policy learning, end-to-end visuomotor policies are increasingly successful at solving complex tasks by learning from human demonstrations. Nevertheless, since real-world evaluation costs afford users only a small number of policy rollouts, it remains a challenge to accurately gauge the performance of such policies. This is exacerbated by distribution shifts causing unpredictable changes in performance during deployment. To rigorously evaluate behavior cloning policies, we present a framework …

abstract arxiv behavior cloning costs cs.ai cs.lg cs.ro evaluation generative generative models human performance policies policy robot small stat.ap statistical stochastic tasks trustworthy type world