Towards Generalist Robot Learning from Internet Video: A Survey

arXiv:2404.19664v1 Announce Type: cross
Abstract: This survey presents an overview of methods for learning from video (LfV) in the context of reinforcement learning (RL) and robotics. We focus on methods capable of scaling to large internet video datasets and, in the process, extracting foundational knowledge about the world's dynamics and physical human behaviour. Such methods hold great promise for developing general-purpose robots.
We open with an overview of fundamental concepts relevant to the LfV-for-robotics setting. This includes a discussion of …

abstract arxiv context cs.lg cs.ro datasets dynamics focus foundational internet knowledge overview process reinforcement reinforcement learning robot robotics scaling survey type video world