Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation

arXiv:2403.04436v1 Announce Type: cross
Abstract: We present Human to Humanoid (H2O), a reinforcement learning (RL) based framework that enables real-time whole-body teleoperation of a full-sized humanoid robot with only an RGB camera. To create a large-scale retargeted motion dataset of human movements for humanoid robots, we propose a scalable "sim-to-data" process to filter and pick feasible motions using a privileged motion imitator. Afterwards, we train a robust real-time humanoid motion imitator in simulation using these refined motions and transfer it …

abstract arxiv cs.ai cs.lg cs.ro cs.sy data dataset eess.sy filter framework h2o human humanoid humanoid robot movements process real-time reinforcement reinforcement learning robot robots scalable scale sim teleoperation type