Real-Time Simulated Avatar from Head-Mounted Sensors

arXiv:2403.06862v1 Announce Type: new
Abstract: We present SimXR, a method for controlling a simulated avatar from information (headset pose and cameras) obtained from AR / VR headsets. Due to the challenging viewpoint of head-mounted cameras, the human body is often clipped out of view, making traditional image-based egocentric pose estimation challenging. On the other hand, headset poses provide valuable information about overall body motion, but lack fine-grained details about the hands and feet. To synergize headset poses with cameras, we …

abstract arxiv avatar cameras cs.cv cs.gr cs.ro head headset headsets human image information making real-time sensors type view vr headsets