SkelFormer: Markerless 3D Pose and Shape Estimation using Skeletal Transformers

arXiv:2404.12625v1 Announce Type: new
Abstract: We introduce SkelFormer, a novel markerless motion capture pipeline for multi-view human pose and shape estimation. Our method first uses off-the-shelf 2D keypoint estimators, pre-trained on large-scale in-the-wild data, to obtain 3D joint positions. Next, we design a regression-based inverse-kinematic skeletal transformer that maps the joint positions to pose and shape representations from heavily noisy observations. This module integrates prior knowledge about pose space and infers the full pose state at runtime. Separating the 3D …

abstract arxiv cs.cv data design human maps motion capture next novel pipeline regression scale transformer transformers type view