Reality's Canvas, Language's Brush: Crafting 3D Avatars from Monocular Video

arXiv:2312.04784v2 Announce Type: replace
Abstract: Recent advancements in 3D avatar generation excel with multi-view supervision for photorealistic models. However, monocular counterparts lag in quality despite broader applicability. We propose ReCaLaB to close this gap. ReCaLaB is a fully-differentiable pipeline that learns high-fidelity 3D human avatars from just a single RGB video. A pose-conditioned deformable NeRF is optimized to volumetrically represent a human subject in canonical T-pose. The canonical representation is then leveraged to efficiently associate neural textures using 2D-3D correspondences. …

3d avatars abstract arxiv avatar avatars canvas cs.cv differentiable excel fidelity gap however human language photorealistic pipeline quality reality supervision type video view