Coherent 3D Portrait Video Reconstruction via Triplane Fusion

arXiv:2405.00794v1 Announce Type: new
Abstract: Recent breakthroughs in single-image 3D portrait reconstruction have enabled telepresence systems to stream 3D portrait videos from a single camera in real-time, potentially democratizing telepresence. However, per-frame 3D reconstruction exhibits temporal inconsistency and forgets the user's appearance. On the other hand, self-reenactment methods can render coherent 3D portraits by driving a personalized 3D prior, but fail to faithfully reconstruct the user's per-frame appearance (e.g., facial expressions and lighting). In this work, we recognize the need …

3d reconstruction abstract arxiv cs.cv fusion however image per portrait real-time systems temporal type via video videos