SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer

arXiv:2404.03736v1 Announce Type: new
Abstract: Recent advances in 2D/3D generative models enable the generation of dynamic 3D objects from a single-view video. Existing approaches utilize score distillation sampling to form the dynamic scene as dynamic NeRF or dense 3D Gaussians. However, these methods struggle to strike a balance among reference view alignment, spatio-temporal consistency, and motion fidelity under single-view conditions due to the implicit nature of NeRF or the intricate dense Gaussian motion prediction. To address these issues, this paper …

3d objects abstract advances alignment arxiv balance cs.cv distillation dynamic form generative generative models however nerf objects reference sampling strike struggle temporal transfer type video view