Learning Priors for Non Rigid SfM from Casual Videos

arXiv:2404.07097v1 Announce Type: new
Abstract: We tackle the long-standing challenge of reconstructing 3D structures and camera positions from videos. The problem is particularly hard when objects are transformed in a non-rigid way. Current approaches to this problem make unrealistic assumptions or require a long optimization time.
We present TracksTo4D, a novel deep learning-based approach that enables inferring 3D structure and camera positions from dynamic content originating from in-the-wild videos using a single feed-forward pass on a sparse point track matrix. …

abstract arxiv assumptions challenge cs.cv current novel objects optimization type videos