Semantic Flow: Learning Semantic Field of Dynamic Scenes from Monocular Videos

arXiv:2404.05163v1 Announce Type: new
Abstract: In this work, we pioneer Semantic Flow, a neural semantic representation of dynamic scenes from monocular videos. In contrast to previous NeRF methods that reconstruct dynamic scenes from the colors and volume densities of individual points, Semantic Flow learns semantics from continuous flows that contain rich 3D motion information. As there is 2D-to-3D ambiguity problem in the viewing direction when extracting 3D flow features from 2D video frames, we consider the volume densities as opacity …

arxiv cs.cv dynamic flow semantic type videos