Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Video

arXiv:2404.09833v1 Announce Type: new
Abstract: Creating high-quality and interactive virtual environments, such as games and simulators, often involves complex and costly manual modeling processes. In this paper, we present Video2Game, a novel approach that automatically converts videos of real-world scenes into realistic and interactive game environments. At the heart of our system are three core components:(i) a neural radiance fields (NeRF) module that effectively captures the geometry and visual appearance of the scene; (ii) a mesh module that distills the …

abstract arxiv browser cs.ai cs.cv environment environments game games interactive modeling novel paper processes quality real-time type video videos virtual virtual environments world