MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators

arXiv:2404.05014v1 Announce Type: new
Abstract: Recent advances in Text-to-Video generation (T2V) have achieved remarkable success in synthesizing high-quality general videos from textual descriptions. A largely overlooked problem in T2V is that existing models have not adequately encoded physical knowledge of the real world, thus generated videos tend to have limited motion and poor variations. In this paper, we propose \textbf{MagicTime}, a metamorphic time-lapse video generation model, which learns real-world physics knowledge from time-lapse videos and implements metamorphic generation. First, we …

abstract advances arxiv cs.cv general generated knowledge quality success text text-to-video textual type video video generation videos world