VSTAR: Generative Temporal Nursing for Longer Dynamic Video Synthesis

arXiv:2403.13501v1 Announce Type: cross
Abstract: Despite tremendous progress in the field of text-to-video (T2V) synthesis, open-sourced T2V diffusion models struggle to generate longer videos with dynamically varying and evolving content. They tend to synthesize quasi-static videos, ignoring the necessary visual change-over-time implied in the text prompt. At the same time, scaling these models to enable longer, more dynamic video synthesis often remains computationally intractable. To address this challenge, we introduce the concept of Generative Temporal Nursing (GTN), where we aim …

abstract arxiv change cs.ai cs.cv cs.lg cs.mm diffusion diffusion models dynamic generate generative nursing progress prompt scaling struggle synthesis temporal text text-to-video type video videos visual