Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs

arXiv:2308.13812v2 Announce Type: replace-cross
Abstract: Text-to-video (T2V) synthesis has gained increasing attention in the community, in which the recently emerged diffusion models (DMs) have promisingly shown stronger performance than the past approaches. While existing state-of-the-art DMs are competent to achieve high-resolution video generation, they may largely suffer from key limitations (e.g., action occurrence disorders, crude video motions) with respect to the intricate temporal dynamics modeling, one of the crux of video synthesis. In this work, we investigate strengthening the awareness …

arxiv cs.ai cs.cv diffusion dynamics llms text text-to-video type video video diffusion