Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation

arXiv:2305.10874v3 Announce Type: replace
Abstract: With the explosive popularity of AI-generated content (AIGC), video generation has recently received a lot of attention. Generating videos guided by text instructions poses significant challenges, such as modeling the complex relationship between space and time, and the lack of large-scale text-video paired data. Existing text-video datasets suffer from limitations in both content quality and scale, or they are not open-source, rendering them inaccessible for study and use. For model design, previous approaches extend pretrained …

abstract aigc ai-generated content arxiv attention challenges cs.cv data datasets generated modeling relationship scale space space and time text text-to-video type video video generation videos