GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planning

arXiv:2311.12631v2 Announce Type: replace
Abstract: Recent advances in text-to-video generation have harnessed the power of diffusion models to create visually compelling content conditioned on text prompts. However, they usually encounter high computational costs and often struggle to produce videos with coherent physical motions. To tackle these issues, we propose GPT4Motion, a training-free framework that leverages the planning capability of large language models such as GPT, the physical simulation strength of Blender, and the excellent image generation ability of text-to-image diffusion …

abstract advances arxiv blender computational costs cs.cv diffusion diffusion models gpt planning power prompts scripting struggle text text-to-video type via video video generation videos