TrailBlazer: Trajectory Control for Diffusion-Based Video Generation

arXiv:2401.00896v2 Announce Type: replace
Abstract: Within recent approaches to text-to-video (T2V) generation, achieving controllability in the synthesized video is often a challenge. Typically, this issue is addressed by providing low-level per-frame guidance in the form of edge maps, depth maps, or an existing video to be altered. However, the process of obtaining such guidance can be labor-intensive. This paper focuses on enhancing controllability in video synthesis by employing straightforward bounding boxes to guide the subject in various ways, all without …

abstract arxiv challenge control cs.cv diffusion edge form guidance however issue low maps per process synthesized text text-to-video trajectory type video video generation