Animate Your Motion: Turning Still Images into Dynamic Videos

arXiv:2403.10179v1 Announce Type: new
Abstract: In recent years, diffusion models have made remarkable strides in text-to-video generation, sparking a quest for enhanced control over video outputs to more accurately reflect user intentions. Traditional efforts predominantly focus on employing either semantic cues, like images or depth maps, or motion-based conditions, like moving sketches or object bounding boxes. Semantic inputs offer a rich scene context but lack detailed motion specificity; conversely, motion inputs provide precise trajectory information but miss the broader semantic …

abstract arxiv control cs.cv diffusion diffusion models dynamic focus images maps moving quest semantic text text-to-video type video video generation videos