Motion Inversion for Video Customization

arXiv:2403.20193v1 Announce Type: new
Abstract: In this research, we present a novel approach to motion customization in video generation, addressing the widespread gap in the thorough exploration of motion representation within video generative models. Recognizing the unique challenges posed by video's spatiotemporal nature, our method introduces Motion Embeddings, a set of explicit, temporally coherent one-dimensional embeddings derived from a given video. These embeddings are designed to integrate seamlessly with the temporal transformer modules of video diffusion models, modulating self-attention computations …

abstract arxiv challenges cs.cv customization embeddings exploration gap generative generative models nature novel representation research set type video video generation