Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition

arXiv:2403.14148v1 Announce Type: cross
Abstract: Video diffusion models have recently made great progress in generation quality, but are still limited by the high memory and computational requirements. This is because current video diffusion models often attempt to process high-dimensional videos directly. To tackle this issue, we propose content-motion latent diffusion model (CMD), a novel efficient extension of pretrained image diffusion models for video generation. Specifically, we propose an autoencoder that succinctly encodes a video as a combination of a content …

abstract arxiv computational cs.cv cs.lg current diffusion diffusion model diffusion models issue memory process progress quality requirements type via video video diffusion videos