Frame by Familiar Frame: Understanding Replication in Video Diffusion Models

arXiv:2403.19593v1 Announce Type: new
Abstract: Building on the momentum of image generation diffusion models, there is an increasing interest in video-based diffusion models. However, video generation poses greater challenges due to its higher-dimensional nature, the scarcity of training data, and the complex spatiotemporal relationships involved. Image generation models, due to their extensive data requirements, have already strained computational resources to their limits. There have been instances of these models reproducing elements from the training samples, leading to concerns and even …

abstract arxiv building challenges cs.cv data diffusion diffusion models however image image generation image generation models nature relationships replication training training data type understanding video video diffusion video generation