Diffusion Models for Video Prediction and Infilling. (arXiv:2206.07696v2 [cs.CV] UPDATED)

Predicting and anticipating future outcomes or reasoning about missing
information in a sequence are critical skills for agents to be able to make
intelligent decisions. This requires strong, temporally coherent generative
capabilities. Diffusion models have shown remarkable success in several
generative tasks, but have not been extensively explored in the video domain.
We present Random-Mask Video Diffusion (RaMViD), which extends image diffusion
models to videos using 3D convolutions, and introduces a new conditioning
technique during training. By varying the mask …

arxiv cv diffusion diffusion models prediction video