Hybrid Video Diffusion Models with 2D Triplane and 3D Wavelet Representation

arXiv:2402.13729v1 Announce Type: new
Abstract: Generating high-quality videos that synthesize desired realistic content is a challenging task due to their intricate high-dimensionality and complexity of videos. Several recent diffusion-based methods have shown comparable performance by compressing videos to a lower-dimensional latent space, using traditional video autoencoder architecture. However, such method that employ standard frame-wise 2D and 3D convolution fail to fully exploit the spatio-temporal nature of videos. To address this issue, we propose a novel hybrid video diffusion model, called …

abstract architecture arxiv autoencoder complexity cs.cv diffusion diffusion models dimensionality hybrid performance quality representation space type video video diffusion videos wavelet