Frame Interpolation with Consecutive Brownian Bridge Diffusion

arXiv:2405.05953v1 Announce Type: new
Abstract: Recent work in Video Frame Interpolation (VFI) tries to formulate VFI as a diffusion-based conditional image generation problem, synthesizing the intermediate frame given a random noise and neighboring frames. Due to the relatively high resolution of videos, Latent Diffusion Models (LDMs) are employed as the conditional generation model, where the autoencoder compresses images into latent representations for diffusion and then reconstructs images from these latent representations. Such a formulation poses a crucial challenge: VFI expects …

abstract arxiv bridge cs.cv diffusion diffusion models image image generation intermediate interpolation latent diffusion models noise random resolution type video videos work