TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Models

arXiv:2403.17005v1 Announce Type: new
Abstract: Recent advances in text-to-video generation have demonstrated the utility of powerful diffusion models. Nevertheless, the problem is not trivial when shaping diffusion models to animate static image (i.e., image-to-video generation). The difficulty originates from the aspect that the diffusion process of subsequent animated frames should not only preserve the faithful alignment with the given image but also pursue temporal coherence among adjacent frames. To alleviate this, we present TRIP, a new recipe of image-to-video diffusion …

arxiv cs.cv cs.mm diffusion diffusion models image image-to-video noise prior residual temporal trip type video video diffusion