TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Models

arXiv:2404.16306v1 Announce Type: new
Abstract: Text-conditioned image-to-video generation (TI2V) aims to synthesize a realistic video starting from a given image (e.g., a woman's photo) and a text description (e.g., "a woman is drinking water."). Existing TI2V frameworks often require costly training on video-text datasets and specific model designs for text and image conditioning. In this paper, we propose TI2V-Zero, a zero-shot, tuning-free method that empowers a pretrained text-to-video (T2V) diffusion model to be conditioned on a provided image, enabling TI2V …

abstract arxiv cs.cv datasets designs diffusion diffusion models frameworks image image-to-video photo text text-to-video training type video video diffusion video generation water zero-shot