Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models

arXiv:2306.00973v3 Announce Type: replace
Abstract: Generative models have recently exhibited exceptional capabilities in text-to-image generation, but still struggle to generate image sequences coherently. In this work, we focus on a novel, yet challenging task of generating a coherent image sequence based on a given storyline, denoted as open-ended visual storytelling. We make the following three contributions: (i) to fulfill the task of visual storytelling, we propose a learning-based auto-regressive image generation model, termed as StoryGen, with a novel vision-language context …

arxiv cs.cv diffusion diffusion models intelligent latent diffusion models storytelling type via visual