TARN-VIST: Topic Aware Reinforcement Network for Visual Storytelling

arXiv:2403.11550v1 Announce Type: new
Abstract: As a cross-modal task, visual storytelling aims to generate a story for an ordered image sequence automatically. Different from the image captioning task, visual storytelling requires not only modeling the relationships between objects in the image but also mining the connections between adjacent images. Recent approaches primarily utilize either end-to-end frameworks or multi-stage frameworks to generate relevant stories, but they usually overlook latent topic information. In this paper, in order to generate a more coherent …

abstract arxiv captioning cs.cv generate image images mining modal modeling network objects reinforcement relationships story storytelling type visual