MultiSubs: A Large-scale Multimodal and Multilingual Dataset. (arXiv:2103.01910v3 [cs.CL] UPDATED)

This paper introduces a large-scale multimodal and multilingual dataset that
aims to facilitate research on grounding words to images in their contextual
usage in language. The dataset consists of images selected to unambiguously
illustrate concepts expressed in sentences from movie subtitles. The dataset is
a valuable resource as (i) the images are aligned to text fragments rather than
whole sentences; (ii) multiple images are possible for a text fragment and a
sentence; (iii) the sentences are free-form and real-world like; …

arxiv dataset multimodal scale