Understanding News Thumbnail Representativeness by Counterfactual Text-Guided Contrastive Language-Image Pretraining

arXiv:2402.11159v1 Announce Type: cross
Abstract: This paper delves into the critical challenge of understanding the representativeness of news thumbnail images, which often serve as the first visual engagement for readers when an article is disseminated on social media. We focus on whether a news image represents the main subject discussed in the news text. To serve the challenge, we introduce \textsc{NewsTT}, a manually annotated dataset of news thumbnail image and text pairs. We found that pretrained vision and language models, …

abstract article arxiv challenge counterfactual cs.cl cs.cv engagement focus image images language media paper pretraining readers serve social social media text type understanding visual