Visual Commonsense in Pretrained Unimodal and Multimodal Models. (arXiv:2205.01850v1 [cs.CL])

Web: http://arxiv.org/abs/2205.01850

Our commonsense knowledge about objects includes their typical visual
attributes; we know that bananas are typically yellow or green, and not purple.
Text and image corpora, being subject to reporting bias, represent this
world-knowledge to varying degrees of faithfulness. In this paper, we
investigate to what degree unimodal (language-only) and multimodal (image and
language) models capture a broad range of visually salient attributes. To that
end, we create the Visual Commonsense Tests (ViComTe) dataset covering 5
property types (color, shape, …

arxiv models multimodal multimodal models