Evaluating language-biased image classification based on semantic representations. (arXiv:2201.11014v1 [cs.CV])

Web: http://arxiv.org/abs/2201.11014

Humans show language-biased image recognition for a word-embedded image,
known as picture-word interference. Such interference depends on hierarchical
semantic categories and reflects that human language processing highly
interacts with visual processing. Similar to humans, recent artificial models
jointly trained on texts and images, e.g., OpenAI CLIP, show language-biased
image classification. Exploring whether the bias leads to interferences similar
to those observed in humans can contribute to understanding how much the model
acquires hierarchical semantic representations from joint learning of language …

arxiv classification cv language semantic