Kiki or Bouba? Sound Symbolism in Vision-and-Language Models

arXiv:2310.16781v3 Announce Type: replace-cross
Abstract: Although the mapping between sound and meaning in human language is assumed to be largely arbitrary, research in cognitive science has shown that there are non-trivial correlations between particular sounds and meanings across languages and demographic groups, a phenomenon known as sound symbolism. Among the many dimensions of meaning, sound symbolism is particularly salient and well-demonstrated with regards to cross-modal associations between language and the visual domain. In this work, we address the question of …

abstract arxiv cognitive cognitive science correlations cs.cl cs.cv cs.lg dimensions human language language models languages mapping meaning research science sound type vision vision-and-language