Visual Grounding Helps Learn Word Meanings in Low-Data Regimes

arXiv:2310.13257v2 Announce Type: replace
Abstract: Modern neural language models (LMs) are powerful tools for modeling human sentence production and comprehension, and their internal representations are remarkably well-aligned with representations of language in the human brain. But to achieve these results, LMs must be trained in distinctly un-human-like ways - requiring orders of magnitude more language data than children receive during development, and without perceptual or social context. Do models trained more naturalistically -- with grounded supervision -- exhibit more humanlike …

abstract arxiv brain cs.ai cs.cl data human human-like language language models learn lms low modeling modern orders production results tools type visual word