Visually Grounded Speech Models have a Mutual Exclusivity Bias

arXiv:2403.13922v1 Announce Type: new
Abstract: When children learn new words, they employ constraints such as the mutual exclusivity (ME) bias: a novel word is mapped to a novel object rather than a familiar one. This bias has been studied computationally, but only in models that use discrete word representations as input, ignoring the high variability of spoken words. We investigate the ME bias in the context of visually grounded speech models that learn from natural images and continuous speech audio. …

abstract arxiv bias children constraints cs.cl learn mapped novel object speech type word words