Visually grounded few-shot word learning in low-resource settings

arXiv:2306.11371v3 Announce Type: replace-cross
Abstract: We propose a visually grounded speech model that learns new words and their visual depictions from just a few word-image example pairs. Given a set of test images and a spoken query, we ask the model which image depicts the query word. Previous work has simplified this few-shot learning problem by either using an artificial setting with digit word-image pairs or by using a large number of examples per class. Moreover, all previous studies were …

abstract arxiv cs.cl eess.as example few-shot image images low query set simplified speech spoken test type visual word words work