Language-Informed Visual Concept Learning

arXiv:2312.03587v2 Announce Type: replace
Abstract: Our understanding of the visual world is centered around various concept axes, characterizing different aspects of visual entities. While different concept axes can be easily specified by language, e.g. color, the exact visual nuances along each axis often exceed the limitations of linguistic articulations, e.g. a particular style of painting. In this work, our goal is to learn a language-informed visual concept representation, by simply distilling large pre-trained vision-language models. Specifically, we train a set …

abstract arxiv color concept cs.cv language limitations painting style type understanding visual world