I Can't Believe There's No Images! Learning Visual Tasks Using only Language Data. (arXiv:2211.09778v1 [cs.CV])

Many high-level skills that are required for computer vision tasks, such as
parsing questions, comparing and contrasting semantics, and writing
descriptions, are also required in other domains such as natural language
processing. In this paper, we ask whether this makes it possible to learn those
skills from text data and then use them to complete vision tasks without ever
training on visual training data. Key to our approach is exploiting the joint
embedding space of contrastively trained vision and language …

arxiv data images language language data