Improved Baselines for Data-efficient Perceptual Augmentation of LLMs

arXiv:2403.13499v1 Announce Type: new
Abstract: The abilities of large language models (LLMs) have recently progressed to unprecedented levels, paving the way to novel applications in a wide variety of areas. In computer vision, LLMs can be used to prime vision-language tasks such image captioning and visual question answering when coupled with pre-trained vision backbones. While different approaches have been explored to interface LLMs with ``perceptual backbones'' that process, e.g., visual or audio data, they are often explored for different tasks, …

abstract applications arxiv augmentation captioning computer computer vision cs.cv data image language language models large language large language models llms novel prime question question answering tasks the way type vision visual