Efficient Multimodal Learning from Data-centric Perspective

arXiv:2402.11530v1 Announce Type: new
Abstract: Multimodal Large Language Models (MLLMs) have demonstrated notable capabilities in general visual understanding and reasoning tasks. However, their deployment is hindered by substantial computational costs in both training and inference, limiting accessibility to the broader research and user communities. A straightforward solution is to leverage smaller pre-trained vision and language models, which inevitably causes significant performance drop. In this paper, we demonstrate the possibility to beat the scaling law and train a smaller but better …

arxiv cs.cv data data-centric multimodal multimodal learning perspective type