A Comprehensive Overhaul of Multimodal Assistant with Small Language Models

arXiv:2403.06199v1 Announce Type: new
Abstract: Multimodal Large Language Models (MLLMs) have showcased impressive skills in tasks related to visual understanding and reasoning. Yet, their widespread application faces obstacles due to the high computational demands during both the training and inference phases, restricting their use to a limited audience within the research and user communities. In this paper, we investigate the design aspects of Multimodal Small Language Models (MSLMs) and propose an efficient multimodal assistant named Mipha, which is designed to …

arxiv assistant cs.cl cs.cv language language models multimodal small small language models type