X-LLaVA: Optimizing Bilingual Large Vision-Language Alignment

arXiv:2403.11399v1 Announce Type: new
Abstract: The impressive development of large language models (LLMs) is expanding into the realm of large multimodal models (LMMs), which incorporate multiple types of data beyond text. However, the nature of multimodal models leads to significant expenses in the creation of training data. Furthermore, constructing multilingual data for LMMs presents its own set of challenges due to language diversity and complexity. Therefore, in this study, we propose two cost-effective methods to solve this problem: (1) vocabulary …

abstract alignment arxiv beyond bilingual cs.cl data development however language language models large language large language models large multimodal models leads llava llms lmms multilingual multimodal multimodal models multiple nature text training training data type types vision