[R] LaVIN: Large Vision-Language Instructed Model

​

https://preview.redd.it/t37xwe9i6u2b1.png?width=1440&format=png&auto=webp&v=enabled&s=8b0fa631e2988ea7b64a615544a86ef331bd65d2

Paper: [https://arxiv.org/pdf/2305.15023.pdf](https://arxiv.org/pdf/2305.15023.pdf)

Project: [https://github.com/luogen1996/LaVIN](https://github.com/luogen1996/LaVIN)

​

Adapting large language models to multimodal instructions typically requires a significant amount of training time. Both BLIP2 and mini-GPT4 require large sets of paired text and image samples for pretraining. Additionally, LLaVA requires fine-tuning of the entire large language model. These approaches greatly increase the cost of multimodal adaptation and can lead to a decrease in the textual capabilities of the large language model.

In this paper, we propose **an efficient multimodal instruction …

cost fine-tuning gpt4 image language language model language models large language model large language models machinelearning multimodal text training vision