Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimization

arXiv:2403.08730v1 Announce Type: cross
Abstract: Multimodal Large Language Models (MLLMs) excel in generating responses based on visual inputs. However, they often suffer from a bias towards generating responses similar to their pretraining corpus, overshadowing the importance of visual information. We treat this bias as a "preference" for pretraining statistics, which hinders the model's grounding in visual input. To mitigate this issue, we propose Bootstrapped Preference Optimization (BPO), which conducts preference learning with datasets containing negative responses bootstrapped from the model …

abstract arxiv bias cs.cl cs.cv excel however importance information inputs language language model language models large language large language model large language models mllms multimodal multimodal large language model optimization pretraining responses statistics type visual