IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding

arXiv:2402.18476v1 Announce Type: new
Abstract: Despite achieving rapid developments and with widespread applications, Large Vision-Language Models (LVLMs) confront a serious challenge of being prone to generating hallucinations. An over-reliance on linguistic priors has been identified as a key factor leading to these hallucinations. In this paper, we propose to alleviate this problem by introducing a novel image-biased decoding (IBD) technique. Our method derives the next-token probability distribution by contrasting predictions from a conventional LVLM with those of an image-biased LVLM, …

abstract applications arxiv challenge cs.cv decoding hallucinations image key language language models paper reliance type via vision vision-language models