The First to Know: How Token Distributions Reveal Hidden Knowledge in Large Vision-Language Models?

arXiv:2403.09037v1 Announce Type: new
Abstract: Large vision-language models (LVLMs), designed to interpret and respond to human instructions, occasionally generate hallucinated or harmful content due to inappropriate instructions. This study uses linear probing to shed light on the hidden knowledge at the output layer of LVLMs. We demonstrate that the logit distributions of the first tokens contain sufficient information to determine whether to respond to the instructions, including recognizing unanswerable visual questions, defending against multi-modal jailbreaking attack, and identifying deceptive questions. …

abstract arxiv cs.cl cs.cv generate hidden human inappropriate knowledge language language models layer light linear study token type vision vision-language models