HRVDA: High-Resolution Visual Document Assistant

arXiv:2404.06918v1 Announce Type: new
Abstract: Leveraging vast training data, multimodal large language models (MLLMs) have demonstrated formidable general visual comprehension capabilities and achieved remarkable performance across various tasks. However, their performance in visual document understanding still leaves much room for improvement. This discrepancy is primarily attributed to the fact that visual document understanding is a fine-grained prediction task. In natural scenes, MLLMs typically use low-resolution images, leading to a substantial loss of visual information. Furthermore, general-purpose MLLMs do not excel …

abstract arxiv assistant capabilities cs.cv data document document understanding general however improvement language language models large language large language models mllms multimodal performance resolution room tasks training training data type understanding vast visual