TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding

arXiv:2404.09797v1 Announce Type: new
Abstract: The advent of Large Multimodal Models (LMMs) has sparked a surge in research aimed at harnessing their remarkable reasoning abilities. However, for understanding text-rich images, challenges persist in fully leveraging the potential of LMMs, and existing methods struggle with effectively processing high-resolution images. In this work, we propose TextCoT, a novel Chain-of-Thought framework for text-rich image understanding. TextCoT utilizes the captioning ability of LMMs to grasp the global context of the image and the grounding …

arxiv cs.cv image multimodal text type understanding zoom