TinyChart: Efficient Chart Understanding with Visual Token Merging and Program-of-Thoughts Learning

arXiv:2404.16635v1 Announce Type: new
Abstract: Charts are important for presenting and explaining complex data relationships. Recently, multimodal large language models (MLLMs) have shown remarkable capabilities in various chart understanding tasks. However, the sheer size of these models in terms of parameters and computational requirements limits their use in resource-constrained environments. In this paper, we present TinyChart, an efficient MLLM for chart understanding with only 3B parameters. TinyChart overcomes two key challenges in efficient chart understanding: (1) reduce the burden of …

arxiv cs.cv merging thoughts token type understanding visual