Cantor: Inspiring Multimodal Chain-of-Thought of MLLM

arXiv:2404.16033v1 Announce Type: cross
Abstract: With the advent of large language models(LLMs) enhanced by the chain-of-thought(CoT) methodology, visual reasoning problem is usually decomposed into manageable sub-tasks and tackled sequentially with various external tools. However, such a paradigm faces the challenge of the potential "determining hallucinations" in decision-making due to insufficient visual information and the limitation of low-level perception tools that fail to provide abstract summaries necessary for comprehensive reasoning. We argue that converging visual context acquisition and logical reasoning is …

arxiv cs.cl cs.cv mllm multimodal thought type