Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering. (arXiv:2209.09513v1 [cs.CL])

When answering a question, humans utilize the information available across
different modalities to synthesize a consistent and complete chain of thought
(CoT). This process is normally a black box in the case of deep learning models
like large-scale language models. Recently, science question benchmarks have
been used to diagnose the multi-hop reasoning ability and interpretability of
an AI system. However, existing datasets fail to provide annotations for the
answers, or are restricted to the textual-only modality, small scales, and
limited …

arxiv learn multimodal question answering reasoning science