What If the TV Was Off? Examining Counterfactual Reasoning Abilities of Multi-modal Language Models

arXiv:2310.06627v4 Announce Type: replace-cross
Abstract: Counterfactual reasoning, a fundamental aspect of human cognition, involves contemplating alternatives to established facts or past events, significantly enhancing our abilities in planning and decision-making. In light of the advancements in current multi-modal large language models, we explore their effectiveness in counterfactual reasoning. To facilitate this investigation, we introduce a novel dataset, C-VQA, specifically designed to test the counterfactual reasoning capabilities of modern multi-modal large language models. This dataset is constructed by infusing original questions …

arxiv counterfactual cs.cl cs.cv cs.lg language language models modal multi-modal reasoning type