CausalBench: A Comprehensive Benchmark for Causal Learning Capability of Large Language Models

arXiv:2404.06349v1 Announce Type: new
Abstract: Causality reveals fundamental principles behind data distributions in real-world scenarios, and the capability of large language models (LLMs) to understand causality directly impacts their efficacy across explaining outputs, adapting to new evidence, and generating counterfactuals. With the proliferation of LLMs, the evaluation of this capacity is increasingly garnering attention. However, the absence of a comprehensive benchmark has rendered existing evaluation studies being straightforward, undiversified, and homogeneous. To address these challenges, this paper proposes a comprehensive …

abstract arxiv benchmark capability causal causality cs.lg data evaluation evidence impacts language language models large language large language models llms type world