A Comprehensive Evaluation on Event Reasoning of Large Language Models

arXiv:2404.17513v1 Announce Type: new
Abstract: Event reasoning is a fundamental ability that underlies many applications. It requires event schema knowledge to perform global reasoning and needs to deal with the diversity of the inter-event relations and the reasoning paradigms. How well LLMs accomplish event reasoning on various relations and reasoning paradigms remains unknown. To mitigate this disparity, we comprehensively evaluate the abilities of event reasoning of LLMs. We introduce a novel benchmark EV2 for EValuation of EVent reasoning. EV2 consists …

abstract applications arxiv cs.ai cs.cl deal diversity evaluation event fundamental global knowledge language language models large language large language models llms reasoning relations schema type