AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph

arXiv:2311.09174v3 Announce Type: replace
Abstract: Cognitive research indicates that abstraction ability is essential in human intelligence, which remains under-explored in language models. In this paper, we present AbsPyramid, a unified entailment graph of 221K textual descriptions of abstraction knowledge. While existing resources only touch nouns or verbs within simplified events or specific domains, AbsPyramid collects abstract knowledge for three components of diverse events to comprehensively evaluate the abstraction ability of language models in the open domain. Experimental results demonstrate that …

abstract abstraction arxiv benchmarking cognitive cs.ai cs.cl graph human human intelligence intelligence knowledge language language models paper research resources textual type