LIEDER: Linguistically-Informed Evaluation for Discourse Entity Recognition

arXiv:2403.06301v1 Announce Type: new
Abstract: Discourse Entity (DE) recognition is the task of identifying novel and known entities introduced within a text. While previous work has found that large language models have basic, if imperfect, DE recognition abilities (Schuster and Linzen, 2022), it remains largely unassessed which of the fundamental semantic properties that govern the introduction and subsequent reference to DEs they have knowledge of. We propose the Linguistically-Informed Evaluation for Discourse Entity Recognition (LIEDER) dataset that allows for a …

abstract arxiv basic cs.cl discourse evaluation found language language models large language large language models novel recognition semantic text type work