EpiK-Eval: Evaluation for Language Models as Epistemic Models

arXiv:2310.15372v2 Announce Type: replace
Abstract: In the age of artificial intelligence, the role of large language models (LLMs) is becoming increasingly central. Despite their growing prevalence, their capacity to consolidate knowledge from different training documents - a crucial ability in numerous applications - remains unexplored. This paper presents the first study examining the capability of LLMs to effectively combine such information within their parameter space. We introduce EpiK-Eval, a novel question-answering benchmark tailored to evaluate LLMs' proficiency in formulating a …

arxiv cs.ai cs.cl epik evaluation language language models type