Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall

arXiv:2404.16164v1 Announce Type: cross
Abstract: Large language models (LLMs) have shown remarkable performance on a variety of NLP tasks, and are being rapidly adopted in a wide range of use cases. It is therefore of vital importance to holistically evaluate the factuality of their generated outputs, as hallucinations remain a challenging issue.
In this work, we focus on assessing LLMs' ability to recall factual knowledge learned from pretraining, and the factors that affect this ability. To that end, we construct …

abstract arxiv cases cs.ai cs.cl cs.lg evaluation generated hallucinations importance knowledge language language models large language large language models llms nlp performance recall tasks type use cases vital