On the Limitations of Large Language Models (LLMs): False Attribution

arXiv:2404.04631v1 Announce Type: new
Abstract: In this work, we provide insight into one important limitation of large language models (LLMs), i.e. false attribution, and introduce a new hallucination metric - Simple Hallucination Index (SHI). The task of automatic author attribution for relatively small chunks of text is an important NLP task but can be challenging. We empirically evaluate the power of 3 open SotA LLMs in zero-shot setting (LLaMA-2-13B, Mixtral 8x7B, and Gemma-7B), especially as human annotation can be costly. …

abstract arxiv attribution author cs.cl false hallucination hallucination index index insight language language models large language large language models limitations llms nlp simple small text type work