Detecting Hallucination and Coverage Errors in Retrieval Augmented Generation for Controversial Topics

arXiv:2403.08904v1 Announce Type: new
Abstract: We explore a strategy to handle controversial topics in LLM-based chatbots based on Wikipedia's Neutral Point of View (NPOV) principle: acknowledge the absence of a single true answer and surface multiple perspectives. We frame this as retrieval augmented generation, where perspectives are retrieved from a knowledge base and the LLM is tasked with generating a fluent and faithful response from the given perspectives. As a starting point, we use a deterministic retrieval system and then …

abstract arxiv chatbots coverage cs.cl errors explore hallucination llm multiple perspectives retrieval retrieval augmented generation strategy surface topics true type view wikipedia