all AI news
Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledge
Feb. 19, 2024, 5:43 a.m. | Genglin Liu, Xingyao Wang, Lifan Yuan, Yangyi Chen, Hao Peng
cs.LG updates on arXiv.org arxiv.org
Abstract: Can large language models (LLMs) express their uncertainty in situations where they lack sufficient parametric knowledge to generate reasonable responses? This work aims to systematically investigate LLMs' behaviors in such situations, emphasizing the trade-off between honesty and helpfulness. To tackle the challenge of precisely determining LLMs' knowledge gaps, we diagnostically create unanswerable questions containing non-existent concepts or false premises, ensuring that they are outside the LLMs' vast training data. By compiling a benchmark, UnknownBench, which …
abstract arxiv challenge cs.ai cs.cl cs.lg express generate honesty knowledge language language models large language large language models llms parametric questions responses trade trade-off type uncertainty work
More from arxiv.org / cs.LG updates on arXiv.org
Jobs in AI, ML, Big Data
Data Engineer
@ Lemon.io | Remote: Europe, LATAM, Canada, UK, Asia, Oceania
Artificial Intelligence – Bioinformatic Expert
@ University of Texas Medical Branch | Galveston, TX
Lead Developer (AI)
@ Cere Network | San Francisco, US
Research Engineer
@ Allora Labs | Remote
Ecosystem Manager
@ Allora Labs | Remote
Founding AI Engineer, Agents
@ Occam AI | New York