On the Calibration of Multilingual Question Answering LLMs

arXiv:2311.08669v2 Announce Type: replace-cross
Abstract: Multilingual pre-trained Large Language Models (LLMs) are incredibly effective at Question Answering (QA), a core task in Natural Language Understanding, achieving high accuracies on several multilingual benchmarks. However, little is known about how well their confidences are calibrated. In this paper, we comprehensively benchmark the calibration of several multilingual LLMs (MLLMs) on a variety of QA tasks. We perform extensive experiments, spanning encoder-only, encoder-decoder, and decoder-only QA models (size varying from 110M to 7B parameters) …

abstract arxiv benchmark benchmarks core cs.cl cs.lg however language language models language understanding large language large language models llms multilingual natural natural language paper question question answering type understanding