Uncertainty quantification in fine-tuned LLMs using LoRA ensembles

arXiv:2402.12264v1 Announce Type: new
Abstract: Fine-tuning large language models can improve task specific performance, although a general understanding of what the fine-tuned model has learned, forgotten and how to trust its predictions is still missing. We derive principled uncertainty quantification for fine-tuned LLMs with posterior approximations using computationally efficient low-rank adaptation ensembles. We analyze three common multiple-choice datasets using low-rank adaptation ensembles based on Mistral-7b, and draw quantitative and qualitative conclusions on their perceived complexity and model efficacy on the …

abstract arxiv cs.ai cs.cl cs.lg fine-tuning general language language models large language large language models llms lora low low-rank adaptation performance posterior predictions quantification stat.ml trust type uncertainty understanding