Comparing Plausibility Estimates in Base and Instruction-Tuned Large Language Models

arXiv:2403.14859v1 Announce Type: new
Abstract: Instruction-tuned LLMs can respond to explicit queries formulated as prompts, which greatly facilitates interaction with human users. However, prompt-based approaches might not always be able to tap into the wealth of implicit knowledge acquired by LLMs during pre-training. This paper presents a comprehensive study of ways to evaluate semantic plausibility in LLMs. We compare base and instruction-tuned LLM performance on an English sentence plausibility task via (a) explicit prompting and (b) implicit estimation via direct …

abstract acquired arxiv cs.ai cs.cl however human instruction-tuned knowledge language language models large language large language models llms paper pre-training prompt prompts queries study training type wealth