Evaluating the Quality of Answers in Political Q&A Sessions with Large Language Models

arXiv:2404.08816v1 Announce Type: new
Abstract: This paper presents a new approach to evaluating the quality of answers in political question-and-answer sessions. We propose to measure an answer's quality based on the degree to which it allows us to infer the initial question accurately. This conception of answer quality inherently reflects their relevance to initial questions. Drawing parallels with semantic search, we argue that this measurement approach can be operationalized by fine-tuning a large language model on the observed corpus of …

abstract arxiv cs.cl econ.em language language models large language large language models paper political quality question type