Measurement in the Age of LLMs: An Application to Ideological Scaling

arXiv:2312.09203v2 Announce Type: replace
Abstract: Much of social science is centered around terms like ``ideology'' or ``power'', which generally elude precise definition, and whose contextual meanings are trapped in surrounding language. This paper explores the use of large language models (LLMs) to flexibly navigate the conceptual clutter inherent to social scientific measurement tasks. We rely on LLMs' remarkable linguistic fluency to elicit ideological scales of both legislators and text, which accord closely to established methods and our own judgement. A …

abstract age application arxiv cs.cl definition language language models large language large language models llms measurement paper power scaling science social social science terms type