Assessing Generalization for Subpopulation Representative Modeling via In-Context Learning

This study evaluates the ability of Large Language Model (LLM)-based Subpopulation Representative Models (SRMs) to generalize from empirical data, utilizing in-context learning with data from the 2016 and 2020 American National Election Studies. We explore generalization across response variables and demographic subgroups. While conditioning with empirical data improves performance on the whole, the benefit of in-context learning varies considerably across demographics, sometimes hurting performance for one demographic while helping performance for others. The inequitable benefits of in-context learning for SRM …

context cs.ai cs.cl cs.cy cs.lg data election explore in-context learning language language model large language large language model llm modeling performance studies study subgroups variables via