Measuring Political Bias in Large Language Models: What Is Said and How It Is Said

arXiv:2403.18932v1 Announce Type: new
Abstract: We propose to measure political bias in LLMs by analyzing both the content and style of their generated content regarding political issues. Existing benchmarks and measures focus on gender and racial biases. However, political bias exists in LLMs and can lead to polarization and other harms in downstream applications. In order to provide transparency to users, we advocate that there should be fine-grained and explainable measures of political biases generated by LLMs. Our proposed measure …

abstract arxiv benchmarks bias biases cs.ai cs.cl focus gender generated however language language models large language large language models llms measuring political racial style type