Protected group bias and stereotypes in Large Language Models

arXiv:2403.14727v1 Announce Type: cross
Abstract: As modern Large Language Models (LLMs) shatter many state-of-the-art benchmarks in a variety of domains, this paper investigates their behavior in the domains of ethics and fairness, focusing on protected group bias. We conduct a two-part study: first, we solicit sentence continuations describing the occupations of individuals from different protected groups, including gender, sexuality, religion, and race. Second, we have the model generate stories about individuals who hold different types of occupations. We collect >10k …

abstract art arxiv behavior benchmarks bias cs.cl cs.cy cs.lg domains ethics fairness language language models large language large language models llms modern paper part state stereotypes study type