Gender Bias in News Summarization: Measures, Pitfalls and Corpora

arXiv:2309.08047v2 Announce Type: replace
Abstract: Summarization is an important application of large language models (LLMs). Most previous evaluation of summarization models has focused on their performance in content selection, faithfulness, grammaticality and coherence. However, it is well known that LLMs reproduce and reinforce harmful social biases. This raises the question: Do these biases affect model outputs in a relatively constrained setting like summarization?
To help answer this question, we first motivate and introduce a number of definitions for biased behaviours …

abstract application arxiv bias biases cs.cl evaluation gender gender bias language language models large language large language models llms performance question raises reinforce social summarization type