Pitfalls of Conversational LLMs on News Debiasing

arXiv:2404.06488v1 Announce Type: new
Abstract: This paper addresses debiasing in news editing and evaluates the effectiveness of conversational Large Language Models in this task. We designed an evaluation checklist tailored to news editors' perspectives, obtained generated texts from three popular conversational models using a subset of a publicly available dataset in media bias, and evaluated the texts according to the designed checklist. Furthermore, we examined the models as evaluator for checking the quality of debiased model outputs. Our findings indicate …

abstract arxiv bias checklist conversational cs.ai cs.cl dataset editing editors evaluation generated language language models large language large language models llms media paper perspectives popular type