A Multi-Aspect Framework for Counter Narrative Evaluation using Large Language Models

arXiv:2402.11676v1 Announce Type: new
Abstract: Counter narratives - informed responses to hate speech contexts designed to refute hateful claims and de-escalate encounters - have emerged as an effective hate speech intervention strategy. While previous work has proposed automatic counter narrative generation methods to aid manual interventions, the evaluation of these approaches remains underdeveloped. Previous automatic metrics for counter narrative evaluation lack alignment with human judgment as they rely on superficial reference comparisons instead of incorporating key aspects of counter narrative …

abstract arxiv cs.ai cs.cl evaluation framework hate speech language language models large language large language models narrative responses speech strategy type work