D3CODE: Disentangling Disagreements in Data across Cultures on Offensiveness Detection and Evaluation

arXiv:2404.10857v1 Announce Type: new
Abstract: While human annotations play a crucial role in language technologies, annotator subjectivity has long been overlooked in data collection. Recent studies that have critically examined this issue are often situated in the Western context, and solely document differences across age, gender, or racial groups. As a result, NLP research on subjectivity have overlooked the fact that individuals within demographic groups may hold diverse values, which can influence their perceptions beyond their group norms. To effectively …

abstract age annotations arxiv collection context cs.cl data data collection detection differences document evaluation gender human issue language racial role studies technologies type