all AI news
The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels
May 10, 2024, 4:41 a.m. | Eve Fleisig, Su Lin Blodgett, Dan Klein, Zeerak Talat
cs.LG updates on arXiv.org arxiv.org
Abstract: Longstanding data labeling practices in machine learning involve collecting and aggregating labels from multiple annotators. But what should we do when annotators disagree? Though annotator disagreement has long been seen as a problem to minimize, new perspectivist approaches challenge this assumption by treating disagreement as a valuable source of information. In this position paper, we examine practices and assumptions surrounding the causes of disagreement--some challenged by perspectivist approaches, and some that remain to be addressed--as …
abstract arxiv assumptions challenge challenges cs.cl cs.cy cs.lg data data labeling human labeling labels machine machine learning multiple paradigm practices shift type
More from arxiv.org / cs.LG updates on arXiv.org
Jobs in AI, ML, Big Data
Software Engineer for AI Training Data (School Specific)
@ G2i Inc | Remote
Software Engineer for AI Training Data (Python)
@ G2i Inc | Remote
Software Engineer for AI Training Data (Tier 2)
@ G2i Inc | Remote
Data Engineer
@ Lemon.io | Remote: Europe, LATAM, Canada, UK, Asia, Oceania
Artificial Intelligence – Bioinformatic Expert
@ University of Texas Medical Branch | Galveston, TX
Lead Developer (AI)
@ Cere Network | San Francisco, US