Annotator-Centric Active Learning for Subjective NLP Tasks

arXiv:2404.15720v1 Announce Type: new
Abstract: To accurately capture the variability in human judgments for subjective NLP tasks, incorporating a wide range of perspectives in the annotation process is crucial. Active Learning (AL) addresses the high costs of collecting human annotations by strategically annotating the most informative samples. We introduce Annotator-Centric Active Learning (ACAL), which incorporates an annotator selection strategy following data sampling. Our objective is two-fold: (1) to efficiently approximate the full diversity of human judgments, and to assess model …

abstract active learning annotation annotations arxiv costs cs.cl human nlp perspectives process samples tasks type