LLMs in the Loop: Leveraging Large Language Model Annotations for Active Learning in Low-Resource Languages

arXiv:2404.02261v1 Announce Type: cross
Abstract: Low-resource languages face significant barriers in AI development due to limited linguistic resources and expertise for data labeling, rendering them rare and costly. The scarcity of data and the absence of preexisting tools exacerbate these challenges, especially since these languages may not be adequately represented in various NLP datasets. To address this gap, we propose leveraging the potential of LLMs in the active learning loop for data annotation. Initially, we conduct evaluations to assess inter-annotator …

abstract active learning ai development annotations arxiv challenges cs.ai cs.cl cs.ir cs.lg data data labeling development expertise face labeling language language model languages large language large language model llms loop low rendering resources them tools type