Towards Efficient Active Learning in NLP via Pretrained Representations

arXiv:2402.15613v1 Announce Type: new
Abstract: Fine-tuning Large Language Models (LLMs) is now a common approach for text classification in a wide range of applications. When labeled documents are scarce, active learning helps save annotation efforts but requires retraining of massive models on each acquisition iteration. We drastically expedite this process by using pretrained representations of LLMs within the active learning loop and, once the desired amount of labeled data is acquired, fine-tuning that or even a different pretrained LLM on …

abstract acquisition active learning annotation applications arxiv classification cs.cl cs.lg documents fine-tuning iteration language language models large language large language models llms massive nlp process retraining save text text classification type via