LlamBERT: Large-scale low-cost data annotation in NLP

arXiv:2403.15938v1 Announce Type: cross
Abstract: Large Language Models (LLMs), such as GPT-4 and Llama 2, show remarkable proficiency in a wide range of natural language processing (NLP) tasks. Despite their effectiveness, the high costs associated with their use pose a challenge. We present LlamBERT, a hybrid approach that leverages LLMs to annotate a small subset of large, unlabeled databases and uses the results for fine-tuning transformer encoders like BERT and RoBERTa. This strategy is evaluated on two diverse datasets: the …

abstract annotation arxiv challenge cost costs cs.ai cs.cl cs.lg data data annotation gpt gpt-4 hybrid hybrid approach language language models language processing large language large language models llama llama 2 llms low natural natural language natural language processing nlp processing scale show tasks type