STAR: Constraint LoRA with Dynamic Active Learning for Data-Efficient Fine-Tuning of Large Language Models

arXiv:2403.01165v1 Announce Type: new
Abstract: Though Large Language Models (LLMs) have demonstrated the powerful capabilities of few-shot learning through prompting methods, supervised training is still necessary for complex reasoning tasks. Because of their extensive parameters and memory consumption, both Parameter-Efficient Fine-Tuning (PEFT) methods and Memory-Efficient Fine-Tuning methods have been proposed for LLMs. Nevertheless, the issue of large annotated data consumption, the aim of Data-Efficient Fine-Tuning, remains unexplored. One obvious way is to combine the PEFT method with active learning. However, …

abstract active learning arxiv capabilities consumption cs.ai cs.cl data dynamic few-shot few-shot learning fine-tuning language language models large language large language models llms lora memory memory consumption parameters peft prompting reasoning star supervised training tasks through training type