InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions

arXiv:2403.11435v1 Announce Type: new
Abstract: Instruction tuning effectively optimizes Large Language Models (LLMs) for downstream tasks. Due to the changing environment in real-life applications, LLMs necessitate continual task-specific adaptation without catastrophic forgetting. Considering the heavy computational cost, replay-based Continual Learning (CL) methods are the simplest and most widely used for LLMs to address the forgetting issue. However, traditional replay-based methods do not fully utilize instructions to customize the replay strategy. In this work, we propose a novel paradigm called Instruction-based …

abstract applications arxiv catastrophic forgetting computational continual cost cs.cl data environment fine-tuning language language models large language large language models life llms paradigm tasks type