Leveraging Zero-Shot Prompting for Efficient Language Model Distillation

arXiv:2403.15886v1 Announce Type: cross
Abstract: This paper introduces a novel approach for efficiently distilling LLMs into smaller, application-specific models, significantly reducing operational costs and manual labor. Addressing the challenge of deploying computationally intensive LLMs in specific applications or edge devices, this technique utilizes LLMs' reasoning capabilities to generate labels and natural language rationales for unlabeled data. Our approach enhances both finetuning and distillation by employing a multi-task training framework where student models mimic these rationales alongside teacher predictions. Key contributions …

abstract application applications arxiv capabilities challenge costs cs.ai cs.cl cs.lg devices distillation edge edge devices generate labels labor language language model llms manual labor model distillation natural natural language novel paper prompting reasoning type zero-shot