Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning

arXiv:2402.10110v1 Announce Type: cross
Abstract: Instruction tuning is critical to large language models (LLMs) for achieving better instruction following and task adaptation capabilities but its success heavily relies on the training data quality. Many recent methods focus on improving the data quality but often overlook the compatibility of the data with the student model being finetuned. This paper introduces Selective Reflection-Tuning, a novel paradigm that synergizes a teacher LLM's reflection and introspection for improving existing data quality with the data …

arxiv cs.ai cs.cl cs.lg data llm recycling type