Instruction Tuning with Human Curriculum

In building instruction-tuned large language models (LLMs), the importance of a deep understanding of human knowledge can be often overlooked by the importance of instruction diversification. This research proposes a novel approach to instruction tuning by integrating a structured cognitive learning methodology that takes inspiration from the systematic progression and cognitively stimulating nature of human education through two key steps. First, our synthetic instruction data generation pipeline, designed with some references to human educational frameworks, is enriched with meta-data detailing …

building cognitive cs.ai cs.cl cs.lg curriculum diversification human importance inspiration instruction-tuned knowledge language language models large language large language models llms methodology nature novel research understanding