Semi-Instruct: Bridging Natural-Instruct and Self-Instruct for Code Large Language Models

arXiv:2403.00338v1 Announce Type: new
Abstract: Instruction tuning plays a pivotal role in Code Large Language Models (Code LLMs) for the task of program synthesis. Presently, two dominant paradigms for collecting tuning data are natural-instruct (human-written) and self-instruct (automatically generated). Natural-instruct includes diverse and correct codes but lacks instruction-code pairs, and exists improper code formats like nested single-line codes. In contrast, self-instruct automatically generates proper paired data. However, it suffers from low diversity due to generating duplicates and cannot ensure the …

abstract arxiv code code llms cs.cl data diverse generated human language language models large language large language models llms natural pivotal role synthesis type