CodecLM: Aligning Language Models with Tailored Synthetic Data

arXiv:2404.05875v1 Announce Type: cross
Abstract: Instruction tuning has emerged as the key in aligning large language models (LLMs) with specific task instructions, thereby mitigating the discrepancy between the next-token prediction objective and users' actual goals. To reduce the labor and time cost to collect or annotate data by humans, researchers start to explore the use of LLMs to generate instruction-aligned synthetic data. Recent works focus on generating diverse instructions and applying LLM to increase instruction complexity, often neglecting downstream use …

abstract arxiv cost cs.ai cs.cl cs.lg data humans key labor language language models large language large language models llms next prediction reduce researchers synthetic synthetic data the key token type