Reformatted Alignment

arXiv:2402.12219v1 Announce Type: cross
Abstract: The quality of finetuning data is crucial for aligning large language models (LLMs) with human values. Current methods to improve data quality are either labor-intensive or prone to factual errors caused by LLM hallucinations. This paper explores elevating the quality of existing instruction data to better align with human values, introducing a simple and effective approach named ReAlign, which reformats the responses of instruction data into a format that better aligns with pre-established criteria and …

abstract alignment arxiv cs.ai cs.cl cs.lg current data data quality errors finetuning hallucinations human labor language language models large language large language models llm llm hallucinations llms paper quality simple type values