COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning

arXiv:2403.18058v1 Announce Type: new
Abstract: Recently, there have been significant advancements in large language models (LLMs), particularly focused on the English language. These advancements have enabled these LLMs to understand and execute complex instructions with unprecedented accuracy and fluency. However, despite these advancements, there remains a noticeable gap in the development of Chinese instruction tuning. The unique linguistic features and cultural depth of the Chinese language pose challenges for instruction tuning tasks. Existing datasets are either derived from English-centric LLMs …

arxiv chinese cs.ai cs.cl fine-tuning quality type