Multi-dimensional data refining strategy for effective fine-tuning LLMs. (arXiv:2311.01049v1 [cs.CL])

Data is a cornerstone for fine-tuning large language models, yet acquiring
suitable data remains challenging. Challenges encompassed data scarcity,
linguistic diversity, and domain-specific content. This paper presents lessons
learned while crawling and refining data tailored for fine-tuning Vietnamese
language models. Crafting such a dataset, while accounting for linguistic
intricacies and striking a balance between inclusivity and accuracy, demands
meticulous planning. Our paper presents a multidimensional strategy including
leveraging existing datasets in the English language and developing customized
data-crawling scripts with …

accounting arxiv challenges crawling data dataset diversity domain fine-tuning language language models large language large language models lessons learned llms paper strategy