CRAFT: Extracting and Tuning Cultural Instructions from the Wild

arXiv:2405.03138v1 Announce Type: new
Abstract: Large language models (LLMs) have rapidly evolved as the foundation of various natural language processing (NLP) applications. Despite their wide use cases, their understanding of culturally-related concepts and reasoning remains limited. Meantime, there is a significant need to enhance these models' cultural reasoning capabilities, especially concerning underrepresented regions. This paper introduces a novel pipeline for extracting high-quality, culturally-related instruction tuning datasets from vast unstructured corpora. We utilize a self-instruction generation pipeline to identify cultural concepts …

abstract applications arxiv capabilities cases concepts craft cs.cl foundation language language models language processing large language large language models llms natural natural language natural language processing nlp processing reasoning type understanding use cases