Language Model Knowledge Distillation for Efficient Question Answering in Spanish

arXiv:2312.04193v2 Announce Type: replace-cross
Abstract: Recent advances in the development of pre-trained Spanish language models has led to significant progress in many Natural Language Processing (NLP) tasks, such as question answering. However, the lack of efficient models imposes a barrier for the adoption of such models in resource-constrained environments. Therefore, smaller distilled models for the Spanish language could be proven to be highly scalable and facilitate their further adoption on a variety of tasks and scenarios. In this work, we …

abstract adoption advances arxiv cs.cl cs.lg development distillation environments however knowledge language language model language models language processing natural natural language natural language processing nlp processing progress question question answering spanish stat.ml tasks type