Sequence-to-Sequence Spanish Pre-trained Language Models

arXiv:2309.11259v2 Announce Type: replace-cross
Abstract: In recent years, significant advancements in pre-trained language models have driven the creation of numerous non-English language variants, with a particular emphasis on encoder-only and decoder-only architectures. While Spanish language models based on BERT and GPT have demonstrated proficiency in natural language understanding and generation, there remains a noticeable scarcity of encoder-decoder models explicitly designed for sequence-to-sequence tasks, which aim to map input sequences to generate output sequences conditionally. This paper breaks new ground by …

arxiv cs.ai cs.cl cs.lg language language models spanish type