A Combination of BERT and Transformer for Vietnamese Spelling Correction

arXiv:2405.02573v1 Announce Type: new
Abstract: Recently, many studies have shown the efficiency of using Bidirectional Encoder Representations from Transformers (BERT) in various Natural Language Processing (NLP) tasks. Specifically, English spelling correction task that uses Encoder-Decoder architecture and takes advantage of BERT has achieved state-of-the-art result. However, to our knowledge, there is no implementation in Vietnamese yet. Therefore, in this study, a combination of Transformer architecture (state-of-the-art for Encoder-Decoder model) and BERT was proposed to deal with Vietnamese spelling correction. The …

abstract architecture art arxiv bert combination cs.cl decoder efficiency encoder encoder-decoder english however knowledge language language processing natural natural language natural language processing nlp processing state studies tasks transformer transformers type