Which NLP Task Does NOT Benefit From Pre-trained Language Models?

There is such a long history of pre-trained general language representations models with the massive impact that we take for granted that they are a completely 100% necessary foundation for all NLP tasks. There were two separate step function innovations that pushed the accuracy of all NLP tasks forward: (1) statistical language models like Word2Vec and GloVe and, more recently, (2) neural language models like BERT, ELMo, and recently BLOOM. Inserting pre-trained neural language models at the …

artificial intelligence deep learning language language models machine learning machine translation nlp