Transformers Well Explained: Masking

This is the second part of a four-article series that explains transforms. Each article is associated with a hands-on notebook. In the previous article, we explained word embeddings in detail and then trained an embedding with the task of predicting the third word of a trigram given two previous words. In this article, we will do the same but with the task of figuring out masked words.

arabic article embedding embeddings explained masking naturallanguageprocessing notebook part series transformers will word word embeddings words