Always Keep your Target in Mind: Studying Semantics and Improving Performance of Neural Lexical Substitution. (arXiv:2206.11815v1 [cs.CL])

Lexical substitution, i.e. generation of plausible words that can replace a
particular target word in a given context, is an extremely powerful technology
that can be used as a backbone of various NLP applications, including word
sense induction and disambiguation, lexical relation extraction, data
augmentation, etc. In this paper, we present a large-scale comparative study of
lexical substitution methods employing both rather old and most recent language
and masked language models (LMs and MLMs), such as context2vec, ELMo, BERT,
RoBERTa, …

arxiv performance semantics studying