Learning Word Embedding with Better Distance Weighting and Window Size Scheduling

arXiv:2404.14631v1 Announce Type: cross
Abstract: Distributed word representation (a.k.a. word embedding) is a key focus in natural language processing (NLP). As a highly successful word embedding model, Word2Vec offers an efficient method for learning distributed word representations on large datasets. However, Word2Vec lacks consideration for distances between center and context words. We propose two novel methods, Learnable Formulated Weights (LFW) and Epoch-based Dynamic Window Size (EDWS), to incorporate distance information into two variants of Word2Vec, the Continuous Bag-of-Words (CBOW) model …

abstract arxiv center cs.cl cs.lg datasets distributed embedding focus however key language language processing large datasets natural natural language natural language processing nlp processing representation scheduling type word word2vec word embedding