Extremal GloVe: Theoretically Accurate Distributed Word Embedding by Tail Inference. (arXiv:2204.13009v1 [cs.CL])

Distributed word embeddings such as Word2Vec and GloVe have been widely
adopted in industrial context settings. Major technical applications of GloVe
include recommender systems and natural language processing. The fundamental
theory behind GloVe relies on the selection of a weighting function in the
weighted least squres formulation that computes the powered ratio of word
occurrence count and the maximum word count in the corpus. However, the initial
formulation of GloVe is not theoretically sound in two aspects, namely the
selection …

arxiv distributed embedding inference