On the Information Content of Predictions in Word Analogy Tests. (arXiv:2210.09972v1 [cs.CL])

An approach is proposed to quantify, in bits of information, the actual
relevance of analogies in analogy tests. The main component of this approach is
a softaccuracy estimator that also yields entropy estimates with compensated
biases. Experimental results obtained with pre-trained GloVe 300-D vectors and
two public analogy test sets show that proximity hints are much more relevant
than analogies in analogy tests, from an information content perspective.
Accordingly, a simple word embedding model is used to predict that analogies …

analogy arxiv information predictions tests