Can language models learn analogical reasoning? Investigating training objectives and comparisons to human performance

arXiv:2310.05597v4 Announce Type: replace
Abstract: While analogies are a common way to evaluate word embeddings in NLP, it is also of interest to investigate whether or not analogical reasoning is a task in itself that can be learned. In this paper, we test several ways to learn basic analogical reasoning, specifically focusing on analogies that are more typical of what is used to evaluate analogical reasoning in humans than those in commonly used NLP benchmarks. Our experiments find that models …

abstract arxiv cs.cl embeddings human human performance language language models learn nlp paper performance reasoning test training type while word word embeddings