NeuroComparatives: Neuro-Symbolic Distillation of Comparative Knowledge

arXiv:2305.04978v3 Announce Type: replace
Abstract: Comparative knowledge (e.g., steel is stronger and heavier than styrofoam) is an essential component of our world knowledge, yet understudied in prior literature. In this paper, we harvest the dramatic improvements in knowledge capabilities of language models into a large-scale comparative knowledge base. While the ease of acquisition of such comparative knowledge is much higher from extreme-scale models like GPT-4, compared to their considerably smaller and weaker counterparts such as GPT-2, not even the most …

abstract acquisition arxiv capabilities cs.cl distillation improvements knowledge knowledge base language language models literature neuro paper prior scale type world