Great Truths are Always Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models. (arXiv:2205.01841v1 [cs.CL])

Commonsense reasoning in natural language is a desired ability of artificial
intelligent systems. For solving complex commonsense reasoning tasks, a typical
solution is to enhance pre-trained language models~(PTMs) with a
knowledge-aware graph neural network~(GNN) encoder that models a commonsense
knowledge graph~(CSKG). Despite the effectiveness, these approaches are built
on heavy architectures, and can't clearly explain how external knowledge
resources improve the reasoning capacity of PTMs. Considering this issue, we
conduct a deep empirical analysis, and find that it is indeed …

arxiv capacity encoder knowledge pre-trained models reasoning