KALM: Knowledge-Aware Integration of Local, Document, and Global Contexts for Long Document Understanding. (arXiv:2210.04105v1 [cs.CL])

With the advent of pre-trained language models (LMs), increasing research
efforts have been focusing on infusing commonsense and domain-specific
knowledge to prepare LMs for downstream tasks. These works attempt to leverage
knowledge graphs, the de facto standard of symbolic knowledge representation,
along with pre-trained LMs. While existing approaches leverage external
knowledge, it remains an open question how to jointly incorporate knowledge
graphs representing varying contexts, from local (e.g., sentence), to
document-level, to global knowledge, to enable knowledge-rich and interpretable
exchange …

arxiv document understanding global integration knowledge understanding