Improving Long Text Understanding with Knowledge Distilled from Summarization Model

arXiv:2405.04955v1 Announce Type: new
Abstract: Long text understanding is important yet challenging for natural language processing. A long article or document usually contains many redundant words that are not pertinent to its gist and sometimes can be regarded as noise. With recent advances of abstractive summarization, we propose our \emph{Gist Detector} to leverage the gist detection ability of a summarization model and integrate the extracted gist into downstream models to enhance their long text understanding ability. Specifically, Gist Detector first …

abstract advances article arxiv cs.ai cs.cl document gist improving knowledge language language processing natural natural language natural language processing noise processing summarization text text understanding type understanding words