Leveraging Locality in Abstractive Text Summarization. (arXiv:2205.12476v1 [cs.CL])

Despite the successes of neural attention models for natural language
generation tasks, the quadratic memory complexity of the self-attention module
with respect to the input length hinders their applications in long text
summarization. Instead of designing more efficient attention modules, we
approach this problem by investigating if models with a restricted context can
have competitive performance compared with the memory-efficient attention
models that maintain a global context by treating the input as an entire
sequence. Our model is applied to …

arxiv summarization text text summarization