Investigating Text Shortening Strategy in BERT: Truncation vs Summarization

arXiv:2403.12799v1 Announce Type: new
Abstract: The parallelism of Transformer-based models comes at the cost of their input max-length. Some studies proposed methods to overcome this limitation, but none of them reported the effectiveness of summarization as an alternative. In this study, we investigate the performance of document truncation and summarization in text classification tasks. Each of the two was investigated with several variations. This study also investigated how close their performances are to the performance of full-text. We used a …

arxiv bert cs.ai cs.cl strategy summarization text type